For certain occupations (e.g. lawyer, chartered accountant, teacher, physiotherapist, doctor, nurse, midwife, community pharmacist, architect) you need to prove you have the required qualifications. Whether your qualifications obtained abroad entitle you to pursue the occupation of your choice in Germany is something you need to check with the agency responsible for recognizing qualifications in the relevant occupation.
Since April 2012 immigrants are legally entitled to an official ruling on whether their occupational qualifications obtained abroad are deemed equivalent to the relevant German qualification.
More information on the recognition of occupational qualifications is available on the following websites:
Recognition in Germany
Federal Ministry of Education and Research