(Senior) Machine Learning / AI Systems Auditor (m/w/d)

With a team that makes a difference every day, we support the employees of the companies in the Schwarz Group. Our working day is characterized by varied tasks as well as diverse areas and endless opportunities. What our 4,000 employees share is their passion and commitment to their daily work. #TeamSCOS

We are ready to take the next step - are you in? Where Excellence Works.

Our specialized AI audit team combines deep tech with strategic risk assessment: we not only examine complex AI development processes, generative multi-agent systems and data pipelines, but also develop our own state-of-the-art AI tools for the audit of tomorrow.

• Technical deep dives & system audits: You carry out well-founded technical audits of generative AI solutions, LLM pipelines, autonomous multi-agent systems and classic data science products within the companies of the Schwarz Group.
• Code & architecture analysis: You examine source code, system architectures and prompt strategies thoroughly. Your focus is on security, performance and system robustness (including avoiding hallucinations, jailbreaks, bias and data leakage).
• In-house development of evaluation frameworks: You design, program and optimize your own AI-supported analysis tools and evaluation frameworks for audit processes using state-of-the-art data science methods.
• Process intelligence & workflow audits: You analyze complex business processes and AI-driven workflows using BI and process mining tools (e.g. Celonis) to uncover anomalies and efficiency potential.
• Technical sparring & international assignments: You support audit projects in Germany and abroad and advise management and developer teams as a technical sparring partner on an equal footing.

• Academic background & professional experience: Successfully completed studies in (business) computer science, data science, mathematics or a comparable technical qualification. You bring at least 3 years of solid professional experience in software development, data engineering or technical IT audit.
• Tech stack (Python & Spark): Very good knowledge of Python and of distributed data systems with Spark. You have solid practical experience with the LLM ecosystem (e.g. LangChain, LlamaIndex, vector databases) and common machine learning frameworks (e.g. Scikit-Learn, PyTorch).
• Software engineering & LLMOps: You master best practices of the software development lifecycle, continuous delivery (CI/CD), distributed cloud systems as well as modern MLOps and LLMOps concepts, including the testing of non-deterministic AI.
• Governance & risk assessment: Solid knowledge of audit methods, risk assessments (AI security, data privacy) and governance frameworks (e.g. EU AI Act) is desirable. You have a trained eye for architectural and security-relevant vulnerabilities.
• Language skills & working style: Fluent German, both spoken and written, is mandatory for audit and advisory reports; good English skills complete your profile for international exchange. You work analytically, proactively and solution-oriented.