Senior Data Engineer (Azure Databricks)

For our client, an insurance company in the greater Frankfurt am Main area, we are looking for two experienced Data Engineers as part of the setup of a new dispositional data platform (Lakehouse) based on Microsoft Azure and Databricks. You will build the data pipelines from the source systems to the core model, taking technical lead in the implementation. You will work in tandem with client employees and actively share your Databricks know-how.

Tasks

• Connection of source systems and external service providers (Delta deliveries, files, APIs) as well as a CRM system, including automated loading processes

• Development of ETL/ELT pipelines with Delta processing, error handling, and traceable restart procedures

• Implementation of duplicate cleansing and merging of customer and broker data

• Implementation of the core data model with historization along the Medallion architecture (Databricks, PySpark, SQL)

• Implementation of data quality rules, validation routines, monitoring, and alerting

• Provision of exports and feedback loops to operational systems (e.g., campaign selections for CRM)

• Involvement in development pipelines and CI/CD, roles and permissions, Data Catalog, and Lineage

• Documentation, runbooks, and knowledge transfer in tandem with the client team

Requirements

• Several years of experience as a Data Engineer, ideally in building data warehouse or lakehouse solutions

• Very good knowledge of Databricks, PySpark/Spark, SQL, and Delta Lake

• Practical experience with Databricks Workflows or Azure Data Factory and other Azure data services

• Experience in data modeling and historization (Data Vault and/or dimensional modeling)

• Knowledge of Git, CI/CD, and automated testing of data pipelines

• Experience with AI-supported development (e.g., coding assistants, agentic workflows) is an advantage

• Experience with source systems and data in the insurance sector is an advantage

• Enjoyment of knowledge transfer and collaboration in a mixed team

• Databricks or Azure Data Engineer certification is an advantage

• Fluent German and good English skills

General Framework Conditions

• Workload: 100%

• Start: Beginning of November 2026

• Duration: until March 31, 2027, with an option for extension (follow-up phases planned until the end of 2028)

• Work location: Greater Frankfurt am Main area (Germany)

• Onsite/Remote: High onsite presence in the first 2–3 months (3–4 days per week), then approx. 2 days onsite, the rest remote

• Client interview: Availability for interviews with the client at the end of October 2026

Find Jobs in Germany on Arbeitnow