Talencia Consulting - Azure Data Engineer

Talencia is looking for an Azure Data Engineer for one of its clients Location: Wallonia, Belgium Work model: Hybrid - 3 days onsite Start: ASAP Seniority: Medior-Senior Our client is currently leading a major data platform modernization program, with the goal of migrating a historical data environment to a modern Databricks Lakehouse architecture on Microsoft Azure. The existing environment notably includes SQL views, a templating system based on Jinja, and orchestration via Azure Data Factory. The transformation will introduce modern Databricks features, including Lakeflow, Unity Catalog, PySpark, and automated CI/CD practices. As an Azure Data Engineer, you will join an existing Data Engineering team and contribute directly to the migration and industrialization of the new platform. You will be responsible for the design, development, and maintenance of data pipelines, migration of existing data marts, implementation of quality and testing frameworks, and validation of result equivalence between the old and new platforms. Responsibilities Data Engineering & ETL: Develop and industrialize Databricks Lakeflow / Spark Declarative Pipelines Build ingestion pipelines using Auto Loader and Copy Into, among others Implement CDC / SCD Type 2 mechanisms for change tracking Develop Silver and Gold transformation layers using SQL, Python, and PySpark Migrate existing SQL views and Jinja templated tables to new Databricks patterns Perform complete data mart migrations, including pipelines, tests, documentation, and reporting models Work with Azure Data Factory for orchestration and integration with Databricks Jobs Data Platform & Governance: Work with Azure Data Lake Storage, Azure Blob Storage, and Databricks Design and maintain Unity Catalog structures, including catalogs, schemas, and access controls Apply security and governance best practices on Dev, UAT, and Production environments Design scalable, performant, and cost-optimized data processing solutions Implement monitoring and cost tracking at the level of different Databricks & Big Data data marts Develop data processing solutions with Databricks and Apache Spark Work intensively with PySpark and SQL on large data volumes Contribute to the migration of historical data workloads to a modern Lakehouse architecture Use AI-assisted development tools where relevant to accelerate migration activities, while ensuring adherence to engineering standards Participate in non-regression testing to ensure migrated data produces equivalent results to the historical environment Data Quality & Analytics: Contribute to the implementation of a structured Data Quality framework Define and implement reusable validation rules and quality controls Ensure monitoring of data quality, freshness, and pipeline performance Collaborate with business stakeholders to define quality rules specific to different functional areas Build and expose data marts via Power BI semantic models, notably with Tmdl and DAX Contribute to the implementation of conversational analytics solutions via Databricks Genie Required Skills and Experience: Proven professional experience as a Data Engineer Solid hands-on experience with Databricks and modern Lakehouse architectures Excellent command of SQL and Data Modelling Very good Python and PySpark programming skills Experience with Databricks Automation Bundles Solid experience in designing Data Warehouses / Data Marts using Star Schema Experience in legacy data platform migration projects Ability to understand, replicate, and validate existing business logic before migrating it to a new platform Experience in environments with established engineering standards and development practices Fluent French and professional English Excellent analytical and problem-solving skills Good communication skills and ability to explain technical concepts to non-technical stakeholders Ability to work effectively within an existing team Appreciated Skills: Experience with Azure Data Factory Experience with Azure DevOps and CI/CD Knowledge in Data Analytics and/or Machine Learning Experience with Data Vault, 3NF, or Merise modeling approaches Knowledge of Data Quality frameworks such as Great Expectations Experience with automated non-regression testing for data pipelines Experience with AI-assisted development tools while adhering to structured engineering standards Experience with Power BI, Tmdl, and DAX Knowledge of Databricks Genie Your Mission: You will join a transformation program whose objective is to gradually migrate existing data marts from a historical environment to a modern Azure Databricks Lakehouse. The role combines Data Engineering, migration, Data Modelling, Data Quality, testing, and cloud data platform development. It is particularly suited for an experienced Data Engineer wishing to work on a complex and structuring modernization project.