Data Engineer

Our client, one of Asia-Pacific’s leading organizations is looking for:

Responsibilities:

• Design, develop and maintain efficient and reliable ETL processes and data pipelines for the ingestion, transformation, and integration of data from multiple source agencies
• Develop and maintain modular, production-quality code for statistical production processes, including optimising legacy processes while preserving business logic and reconciling outputs against existing results
• Design and maintain database schemas, data models and storage structures to support efficient ingestion, validation, transformation and retrieval of large datasets
• Build reusable data engineering components such as common libraries, configuration frameworks, logging standards, and utilities to streamline production workflows
• Implement and maintain CI/CD pipelines for data engineering workflows, including automated testing and deployment
• Establish version control structures using GitLab, set up code review frameworks, and produce technical documentation
• Monitor and troubleshoot pipelines, perform root‑cause analysis and continuously improve operational reliability and system performance for large-scale datasets

Requirements:

• Degree with minimum 4-7 years of professional experience in data engineering, data platform development, or a related software engineering role
• Strong proficiency in Python and SQL, including experience in complex data transformations, analysis, and quality monitoring.
• Hands‑on experience with dataframe processing libraries (e.g. Polars, pandas) for high‑performance execution and memory‑efficient processing of large datasets
• Experience with database management, data warehousing and developing ETL processes for data ingestion and integration
• Experience with version control using Git, including branching strategies, merge requests, code review, and CI/CD pipelines for automated testing and deployment
• Good knowledge of Linux/Unix environments, shell scripting and command-line tools for data processing
• Experience with version control using Git, including branching strategies, merge requests, code review, and CI/CD pipelines for automated testing and deployment
• Experience with cloud platforms and orchestration/CI‑CD tooling is preferred

Only shortlisted candidates will be notified.