Data Engineer

We are seeking a skilled Data Engineer with strong Databricks experience to join our growing data team. In this role, you will design, build, and optimize large-scale data pipelines, ensure high data quality, and support analytics a initiatives.

Responsibilities:

• Develop and maintain scalable ETL/ELT pipelines using Databricks (Spark, Delta Lake).
• Build and optimize data lakehouse architectures for batch and streaming workloads.
• Collaborate with data analysts, data scientists, and product teams to deliver reliable datasets for reporting and ML models.
• Implement best practices around data quality, observability, governance, and documentation.
• Manage and optimize Databricks clusters, workflows, and job performance.
• Integrate data from various sources (APIs, databases, cloud storage, SaaS platforms).
• Support infrastructure using Azure/AWS and CI/CD for data pipelines.
• Ensure compliance with data privacy and security standards.
  
  

Required Qualifications:

• 4+ years of experience as a Data Engineer.
• Hands-on experience with Databricks, Apache Spark, and Delta Lake.
• Strong SQL and Python skills.
• Experience building ETL/ELT pipelines at scale.
• Familiarity with cloud platforms (Azure, AWS, or GCP).
• Experience with Azure Databricks or AWS Databricks.
• Understanding of data modeling, distributed computing, and data warehousing concepts.
• Experience with data orchestration tools (Airflow, dbt, ADF, Prefect, etc.).
• Experience with version control (Git) and CI/CD tools.

About INGRITY:

We are a fast-growing, progressive Sydney-based data & analytics company. We service many large corporates and SMBs in the financial services sector, to derive value from data. At Ingrity, we are committed to building a diverse and inclusive workforce to deliver value to customers. As a value-driven organisation, we nurture and support our people by having a laser-sharp focus on skill and talent development, collaboration, and flexibility.