Data Engineer (Spark, SQL)

We are looking for a proficient Data Engineer with strong skills in Spark and SQL to join Unison Group. The successful candidate will be responsible for designing, building, and managing data pipelines that facilitate real-time data processing. This role involves optimizing data flow and enhancing data quality to support the analytical needs of various teams within the organization. Key Responsibilities: Develop and maintain robust data pipelines using Apache Spark to process large datasets effectively; Design and implement scalable data architectures to meet business requirements; Collaborate with cross-functional teams to integrate data pipelines into existing systems; Ensure data quality, integrity, and security by implementing data validation, error handling, and logging mechanisms; Monitor and troubleshoot data pipeline performance issues; Develop and maintain technical documentation for data pipelines; Participate in code reviews and contribute to the development of new features. This role requires strong knowledge of SQL and Spark programming languages, as well as experience with big data processing technologies such as Hadoop and Hive. The ideal candidate will have a Bachelor's degree in Computer Science or related field, and at least 3 years of experience working as a Data Engineer or similar role.