Specialist in Data Engineering - Affirmative for people with disabilities (PCD)

Job description

Have you ever thought about elevating the data platform and MLOps to the level of companies like Uber, Netflix, and Airbnb? If you're driven by innovation and enjoy working closely with all areas of the company, come join our team!

You will be responsible for:

• Development of a data platform for consumption from various sources, batch, and real-time.
• Development of APIs for access and consumption of data (data products).
• Facilitating dissemination of knowledge on best practices for using data (data contracts, federated governance, and platform engineering).
• Ensuring that the architecture meets technical requirements and needs of the execution team, working together with IT teams (Architecture, Security, Infrastructure) and business areas.
• Autonomy and initiative to act in problem-solving.

What will your daily routine be like?

• Ensure deliveries from the squad on projects agreed upon with client areas.
• Interact with business units, third parties, suppliers, and/or service providers to ensure a thorough understanding and implementation of data and architecture.
• Participate in data-driven discussions within the team and between business areas, supporting important decisions.
• Maintain interface with technology and product teams to define best practices for storing and consuming data, as well as suggesting, testing, and tracking the implementation of new technologies.

Hard skills

• Spark (Batch & Streaming)
• Proficiency in programming languages (Python)
• Pipeline orchestration (Airflow);
• AWS Data Stack (DynamoDB, s3, Lambda, EMR, MSK, Glue, Athena, Lake Formation, RDS)
• Data lake/Lakehouse (iceberg/delta)
• Data architecture
• Best practices in software engineering
• Automated testing
• Knowledge of CI/CD

Soft Skills

• Willingness to learn
• Teamwork / Collaborative Profile;
• Good communication and interpersonal relationships;