Devapo
2 months ago
Data Engineer Mid
Sign up to save this job, get alerts, and apply with an optimized CV.
Company information
- Company
- Devapo
- Location
- Polska, mazowieckie, Warszawa Poland
- Posted
- 2 months ago
Job description
Greenfield project in the medical sector — you are building ETL/ELT pipelines powering a data warehouse from scratch. About the Project You will join the Devops team implementing a greenfield project: designing and implementing from scratch a comprehensive Business Intelligence Platform for an organization in the medical sector. You will have a real impact on the shape of the solution — from the data flow architecture to the final dashboards. Data warehouse PostgreSQL 16 in Medallion architecture (Raw / Silver / Gold) with domain division in the spirit of Data Mesh (domains: clinical, financial, HR, marketing, operational), process orchestration in Apache Airflow, transformations in dbt, data quality control in Great Expectations, integration of over a dozen source systems (HIS, call center, CRM, HR, document workflow, Google / Facebook / LinkedIn advertising platforms) via REST API, SOAP, files and database connectors, reporting layer in Microsoft Power BI Service with On-premises Data Gateway, RLS/OLS and SSO via Microsoft Entra ID, entire on-premises in Docker containers, version control in Git, monitoring Prometheus + Grafana, high security standards: medical data, full compliance with GDPR and NIS-2. Scope of Tasks Building ETL/ELT processes powering the data warehouse (full load and incremental load, watermarks, CDC), implementing connectors to source systems: REST API (OAuth 2.0 / JWT), SOAP, files (CSV/XLSX/XML/JSON, SFTP), database connections, creating and maintaining DAGs in Apache Airflow (schedules, sensors, retries, alerting), building transformation models in dbt (Raw → Silver → Gold layers) along with tests, implementing data quality rules in Great Expectations (completeness, integrity, business rules, schema change detection), working in an on-premises environment: Docker, Git, CI/CD, monitoring Prometheus + Grafana. Requirements Minimum 3 years of experience as a Data Engineer or in a related role, practical knowledge of: SQL, Python, Apache Airflow, dbt, PostgreSQL, data modeling, design and construction of ETL/ELT processes, integration using REST API, Git, CI/CD, Docker. Additional advantages will include knowledge of Apache Hop or another visual ETL pipeline design tool (e.g., Pentaho DI, Talend, NiFi), Great Expectations or another data quality control framework, Kong API Gateway, OAuth 2.0 / JWT authentication mechanisms in API integrations, Medallion architecture and Data Mesh concepts, Prometheus / Grafana. Experience with Power BI, experience with sensitive data or in the healthcare sector (GDPR) is welcome. We Offer Duties Requirements
Required skills
- docker
- ci/cd
- sql
- etl
- data modeling
- python
- git
- rest api
- gdpr
- data engineer
- postgresql
- prometheus
- grafana
- dbt
- data warehouse
- healthcare sector
- oauth 2.0
- sso
- jwt
- soap
- talend
- data mesh
- apache airflow
- sftp
- microsoft entra id
- elt
- nifi
- greenfield
- great expectations
- rls
- medallion architecture
- kong api gateway
- nis-2
- apache hop
- microsoft power bi service
- ols
- on-premises data gateway
- pentaho di
Interested in this position?
Create your free account and tailor your CV to match this job.