Goldman Tech Resourcing
3 weeks ago
Python / Spark / AI Developer
Sign up to save this job, get alerts, and apply with an optimized CV.
Company information
- Company
- Goldman Tech Resourcing
- Location
- South Africa, Western Cape South Africa
- Posted
- 3 weeks ago
Job description
Our client is seeking an experienced Python/Spark/AI Developer to help modernise data platforms by replatforming legacy T-SQL workloads onto Spark and Delta Lake pipelines. The successful candidate will work across data engineering, API development and AI-enabled capabilities, with a strong emphasis on type-safe, well-tested and specification-driven Python development.
Skills Required
· 4+ years of professional Python development experience.
· 2+ years of production experience building Spark or comparable distributed data pipelines.
· Strong Python 3.12 knowledge, including type hints, Pydantic, ABC-based provider patterns and modern packaging tools such as uv or Poetry.
· Experience with Click or similar CLI frameworks.
· Strong Apache Spark/PySpark experience, including DataFrame API, Spark SQL, partitioning, performance tuning and driver/executor architecture.
· Understanding of Spark Connect.
· Production experience with Delta Lake or equivalent lakehouse formats such as Iceberg or Hudi.
· Strong SQL skills and the ability to faithfully translate legacy T-SQL and stored-procedure logic into Spark SQL.
· Strong understanding of hashing, surrogate keys and set-based deduplication.
· Hands-on pytest experience, including fixtures, markers and unit, integration and end-to-end testing.
· Experience developing REST APIs using FastAPI or an equivalent framework.
· Practical experience with Docker and Compose-based development environments.
· Strong Git and CI practices, including PR workflows, linting, type checking and automated test gates.
Should you meet the requirements for this position, please email your CV to [email protected]. You can also contact us on XXX-XXXX or alternatively you can visit our website www.stand-outstaffing.co.za
Should you not hear from us within 3 days, please consider your application unsuccessful.
Required skills
- docker
- api development
- automated testing
- ci/cd
- sql
- ai
- python
- git
- unit testing
- rest api
- spark
- developer
- delta lake
- data engineering
- pyspark
- fastapi
- pytest
- poetry
- t-sql
- iceberg
- integration testing
- uv
- hudi
- apache spark
- ci
- stored procedures
- spark sql
- end-to-end testing
- docker compose
- lakehouse
- pydantic
- linting
- deduplication
- click
- python 3.12
- type hints
- dataframe api
- spark connect
- type checking
Interested in this position?
Create your free account and tailor your CV to match this job.