Python Spark AI Developer (Data Platforms)
Goldman Tech Resourcing · Gauteng, KwaZulu Natal, Western Cape
Overview
A great opportunity for a skilled Python and Spark developer to work on modernising data platforms by migrating legacy T-SQL to Spark and Delta Lake pipelines. This role is based in Gauteng, KwaZulu Natal, or Western Cape and offers exposure to AI enabled capabilities and API development. If you have 4 or more years of Python experience and a passion for clean, well tested code, this could be your next big move.
About the role
Our client is looking for an experienced Python, Spark and AI Developer to help modernise their data platforms. You will focus on replatforming legacy T-SQL workloads onto Spark and Delta Lake pipelines while working across data engineering, API development, and AI enabled features.
The role places strong emphasis on type safe, well tested, and specification driven Python development. You will be part of a team that values clean code, automated testing, and modern development practices.
To apply, please email your CV to it.vacancy@stand-outstaffing.co.za. You can also call 031 350 4019 or visit www.stand-outstaffing.co.za. If you have not heard from us within 3 days, please consider your application unsuccessful.
Requirements
- 4 or more years of professional Python development experience
- 2 or more years of production experience building Spark or similar distributed data pipelines
- Strong Python 3.12 knowledge, including type hints, Pydantic, ABC based provider patterns, and modern packaging tools such as uv or Poetry
- Experience with Click or similar CLI frameworks
- Strong Apache Spark or PySpark experience, including DataFrame API, Spark SQL, partitioning, performance tuning, and driver or executor architecture
- Understanding of Spark Connect
- Production experience with Delta Lake or equivalent lakehouse formats such as Iceberg or Hudi
- Strong SQL skills and the ability to translate legacy T-SQL and stored procedure logic into Spark SQL
- Strong understanding of hashing, surrogate keys, and set based deduplication
- Hands on pytest experience, including fixtures, markers, and unit, integration, and end to end testing
- Experience developing REST APIs using FastAPI or an equivalent framework
- Practical experience with Docker and Compose based development environments
- Strong Git and CI practices, including PR workflows, linting, type checking, and automated test gates
Responsibilities
- Replatform legacy T-SQL workloads onto Spark and Delta Lake pipelines
- Build and maintain distributed data pipelines using Spark and PySpark
- Develop REST APIs using FastAPI or similar frameworks
- Work on AI enabled capabilities within data platforms
- Write type safe, well tested, and specification driven Python code
- Implement modern packaging and CLI tools
- Apply best practices in Git, CI, linting, type checking, and automated testing
- Collaborate with the team to modernise data platforms and improve performance
Job details
- Organisation
- Goldman Tech Resourcing
- Location
- Gauteng, KwaZulu Natal, Western Cape
- Job type
- Full time
- Experience level
- Mid (4+ years)
- Category
- IT & Software Development
- Application Email
- it.vacancy@stand-outstaffing.co.za
- Application Method
- Apply via email
Success Guide
Hello there! This Python, Spark, and AI Developer role is a brilliant chance for you to work on modernising data platforms. If you have at least 4 years of Python experience and a solid background in Spark, this is a role worth going for.
To stand out, make sure your CV highlights your practical experience with Spark, Delta Lake, and building data pipelines. Also mention any REST API development using FastAPI. A link to your GitHub profile can really help show your skills. When you apply, send your CV directly to it.vacancy@stand-outstaffing.co.za and use a clear subject line like the job title.
The team values clean, well tested code, so be ready to talk about your testing practices in an interview. The role covers multiple provinces, so if you are in Gauteng, KwaZulu Natal, or Western Cape, you are in a good spot. Even if you do not hear back within 3 days, do not be discouraged. Keep pushing forward and good luck!
How we help you apply
You focus on getting ready — SPAAN NOW handles the paperwork. Here's what happens when you apply:
- Our AI tailors your CV to match this job's keywords and requirements, so it stands out to recruiters and screening systems.
- A custom cover letter is written specifically for this role — no generic templates.
- Your application is submitted with the documents this job needs: CV. SPAAN NOW pulls these from your profile so you don't upload them every time.
- You're notified the moment it's sent, and you can track its status any time under Applications.
- If you're shortlisted, use AI Interview Prep to practise questions commonly asked for roles like this — so you walk in confident and ready.
Everything is done for you, but you stay in control — review or edit anything before it goes out.
Spaan Talk
Nobody has said anything yet. Be the first.