Machine Learning Engineer (m/w/d) - Foundation Models

WeSort is a high-tech startup from Würzburg, developing AI-based recycling solutions and contributing significantly to the circular economy of critical raw materials. Our systems are already in industrial use, including at Schwarz/Lidl or their environmental services provider PreZero. Our technology has been recognized multiple times, including the German Founder's Prize (awarded by Porsche and ZDF), and WeSort is part of the SprinD program (Agency for Spring Innovations). Furthermore, our team has been represented in formats such as Galileo, ZDF WISO, the Economic Weekly, and the Süddeutsche Zeitung.

We are looking for a talented Machine Learning Engineer (m/w/d) with a focus on Computer Vision and Foundation Models who will build their own "Waste Foundation Model" based on modern architectures like DINOv2, SigLIP or EVA-02 – the technological foundation on which all our future computer vision applications will be built.

We operate one of the largest continuously growing databases of labeled waste images worldwide – from real sorting facilities, across multiple material flows, lighting conditions, and degrees of contamination. These data are our strategic advantage. From this, we want to develop a domain-adapted Vision Foundation Model that serves as a backbone for all downstream tasks (detection, classification, anomaly detection, few-shot learning).

We train our KI models in Python (PyTorch), our backend platform in Rust.

Field: Software, Data & Artificial Intelligence
Workplace: Office-based in Würzburg
Contract type: Full-time employment contract
Start date: immediately

Tasks

This is your new passion:

• You develop and train our own "Waste Foundation Model" – based on state-of-the-art architectures like DINOv2, SigLIP or EVA-02 – through continued pretraining (self-supervised) on our waste image database
• You set up our complete ML training pipeline: from data preparation (WebDataset, FFCV) to distributed training (PyTorch FSDP/DDP, Mixed Precision) and model versioning
• You build and maintain our Eval Suite – the central infrastructure that measures whether our Foundation Models are really getting better: Linear Probing, k-NN-Probing, Few-Shot-Detection, Cross-Domain-Generalization, Anomaly-Detection
• You fine-tune and distill our models for specific downstream tasks and edge hardware (sorting facilities, GPU inference)
• You systematically analyze training runs, identify problems like feature collapse or domain shift, and develop sustainable solutions instead of quick fixes
• You work closely with the cloud backend team to efficiently deploy models (ONNX, TensorRT, OpenVINO)
• You actively follow research developments in the area of Vision Foundation Models and translate relevant papers into productive solutions
• You think beyond the model and have a view on how your work will affect real-world operations – for sorting facilities, customers, and the overall system

Qualification

This is what excites us:

• You bring several years of experience in developing and training computer vision models with, ideally, Vision Transformers (ViT) and self-supervised learning methods (DINO, MAE, iBOT, CLIP)
• You are proficient in PyTorch – including distributed training (DDP, FSDP), mixed precision (bf16/fp16), and performance optimization (torch.compile, profiling)
• You understand not just how to train a model but also how to evaluate it. You know that a weak eval suite makes every pretraining worthless
• You have experience with modern ML tooling stacks: Hydra for configs, Weights & Biases or MLflow for tracking, DVC for data versioning, timm for backbones
• You use modern AI tools (e.g. Claude, Copilot) to accelerate routine coding and focus on the really hard research and architecture questions
• You have a good understanding of data pipelines with large datasets (millions of images): tar-sharding, GPU augmentations (DALI), I/O bottlenecks
• Experience with detection/segmentation frameworks (MMDetection, MMSegmentation) as well as anomaly detection (anomalib) is an advantage
• You are familiar with inference optimization and model distillation (e.g. ViT-L → ViT-S) and have ideally already deployed models on edge hardware
• You possess strong problem-solving skills, analytical thinking, and scientific rigor – you work hypothesis-driven and not by the try-and-error principle
• You are comfortable with cloud GPU infrastructure (AWS, Azure, GCP or On-Premise H100/A100 cluster)
• Fluent German and good English language skills are expected
• Ideal candidates have their own research experience (papers, open-source contributions, conference talks) or are PhD-holders – not a must, but a plus

Benefits

This is what you can look forward to:

• Work on the "green field" – building your own Foundation Model strategy without legacy issues or technical debt
• Access to a unique, growing database of labeled waste images from real sorting facilities – a strategic advantage that no university and few competitors have
• Use of current frameworks and a top-modern tech stack (PyTorch 2.x, FSDP, Hydra, W&B, DVC, timm)
• Sufficient compute resources for pretraining runs – we know that serious Foundation Model training is no hobby project
• Closely working with research partners (e.g. THWS Würzburg as part of the Green-INNO program) and the opportunity to publish your own research results
• Working in a dynamic and interdisciplinary startup team with a lot of responsibility and design freedom from day one
• Fast decision-making processes and direct communication
• Technology with meaning: You work on the biggest levers of our time – AI, recycling, and circular economy

Have we piqued your interest? Then we look forward to your application! A formal letter is not necessarily required, but please provide your expected annual gross salary.

Find Jobs in Germany on Arbeitnow