Senior AI Researcher- Pre-training (f/m/d)

About the Position

Senior AI Researcher - Pre-training (f/m/d)

Aleph Alpha is one of the few companies in Europe doing serious foundation model pre-training like our recently released Kolibri-1 model. Our customers - in finance, manufacturing, public administration - need models that understand German, meet European regulatory requirements, and work reliably in high-stakes settings. We're building that in Heidelberg.

We are hiring a Senior AI Researcher to join our Pre-training team and to advance the architecture and training of our next generation of foundation models. If you are excited about designing inference-efficient architectures, optimizing training recipes that scale reliably, and training models on a large scale cluster (thousands of NVIDIA Blackwell GPUs), we would love to hear from you.

Responsibilities

• Recipe & Architecture Optimization: Own core elements of the training recipe (optimizers, schedules, initialization) and design PyTorch-based architectural improvements to maximize convergence, stability, and training efficiency.
• Scaling Strategy & Predictability: Develop hyperparameter scaling laws and scale-up methodologies, using small-scale proxy experiments to reliably predict multi-thousand-GPU behavior and de-risk major training decisions.
• Stability, Diagnostics & Debugging: Investigate complex convergence issues (loss spikes, divergence) and resolve hard-to-reproduce distributed system failures like communication bottlenecks, race conditions, and synchronization errors.
• System-Model Co-Design: Partner with Compute Performance, Data, Evaluation, and Post-Training teams to align the model lifecycle with hardware constraints, memory bandwidth, and communication topologies.

Requirements

• Proficient in Python and deeply familiar with PyTorch-based training workflows.
• Strong track record in machine learning research and software engineering, demonstrated through shipped models, impactful open-source contributions, or published research.
• Strong mathematical foundation and comfortable reasoning formally about optimization, scaling behavior, and training dynamics.
• Deep understanding of transformer training dynamics, optimization, and the behavior of large distributed training jobs.
• Ability to design rigorous experiments, reason clearly from noisy results, and translate empirical observations into robust training decisions.
• Hands-on experience pre-training large models (e.g., 7B+ parameters) on substantial infrastructure (e.g., 100+ GPU clusters).
• Strong software engineering practices, including writing maintainable, well-tested code and supporting reproducible experimentation workflows.
• Ability to implement complex model architectures efficiently and reliably and to debug complex issues across model code, training dynamics, and distributed systems.
• Effective collaboration within a research and engineering team and clear communication about your work across Pre-training and the broader Aleph Alpha organization.
• Ability to work in Germany and collaborate regularly on site in Heidelberg as part of the Pre-training team.

Preferred Qualifications

• Large-Scale Training: Hands-on experience training LLMs or multimodal models on large GPU clusters using distributed frameworks (e.g., Megatron-LM, DeepSpeed, torchtitan).
• Predictive Scaling: Familiarity with scaling laws, hyperparameter transfer, or methods for predicting large-scale training behavior from smaller proxy runs.
• Stability & Performance: Experience profiling distributed jobs and diagnosing training anomalies like loss spikes, numerical instability, or optimizer pathologies.
• Advanced Architectures: Exposure to sparse training approaches (e.g., Mixture-of-Experts) and an understanding of their routing and systems trade-offs.
• Track Record of Impact: Demonstrated research excellence through top-tier publications (NeurIPS, ICML, ICLR), impactful open-source contributions, or significant shipped technical work.
• Systems Curiosity: Low-level kernel optimization is not required, but we highly value a strong curiosity about the hardware and systems constraints that shape scale.

We Offer

• Become part of an AI revolution!
• 30 days of paid vacation
• Access to a variety of fitness & wellness offerings via Wellhub
• Mental health support through nilo.health
• Substantially subsidized company pension plan for your future security
• Subsidized Germany-wide transportation ticket
• Budget for additional technical equipment
• Flexible working hours for better work-life balance and hybrid working model
• JobRad® Bike Lease

About the Company

Aleph Alpha is one of the few companies in Europe doing serious foundation model pre-training. Our mission is to build models that understand German, meet European regulatory requirements, and work reliably in high-stakes settings. We are based in Heidelberg and are looking for passionate researchers and engineers to join our team.