Amsterdam
2 weeks ago
Training Performance Engineer - Acceleration
kaiko.ai
Amsterdam
2 weeks ago
Training Performance Engineer - Acceleration
Kaiko is seeking a Training Performance Engineer to optimize throughput and utilization on a Blackwell training cluster for clinical foundation models. The role involves instrumenting runs, tuning MoE parallelism, and landing custom kernels, based in the Netherlands or Switzerland with at least 50% office time.
AIHybridFull-timeEntry LevelCUDATriton
Training Performance Engineer - Acceleration
Kaiko is seeking a Training Performance Engineer to optimize throughput and utilization on a Blackwell training cluster for clinical foundation models. The role involves instrumenting runs, tuning MoE parallelism, and landing custom kernels, based in the Netherlands or Switzerland with at least 50% office time.
AIHybridFull-timeEntry LevelCUDA
Salary
Not specified
Core Qualifications
Technical (Must-have)
CUDATritonCUTLASSFlash AttentionPyTorchNsightNCCLInfiniBandNVLinkTPPPCPEPDPMoE
Preferred Qualifications
Technical (Nice-to-have)
FP8Expert-only quantizationDynamic loss scalingSparse attentionHybrid attentionMLA attentionMegatronNeMo
Key Responsibilities
- Instrument and analyze runs — MFU, throughput, uptime — and close gaps against predicted wall-clocks.
- Benchmark NCCL collectives over InfiniBand and NVLink — including rail/topology behaviour and congestion at scale.
- Drive low-precision training in our stack and validate the speed-up.
- Tune MoE parallelism (TP / PP / CP / EP / DP) per phase and characterise expert-parallel comm cost on the cluster fabric.
- Land custom attention-variant kernels (e.g. hybrid, latent-attention) into the training stack.
GPU systemsCUDANCCLInfiniBandMoE traininglow-precision trainingAmsterdamclinical AIhealthcare