2 weeks ago

Inference Engineer - Acceleration

kaiko.ai

Amsterdam

2 weeks ago

Apply

Kaiko.ai is seeking an Inference Engineer to optimize cost-per-token on their Blackwell inference cluster, focusing on disaggregated prefill/decode, KV-cache hierarchy, and low-precision MoE serving. The role involves working with open-weight MoE models and requires deep GPU systems experience. Based in Netherlands or Switzerland, hybrid work model.

AIHybridFull-timeSeniorCUDATriton

Inference Engineer - Acceleration

Apply

AIHybridFull-timeSeniorCUDA

Salary

Not specified

Work Location

Amsterdam, North Holland, Netherlands, NL

Work Model

Hybrid: at least 50% of time at the office

Employment Type

Full-time

Experience Level

Senior (deep GPU systems experience, shipped production inference stack at scale)

Core Qualifications

Technical (Must-have)

CUDATritonCUTLASSFlashInferFlash AttentionNsightvLLMSGLangTensorRT-LLMMoEKV-cacheRDMAAttention mechanismsRoofline model

Soft Skills

OwnershipCollaborationAmbitionCuriosity

Preferred Qualifications

Technical (Nice-to-have)

FP8FP4AWQGPTQDequantizationExpert parallelismRoutingBatchingHealthcare

Key Responsibilities

•Instrument and analyze the inference stack on Blackwell, token cost, throughput, latency, uptime — and own the path to the cost target.
•Tune scheduling and admission control to hold the stack at its cost floor across ramp-up and steady-state regimes.
•Own the KV-cache hierarchy and the prefill / decode split.
•Drive low-precision MoE serving with quality regression gates.

Inference EngineerGPU optimizationCUDAMoEBlackwellHealthcare AIHybridSenior

Key Responsibilities

•Instrument and analyze the inference stack on Blackwell, token cost, throughput, latency, uptime — and own the path to the cost target.

•Tune scheduling and admission control to hold the stack at its cost floor across ramp-up and steady-state regimes.

•Own the KV-cache hierarchy and the prefill / decode split.

•Drive low-precision MoE serving with quality regression gates.