Staff Applied AI Scientist
Culture Amp
Staff Applied AI Scientist
Culture Amp is seeking a Staff Applied AI Scientist to join the team behind AI Coach, focusing on measuring and improving AI products in production. The role involves designing evaluation systems, building LLM-powered analysis, and enabling other teams through frameworks and tooling. Candidates need strong experience in LLM evaluation, observability, and production AI systems.
Staff Applied AI Scientist
Culture Amp is seeking a Staff Applied AI Scientist to join the team behind AI Coach, focusing on measuring and improving AI products in production. The role involves designing evaluation systems, building LLM-powered analysis, and enabling other teams through frameworks and tooling. Candidates need strong experience in LLM evaluation, observability, and production AI systems.
Salary
Core Qualifications
Technical (Must-have)
Soft Skills
Key Responsibilities
- Design and run sampling, LLM-as-a-judge, and labelling systems over de-identified production traces to build longitudinal evaluation monitoring and alerting.
- Build LLM-powered analysis that works out why performance moved and recommends prompt or system changes.
- Own the full feedback loop: prompt engineering, evaluation at scale, data labelling and continuous improvement.
- Enable others through reusable frameworks, tooling and documentation so product and engineering teams run their own evaluations.
- Partner closely with Coach, product, data science and people science.
- Stay current with the latest evaluation, observability and LLMOps research.