Data Scientist
Kaizo
Data Scientist
Kaizo is seeking a Data Scientist to own the evaluation and quality loop of their AutoQA product, which uses LLMs to review customer support conversations. The role involves translating customer rubrics into testable instructions, running experiments, building datasets, and making results actionable. Candidates should have strong Python skills, applied statistics knowledge, and 0-2 years of industry experience.
Data Scientist
Kaizo is seeking a Data Scientist to own the evaluation and quality loop of their AutoQA product, which uses LLMs to review customer support conversations. The role involves translating customer rubrics into testable instructions, running experiments, building datasets, and making results actionable. Candidates should have strong Python skills, applied statistics knowledge, and 0-2 years of industry experience.
Salary
Core Qualifications
Technical (Must-have)
Soft Skills
Preferred Qualifications
Technical (Nice-to-have)
Key Responsibilities
- Translate customer rubrics into AutoQA instructions
- Run experiments to improve accuracy using LangSmith and BigQuery
- Build golden datasets and generate synthetic data
- Build LLM-as-a-judge pipelines
- Make results actionable by driving decisions on prompts, tools, and capabilities
- Evaluate retrieval, tool calling, and speech pipelines
- Join customer calls to understand quality definitions