Melbourne
23 Jul 2026
Senior Quality Engineer - AI
StarRez, Inc.
Melbourne
23 Jul 2026
Senior Quality Engineer - AI
StarRez, a global leader in student housing software, seeks a Senior Quality Engineer - AI to shape quality practices for AI-powered product experiences. The role involves defining AI quality standards, building evaluation systems, and coaching teams, requiring QA experience in SaaS and a data-driven approach.
Core AIRemoteFull-timeSeniorQuality AssuranceQuality Engineering
Senior Quality Engineer - AI
StarRez, a global leader in student housing software, seeks a Senior Quality Engineer - AI to shape quality practices for AI-powered product experiences. The role involves defining AI quality standards, building evaluation systems, and coaching teams, requiring QA experience in SaaS and a data-driven approach.
Core AIRemoteFull-timeSeniorQuality Assurance
Salary
Not specified
Core Qualifications
Technical (Must-have)
Quality AssuranceQuality EngineeringSaaSB2BB2B2CTest AutomationEvaluation ToolingObservabilityScriptingMetricsDashboardsTrend AnalysisReviewer AgreementOperational ReportingTechnical Communication
Soft Skills
CollaborationInfluenceCoachingMentoringData-Driven MindsetProblem SolvingCritical Thinking
Preferred Qualifications
Technical (Nice-to-have)
LLM Evaluation PipelinesAutomated Eval ToolingLLM JudgesAgent Observability PlatformsSafety EvaluationBias EvaluationCompliance EvaluationProduction MonitoringDrift DetectionPost-Release Quality Tracking
Key Responsibilities
- Lead or contribute to end-to-end AI quality and evaluation strategy for product experiences involving LLMs, RAG, prompts, tool-use, or agent workflows.
- Provide quality-focused input during ticket grooming, discovery, and feature discussions.
- Champion shift-left quality practices by influencing or mentoring developers and cross-functional partners.
- Coach or calibrate human reviewers, subject matter experts, or product teams.
- Contribute to the evolution of development or quality pipelines, including automated and human evaluation workflows.
- Define practical AI quality standards, rubrics, and evaluation criteria.
- Partner with engineering and product teams to identify AI quality risks early.
- Build or support repeatable evaluation approaches.
- Translate AI failure patterns into actionable product and engineering improvements.
- Use metrics, reviewer feedback, quality trends, and operational signals to support release readiness.
Senior Quality EngineerAIQuality AssuranceLLMRAGSaaSRemoteAustraliaStarRezStudent Housing