Senior AI Engineer

OpenUp

Amsterdam

2 weeks ago

Amsterdam

2 weeks ago

Apply

Senior AI Engineer

Senior AI Engineer needed at OpenUp, a mental health care company, to design and build AI-powered product experiences. Responsibilities include multi-agent architecture, evaluation infrastructure, prompt engineering, and production AI systems. Requires 5+ years software engineering experience with at least 2+ years in AI/LLM systems, strong Python skills, and experience with Azure OpenAI.

Apply

Hybrid

Full-time

Senior

Python

Salary

Not specified

Work Location

Amsterdam, North Holland, Netherlands, NL

Work Model

Flexible work model (hybrid and options for remote work)

Experience Required

5 years

Employment Type

Full-time

Experience Level

Senior AI Engineer

Core Qualifications

Technical (Must-have)

PythonAILLMmulti-agent architecturesorchestrationtool callingmemorystate managementevaluation pipelinesprompt engineeringRAGembeddingsvector searchgrounding techniquesAzure OpenAINLPconversational AIdialogue systems

Soft Skills

curiositymotivationcollaborationcommunicationproblem solving

Tools (Must-have)

SQL ServerReactJS

Preferred Qualifications

Technical (Nice-to-have)

.NETopen-source LLMsLlamaMistralself-hosted model deploymentAzure AI Agent ServiceSemantic Kernelfine-tuning techniquesmodel adaptationEU AI Act compliance

Key Responsibilities

•Multi-agent architecture and orchestration: Design and build agent-based systems that go beyond rigid scripts. Implement orchestration patterns including tool usage, inter-agent communication, memory management, and context handling. Own architectural decisions about when to use single-agent vs. multi-agent approaches and how agents hand off to each other.
•Evaluation and quality infrastructure: Build and maintain evaluation pipelines that catch regressions before users do. Design golden datasets, automated evaluation rubrics, and monitoring dashboards. Establish quality gates that every prompt change and model update must pass. This is the most critical part of the role in a mental health context.
•Prompt engineering and optimization at scale: Design, version, and optimize prompts across multiple agent personas and conversation types. Build the tooling and processes that make prompt iteration systematic: A/B testing frameworks, performance benchmarks, cost analysis, and safety checks. Treat prompt engineering as a proper engineering discipline with version control, testing, and rollback.
•RAG and grounding systems: Build AI systems grounded in OpenUp content using techniques such as RAG and agentic retrieval. Ensure the AI Guide draws from verified clinical content, articles, and expert knowledge rather than generating unsupported claims. Optimize retrieval quality, relevance, and latency.
•Production AI systems: Productionize AI solutions on Azure, including monitoring, observability, and continuous improvement. Own the operational health of AI features: latency, cost, error rates, and user satisfaction metrics. Work with the platform team on deployment, scaling, and reliability.
•Model-agnostic design for data sovereignty: Contribute to architecture decisions that keep the application layer decoupled from any single model provider. Design configuration-driven model selection and data isolation patterns that enable future private cloud and on-premise deployment scenarios for enterprise customers.
•Cross-functional collaboration: Collaborate with Product and Design to align AI behavior with intended user experience. Work with clinical advisors on safety and appropriateness. Balance user experience with privacy, safety, and compliance requirements (GDPR, EU AI Act).

Senior AI EngineerMental Health CareEngineeringInformation TechnologyAILLMPythonAzureMulti-agentConversational AI

Core Qualifications

Technical (Must-have)

Soft Skills

curiositymotivationcollaborationcommunicationproblem solving

Tools (Must-have)

SQL ServerReactJS

Key Responsibilities

•Multi-agent architecture and orchestration: Design and build agent-based systems that go beyond rigid scripts. Implement orchestration patterns including tool usage, inter-agent communication, memory management, and context handling. Own architectural decisions about when to use single-agent vs. multi-agent approaches and how agents hand off to each other.

•Evaluation and quality infrastructure: Build and maintain evaluation pipelines that catch regressions before users do. Design golden datasets, automated evaluation rubrics, and monitoring dashboards. Establish quality gates that every prompt change and model update must pass. This is the most critical part of the role in a mental health context.

•Prompt engineering and optimization at scale: Design, version, and optimize prompts across multiple agent personas and conversation types. Build the tooling and processes that make prompt iteration systematic: A/B testing frameworks, performance benchmarks, cost analysis, and safety checks. Treat prompt engineering as a proper engineering discipline with version control, testing, and rollback.

•RAG and grounding systems: Build AI systems grounded in OpenUp content using techniques such as RAG and agentic retrieval. Ensure the AI Guide draws from verified clinical content, articles, and expert knowledge rather than generating unsupported claims. Optimize retrieval quality, relevance, and latency.

•Production AI systems: Productionize AI solutions on Azure, including monitoring, observability, and continuous improvement. Own the operational health of AI features: latency, cost, error rates, and user satisfaction metrics. Work with the platform team on deployment, scaling, and reliability.

•Model-agnostic design for data sovereignty: Contribute to architecture decisions that keep the application layer decoupled from any single model provider. Design configuration-driven model selection and data isolation patterns that enable future private cloud and on-premise deployment scenarios for enterprise customers.

•Cross-functional collaboration: Collaborate with Product and Design to align AI behavior with intended user experience. Work with clinical advisors on safety and appropriateness. Balance user experience with privacy, safety, and compliance requirements (GDPR, EU AI Act).