Data & AI Engineer – Finance AI Team

Salary

Core Qualifications

Preferred Qualifications

Key Responsibilities

• •Design, implement, and maintain scalable cloud-based data lake and pipelines, leveraging Databricks for robust, high-performance solutions.
• •Develop efficient data pipelines for ingesting, processing, and storing large datasets, incorporating best practices for data quality, lineage tracking, and metadata management - specifically optimized for LLM consumption.
• •Enforce data quality checks, monitor pipeline health, and continuously optimize reliability, scalability, and performance.
• •Move beyond 'naive RAG' to implement hybrid search, re-ranking, and contextual compression to handle dense financial documents and tables.
• •Design and deploy multi-agent systems using LangGraph, CrewAI, or Agent Bricks.
• •Implement rigorous 'LLM-as-a-Judge' frameworks to quantify accuracy, faithfulness, and latency.
• •Deploy security layers to ensure AI outputs comply with financial regulations and privacy standards.
• •Create reusable 'Agent templates' that empower Finance business partners to build their own safe AI-driven workflows.
• •Work closely with AI teams and key stakeholders across finance to understand data requirements for AI systems and ensure their smooth deployment.
• •Provide support and guidance on data engineering best practices and tools to other team members.