Senior Applied AI Engineer, Agent Quality & Evaluations

• A track record of building or meaningfully improving production LLM or agentic products, not just prototypes
• Strong software engineering skills in Python, TypeScript or a similar language that let you build reliable internal tools, evaluation systems and prototypes
• The interaction between prompts, context, tools, agent loops, orchestration and product state is clear to you
• LLM evaluation, observability and experimental design are areas where you have direct experience, and you know how to combine automated checks with human judgment
• A strong product sense that helps you translate ambiguous member feedback into testable quality definitions and concrete system changes
• Systems thinking and reusable patterns guide your work, so today's email experience gets better without building in assumptions that make tomorrow's surfaces harder to build
• Across technical and nontechnical disciplines, your communication is clear and you can explain tradeoffs involving quality, latency, reliability and cost

Bonus points if you've:

• Built content-generation, creative or marketing tools
• Built multi-turn agents that use tools and preserve user intent across edits
• Used Langfuse, Braintrust or comparable evaluation and observability platforms
• Worked on products for small businesses, creators or marketers

What we bring

• $150,000 - $250,000 base salary, depending on your location and experience
• Tier 1 city residents: $165,000 - $250,000
• All other US based locations: $150,000 - $235,000
• Fully paid health insurance for individual coverage
• 16 weeks paid parental leave for non-birthing parents; 22 weeks paid maternity leave for birthing parents
• Unlimited flexible time off
• 401(k) match (US employees only)
• $1,000 annual stipend for learning and development