prompt-optimization
(3)
Confident AI
Confident AI is a San Francisco startup (YC W25) that builds the evaluation and testing layer for LLM applications. Its open-source frameworks DeepEval and DeepTeam - often described as 'Pytest for LLMs' - let engineers unit-test, benchmark, red team, and monitor chatbots, agents, and RAG pipelines with research-backed metrics. The cloud platform adds dataset management, observability, adversarial testing, and governance so teams can prove AI quality before and after shipping.
Comet
Comet is an AI developer platform that helps machine learning and AI teams track, evaluate, monitor, and manage models across their full lifecycle - from training experiments to production LLM and agentic applications. Founded in 2017 by Gideon Mendels and Nimrod Lahav, the New York-based company built its reputation on experiment tracking ('GitHub for machine learning') and now extends that into the generative-AI era with Opik, its open-source LLM observability and evaluation platform.
ZeroEval
ZeroEval is a New York-based AI startup from Y Combinator's Summer 2025 batch building an auto-optimizer for AI agents. Founded by Jonathan Chavez and Sebastian Crossa - two friends who met in college in Mexico - the platform captures every interaction your AI agent makes, scores quality with custom LLM judges, and automatically turns real production data into better prompts. The result: agents that get smarter after launch without manual intervention. Trusted by DoorDash, Datadog, Hugging Face, and Harvard Medical School, ZeroEval closes what the founders call 'the last mile reliability gap' in agentic AI.