llm-observability
(7)
Agnost AI
Agnost AI is a Y Combinator (S26) startup building product analytics for teams shipping conversational AI agents. It reads every production chat and voice conversation, clusters what users want, where they get stuck, and why they churn, then turns those patterns into evals, debug trails, and automatic pull requests that improve the agent. It installs in three lines of code or via an OpenTelemetry exporter, works with any LLM and framework, and processes over a million events a day for customers including Google, Exa, and Corgi.
Confident AI
Confident AI is a San Francisco startup (YC W25) that builds the evaluation and testing layer for LLM applications. Its open-source frameworks DeepEval and DeepTeam - often described as 'Pytest for LLMs' - let engineers unit-test, benchmark, red team, and monitor chatbots, agents, and RAG pipelines with research-backed metrics. The cloud platform adds dataset management, observability, adversarial testing, and governance so teams can prove AI quality before and after shipping.
SolidCore.ai
SolidCore.ai is a Menlo Park cybersecurity startup building the security and governance layer for enterprise generative AI. Founded in 2024 by HyTrust veteran Eric Chiu and former Microsoft security architect Hemma Prafullchandra, its platform captures an immutable record of every LLM prompt, response, and configuration inside a customer's own cloud, then continuously monitors for misconfigurations, policy violations, and risky usage while generating audit-ready evidence for frameworks like NIST AI RMF, ISO 42001, and the EU AI Act. The company emerged from stealth in September 2025 with $4M in seed funding led by Runtime Ventures.
Mainak Mazumdar
Mainak Mazumdar is Chief Business Officer at Fiddler AI, the AI observability and trust company, where he leads sales, customer success and partnerships. A data scientist by training with a Ph.D. from Brown, he spent two decades remaking how the media and advertising industries measure audiences - as Chief Data Officer at Nielsen (where he launched NielsenOne), Chief Research and Analytics Officer at Fox, and a leader at DoubleClick/Google, GfK and Simulmedia. His 2020 TED Talk, 'How bad data keeps us from good AI,' argued that biased data, not algorithms, is the root of unfair AI. He holds more than 15 U.S. patents and has served on the U.S. Census Scientific Advisory Committee.
Comet
Comet is an AI developer platform that helps machine learning and AI teams track, evaluate, monitor, and manage models across their full lifecycle - from training experiments to production LLM and agentic applications. Founded in 2017 by Gideon Mendels and Nimrod Lahav, the New York-based company built its reputation on experiment tracking ('GitHub for machine learning') and now extends that into the generative-AI era with Opik, its open-source LLM observability and evaluation platform.
Alex Reibman
Alex Reibman is the Co-Founder and CEO of AgentOps (Agency), a San Francisco-based developer platform for building, testing, and monitoring AI agents. A 17-time hackathon champion with 10+ years of machine learning experience, he previously led ML engineering at Ernst & Young on $40M+ engagements for clients like Goldman Sachs and American Express, co-founded Menubites.ai (acquired by Snappr in 2023), and worked as a cybersecurity data scientist. AgentOps raised a $2.6M pre-seed in August 2024 and serves enterprise clients including Microsoft, Google, Samsung, and Meta. He studied Economics and Philosophy at Emory University and won the 2024 Emory Entrepreneur Award.
Observe, Inc.
Observe is an AI-powered observability platform that unifies logs, metrics, and traces into a single streaming data lake built on Snowflake. The company helps enterprises troubleshoot distributed applications 3x faster at one-third the cost of traditional monitoring tools. Founded in 2017 by veterans from Splunk, Wavefront, and Snowflake, Observe was acquired by Snowflake in January 2026 for approximately $1 billion after raising over $665 million in funding.