Hosted quality platform from the DeepEval maintainers. It captures each LLM call as a trace with inputs, outputs, tool calls, latency, token cost and metadata, converts flagged traces into evaluation datasets, and runs metric-based regression tests on pull requests. Governance controls are not documented.
hybrid · generally available · Research snapshot 2026-09-06
Show me how a failing production trace becomes a dataset case that then blocks a pull request when the metric regresses.
Capabilities and evidence
Support labels reflect the supplied research. Documentation and vendor claims are not independent product tests. “Not established” means the researcher did not find support; it does not prove a capability is absent.
Documented by provider
Documentation states every LLM call is captured as a trace with inputs, outputs, tool calls, latency, token cost and metadata, with example traces showing agent, tool and function spans and totals for latency, tokens and cost.
Limit: Does not establish retention duration, tamper-evidence or export format for these records.
The platform converts observability traces into evaluation datasets, auto-categorizes failures and edge cases, and runs regression tests on every pull request.
Limit: No documented policy gate that blocks deployment independently of the customer's own CI configuration.
Confident AI is presented as the AI quality platform built by the creators of the open-source DeepEval evaluation framework, which runs evaluations locally and can push reports to the cloud.
Limit: The hosted platform itself is not documented as self-hostable.