Open-source Python framework for testing LLM applications: it runs metric-based scoring on test cases and on captured agent trajectories, including individual LLM calls, tool use, retrieval and sub-agent handoffs. It executes locally in developer or CI environments and provides no access control or audit records itself.
open_source · generally available · Research snapshot 2026-09-06
Show me a CI test run that scores my agent's full trajectory, including whether the right tools were called with the right arguments.
Capabilities and evidence
Support labels reflect the supplied research. Documentation and vendor claims are not independent product tests. “Not established” means the researcher did not find support; it does not prove a capability is absent.
Documented by provider
DeepEval supports end-to-end and component-level evaluation, scoring complete agent trajectories across decisions and actions as well as individual steps such as LLM calls, tool use, retrieval and sub-agent handoffs.
Limit: The repository README does not establish accuracy of the bundled metrics or any governance controls.