Apache-licensed LLM vulnerability scanner maintained by NVIDIA. It fires static, dynamic and adaptive probes at a model or dialog system to test for jailbreaks, prompt injection, toxicity, data leakage and misinformation, logs each generation and detector verdict, and outputs a report with failure rates and hit logs.
open_source · generally available · Research snapshot 2026-09-06
Show me a garak scan of my chatbot with the probe-by-probe failure rates and the hit log for successful jailbreaks.
Capabilities and evidence
Support labels reflect the supplied research. Documentation and vendor claims are not independent product tests. “Not established” means the researcher did not find support; it does not prove a capability is absent.
Documented by provider
garak combines static, dynamic and adaptive probes to test for hallucination, data leakage, prompt injection, misinformation, toxicity and jailbreaks, evaluates each probe with recommended detectors, and records generations both as received and when evaluated, with a separate hit log for successful attacks.
Limit: The README does not document tool-call or agent-trajectory testing, nor any coverage guarantee.
Supported targets include Hugging Face Hub models, OpenAI chat and completion APIs, AWS Bedrock foundation models, Replicate, LiteLLM, GGUF models via llama.cpp and arbitrary REST endpoints returning plaintext or JSON.
Limit: Support for a target does not establish probe effectiveness against it.
Documentation states garak is Apache 2.0 licensed and produces a full report detailing what worked and what needs improvement when scanning a chatbot or model.
Limit: The documentation site describes itself as a work in progress and states no governance or multi-user features.