Reference · Glossary
DeepEval
Last updated
An open-source LLM evaluation framework that plugs into your existing test runner (like pytest) — you write eval "test cases" the same way you write unit tests, covering correctness, bias, toxicity, and hallucination checks.
#When to use
Teams that want LLM quality checks living next to their normal test suite and running in the same CI pipeline, instead of a separate eval tool with its own workflow.
#When not to
A quick one-off manual comparison of two prompts — a lightweight side-by-side check may be faster than wiring up a formal test case first.
#Example
from deepeval import assert_test
from deepeval.metrics import HallucinationMetric
from deepeval.test_case import LLMTestCase
test_case = LLMTestCase(input="What is our refund window?", actual_output=answer, context=[policy_text])
assert_test(test_case, [HallucinationMetric(threshold=0.5)])