Reference · Snippet · python

Golden eval exit code

Last updated Jul 17, 2026

evalciagents

import json, sys

tasks = json.load(open("eval/golden.json"))
passed = sum(run_task(t) for t in tasks)
rate = passed / len(tasks)
print(f"pass rate: {rate:.0%}")
sys.exit(0 if rate >= 0.8 else 1)