Reference · How-to · ~6 min

How to run golden-task eval locally

Last updated

Score a prompt change before opening a PR.

Score a prompt change before opening a PR.

Steps

1. Create `eval/golden.json` with 10 tasks

2. Script calls staging model with fixed prompt template

3. Score with rules: must_include, must_not_include, optional LLM judge

4. Print pass rate; exit 1 if below threshold

5. Commit golden file — treat changes like code review

**Try the lesson:** `eval-in-code` in Lane D