Reference · How-to · ~10 min
How to log production LLM calls
Last updated
Debug quality, cost, and incidents without **leaking secrets or PII**.
Debug quality, cost, and incidents without **leaking secrets or PII**.
Steps
1. **Generate a request ID** per user turn; propagate through retrieval + LLM
2. **Log metadata** — model, latency, token counts, finish reason, error code
3. **Redact prompts** — hash or truncate; never log API keys or raw passwords
4. **Sample full traces** — 1–5% for deep debugging; rest metadata-only
5. **Track cost** — input/output tokens × price table per model
6. **Alert** — error rate spikes, latency p95, budget thresholds
Log row example
{
"request_id": "req_8f2a",
"model": "gpt-4o-mini",
"latency_ms": 842,
"prompt_tokens": 1204,
"completion_tokens": 186,
"retrieval_hit": true,
"user_id_hash": "u_…"
}