Debug retrieval miss or ungrounded fluent answer in the café FAQ retriever

Do not start with a speculative fix for the café FAQ retriever. Force the failure on purpose, save the before output, then change one cause at a time. Retries are allowed only for transient conditions—not for bad input that will fail forever on build-mini-rag.

def answer(question,docs):
 words=set(question.lower().split()); scored=[(len(words&set(text.lower().split())),k,text) for k,text in docs.items()]
 score,k,text=max(scored)
 return {'answer':text,'source':k} if score else {'answer':'Not in the FAQ.','source':None}
print(answer('refund policy?',{'hours':'open weekdays'}))

Expected evidence: Not in the FAQ with source None. If you cannot reproduce on demand, you do not yet control the failure mode for build-mini-rag.

After repair, rerun the exact reproduction command. Keep the failing fixture as a regression seed for the observability page. For the five-document café FAQ retriever, remember the claim you are restoring: overlap retrieval returns the right note or abstains on zero overlap.

Store a one-command reproduction for: retrieval miss or ungrounded fluent answer. The command should use café notes: hours, wifi, pets, allergens, parking or a minimal mutant of it. Paste the failing output into notes/failure-before.txt (or your shell scrollback as copied text). After the fix, paste notes/failure-after.txt and keep both.

Retries belong only on transient faults. If the failure is bad input, a bad allowlist, or a logic bug in the café FAQ retriever, retrying will amplify cost without repairing trust around answer a café question only when a relevant local note is retrieved and cite its ID.

Classify the failure as prevent, detect, contain, or recover—using this lab’s language, not a generic poster. For build-mini-rag, the first fix should usually be detect+prevent at the boundary, because retrieval miss or ungrounded fluent answer is cheaper to stop early than to explain in production prose.

At the debugging stage for build-mini-rag, the job is narrower than finishing a product demo. You are creating one progressive evidence piece about café notes: hours, wifi, pets, allergens, parking that later pages inherit without redefining success. Keep that fixture small enough to inspect by hand, keep outputs copy-pasteable as text, and refuse to narrate this baseline as if it were a production SLA: always-answer-from-largest-document heuristic.

For this page specifically, success looks like before/after evidence for the characteristic failure while still centering the user decision to answer a café question only when a relevant local note is retrieved and cite its ID. If you cannot point to a file, command, or assertion that proves that for the café FAQ retriever, stay on this page instead of advancing.

How-to: build a 5-document RAG app · Glossary: RAG

Previous · Next