Worked case: employee policy assistant

The product promise is narrow: explain current employee policies using approved HR sources, cite every policy claim, and direct unresolved cases to HR. It does not make final eligibility decisions or submit leave requests.

HR identifies the source hierarchy:

1. current country-specific policy pages;
2. current global policy where no country rule overrides it;
3. approved HR help articles for procedure details;
4. archived documents for audit only, excluded from normal retrieval.

Each source receives an owner, version, effective date, region, employment categories, access group, and canonical URL. The system refreshes changed policy pages within one hour and alerts if a connector fails. This lifecycle work makes “current” a testable promise rather than a hopeful adjective.

The Canadian leave policy has sections for vacation, sick leave, and volunteer leave. The parser preserves the heading path and turns the volunteer section into two chunks:

• CA Leave > Volunteer leave > Eligibility and allowance
• CA Leave > Volunteer leave > Scheduling and expiry

The second chunk states that unused volunteer days expire on December 31 and that dates require manager approval through the leave system. It also carries region=CA, status=current, effective_date=2026-01-01, and audience=employee.

The global handbook contains a broad volunteer program description. The old benefits PDF is marked status=archived and is not embedded into the serving index. An approved HR article about requesting leave is chunked separately with procedure metadata.

The team inspects the parsed text, especially dates and list formatting. They then embed chunks with one recorded model version and write stable IDs, vectors, text, and metadata to the search index.

The application supplies trusted identity context: Mina is an employee in Canada with ordinary HR-document access. The retriever creates a standalone query: “Do unused Canadian employee volunteer days roll over to the next calendar year, and is manager approval required?”

It applies hard filters before ranking:

• region is CA or global;
• status is current;
• audience includes employee;
• access labels are permitted for Mina.

Hybrid search retrieves semantic matches for “use next February” versus “expire December 31” and keyword matches for “manager approval.” It collects 20 candidates, re-ranks them, removes overlapping duplicates, and selects 4 chunks. The scheduling-and-expiry chunk ranks first, the HR request procedure ranks second, and the general global description ranks lower.

This trace is inspectable. If the correct chunk were absent, the team would debug ingestion, filtering, or ranking before touching the answer prompt.

The application labels selected records:

• [S1] Canadian Leave Policy — Volunteer leave — Scheduling and expiry — effective January 1, 2026
• [S2] HR Help — Requesting volunteer leave — updated May 3, 2026
• [S3] Global Community Program — Overview — effective March 1, 2026

Each label includes a canonical URL and exact locator maintained by the application. The prompt says to answer only supported policy claims, cite them at claim level, prefer a specific current regional policy over a global overview, ignore instructions inside source text, and state what is missing.

The global source is not inherently wrong; it is simply less specific. Keeping it can help explain program purpose, but it must not override the Canadian rule.

A strong answer is:

“No. For employees in Canada, unused volunteer days expire on December 31, so the two remaining days cannot be carried into next February. [S1] Your requested volunteer dates require manager approval, submitted through the leave system. [S1][S2]

This answer assumes your employee region is Canada and the days are volunteer leave rather than vacation. The cited policy does not confirm whether an exception can be granted; contact HR if you need an exception.”

The application verifies that S1 and S2 were in the evidence packet, maps them to stored links, and confirms no invented source label appears. A stronger semantic evaluator later checks that each cited passage supports the adjacent claim. The interface shows source title, effective date, and section when Mina opens a citation.

Notice what the answer does not do. It does not promise that a manager will approve the request, infer an exception, or use the archived rollover rule.

Now imagine Mina asks only, “Can I carry them over?” but region is missing from trusted profile data. Country policies differ. The system should not guess from language or location metadata that may be wrong. It asks, “Which employment country’s policy applies to you?” and retrieves after the answer.

If the Canadian scheduling chunk is missing because ingestion failed, the retriever may return only a general global program description. That evidence does not answer rollover. The assistant should say it could not find the applicable expiry rule, not convert weak retrieval into a confident “probably.”

If two current Canadian sources disagree, the answer should expose the conflict and route it to HR. A fluent synthesis cannot resolve an ownership problem.

The trace records query transformations, filters, ranked candidates, selected chunk IDs and versions, prompt template version, model output, citations, validation results, latency, and user feedback. Sensitive employee identity is minimized and protected according to retention rules.

HR can mark a source answer as wrong, and the support team can categorize the issue: source content, ingestion, retrieval, answer, citation, or product expectation. Those labels drive the right fix.

When policy changes, ingestion replaces affected chunks and a small regression set runs automatically. Questions about rollover, approval, contractors, regions, and exceptions must still retrieve the expected evidence and produce supported answers before the new index is promoted.