Worked case: semantic search

The team starts with manuals, troubleshooting articles, and approved support notes. Each source receives a stable ID, product model, language, publication date, permissions, and canonical URL. Obsolete manuals are excluded or marked with validity dates. Search cannot rescue an untrustworthy collection; it can only rank what is present.

Documents are split into passages. One candidate chunk might be:

“On sustained climbs, high motor load can raise battery temperature. Assistance is temporarily reduced or disabled outside 0–45°C. Stop safely, allow the battery to cool, and inspect the temperature warning.”

The chunk keeps its heading, “Assistance loss on climbs,” because headings carry context. The team avoids combining it with three pages of unrelated warranty text. It also avoids splitting the temperature range from the action it governs.

At ingestion, the application hashes source text. Unchanged chunks keep their vectors; changed chunks are re-embedded. Deleted or revoked content is removed from the index. This matters because stale search results can be more dangerous than no result.

The embedding service converts each passage into, say, a 768-dimensional vector. The application stores:

• the vector;
• original passage text;
• document and chunk IDs;
• heading and URL;
• product, language, version, and permissions;
• embedding model name and version;
• source hash and ingestion timestamp.

The vector database builds an index using the metric recommended for the model. If cosine similarity is used and the system requires normalized vectors, normalization happens consistently during both ingestion and querying.

The embedding is not the support answer and should not replace the source. It is an address used to find candidates. The original passage is what a person reads or what a later generation step receives.

The rider’s query arrives with account context showing product TrailVolt X2 and locale en-GB. The system does not add private data to the embedding unless needed and permitted. It may perform a controlled rewrite such as:

“TrailVolt X2 motor assistance disengages during sustained steep uphill riding.”

Rewriting can expand shorthand and include known product context, but it can also distort intent. The application retains the original query, logs the rewrite separately, and evaluates whether rewriting improves retrieval.

The same compatible embedding model encodes the query, using the provider’s query mode if one exists. The search applies hard filters for product family, language, current publication status, and the user’s access rights. A semantically close internal service bulletin must not leak to a customer who cannot view it.

Assume approximate vector search returns these toy candidates:

1. “Assistance loss on climbs” — cosine 0.86
2. “Battery storage temperature” — cosine 0.81
3. “Regional assistance speed limits” — cosine 0.79
4. “Hydraulic brake fade downhill” — cosine 0.74

The scores rank candidates but are not probabilities. Candidate two shares battery and temperature concepts but concerns storage, not riding. Candidate three shares the notion of motor assistance stopping but describes a legal speed cutoff. These are hard negatives: topically close passages that do not answer the question.

A cross-encoder reranker reads the query together with each candidate and may reorder them more precisely. Keyword signals can reward “sustained climb,” while product metadata enforces applicability. The pipeline returns the top three passages, not just one, so the interface can expose supporting context.

If every result falls below a locally validated threshold, the system should say that no strong match was found or route to support. The nearest item in a collection is not necessarily relevant.

For search, the interface shows titles, highlighted passages, product applicability, dates, and links. It may explain that the result matched “steep climbs” with “sustained high-load ascent,” but it should not invent certainty about the model’s internal reasoning.

If the system feeds passages to a language model, it has become a retrieval-augmented generation pipeline. The generated response should stay grounded in retrieved text, cite the manual, preserve safety instructions, and admit when evidence is insufficient. Retrieval relevance does not guarantee generation faithfulness.

The system also distinguishes advice from diagnosis. The passage identifies one plausible cause, but low charge, a sensor fault, a configured speed limit, or hardware damage could create similar symptoms. A responsible answer gives checks supported by sources and escalates unresolved cases rather than declaring the battery defective.

The team builds a test set from real, privacy-reviewed queries and expert-written scenarios. Each query has relevant passages and known hard negatives. Useful retrieval measures include Recall@k, the fraction of answerable queries with a relevant passage in the top k, and MRR, which rewards placing the first relevant passage near the top.

Suppose 100 queries have at least one relevant passage. If 92 have one within the top five, Recall@5 is 0.92. That does not reveal whether results are safe across products or languages, so the team slices the metric by model, locale, query type, and rare issue. It manually reviews false positives and false negatives.

Operational measures matter too: embedding failures, stale-index lag, query latency, filter correctness, and “no result” rate. For a generated answer, evaluate citation support and task success separately from retrieval. One average score cannot locate which pipeline stage failed.