Worked example: classify a product

The team describes each feature without marketing language.

Rewrite: The system uses a draft job description to generate alternative wording, which a recruiter edits before publication.

Summarize: The system uses an applicant’s résumé to generate a summary, which a recruiter reads alongside the original.

Score: The system uses résumé content and job criteria to assign a match score, which recruiters use to prioritize interview review.

Reject: The system uses the match score and a threshold to send rejection messages without prior recruiter review.

The sequence reveals increasing decision power. All four features concern employment, but the first drafts content while the fourth triggers an adverse outcome.

Direct users are recruiters and hiring managers. Affected people are applicants, including people who may never log into Pathfinder. The employer is also affected through hiring quality, discrimination risk, reputation, and compliance duties.

The interests at stake include employment opportunity, fair treatment, privacy, accessibility, and the ability to contest incorrect data. Scale may be hundreds of applicants per role. Rejection deadlines make some errors difficult to reverse: a later apology may not restore an interview after a position is filled.

Applicants may have disabilities, use different résumé formats, write in a second language, or follow career paths underrepresented in training data. These are foreseeable conditions, not exotic edge cases.

Inputs include names, contact details, location, employment history, education, skills, dates, and free text. Résumés may reveal or imply age, disability, nationality, religion, family status, or other sensitive traits. The score is a new inference linked to a person.

The team asks:

• Did applicants receive appropriate information about processing?
• Which fields are needed for each feature?
• Can names, photos, addresses, and unrelated details be removed before scoring?
• Does the vendor retain prompts or use them for model improvement?
• Where are records and logs stored, and who can access them?
• How are access, correction, deletion, and retention handled?

The rewrite feature need not receive applicant data at all. Data boundaries should prevent accidental crossover.

Employment is a sensitive domain because the system can shape opportunity. Dedicated AI rules may classify some employment uses as high risk or impose notice, assessment, audit, or other duties. Existing privacy, employment, anti-discrimination, accessibility, consumer-reporting, records, or local automated-decision rules may also be relevant.

The team does not conclude, “The vendor says it complies, so we comply.” It records jurisdictions, consults appropriate specialists, and assigns an internal high-impact route to scoring and rejection while the exact legal analysis proceeds.

The rewrite feature receives a lower initial classification because it creates an editable draft and does not process applicant data. It still needs quality review: wording could discourage applicants or introduce requirements the employer did not intend.

For summarization, the team tests whether qualifications, dates, negations, and accommodations are omitted or distorted. Reviewers compare summaries with original résumés.

For scoring, the team demands a clear definition of “match.” It examines job relevance, validation methods, representative roles, subgroup outcomes, missing data, alternative résumé formats, language differences, and stability over time. A correlation found in historic hiring data could reproduce historic bias rather than predict job performance.

The team lists failure pairs:

• a qualified applicant receives a low score and loses review;
• an unqualified applicant receives a high score and consumes recruiter attention;
• a parsing error turns “not certified” into “certified”;
• a proxy such as location changes outcomes without job relevance;
• the model produces different scores after an unannounced update.

Average accuracy cannot settle these concerns.

The employer permits rewriting with ordinary recruiter editing and approved templates. Summaries may be piloted only when the original résumé remains prominent and discrepancies can be reported.

Scoring stays in a limited evaluation environment. Before any operational use, it needs domain validation, fairness analysis, privacy review, applicant communication, trained reviewer procedures, logging, monitoring, and a correction route.

Automatic rejection is disabled. A human must review source material before an adverse decision. This does not make the system automatically lawful or fair; it prevents one especially consequential automatic action while broader evidence is gathered.

Technical controls enforce the boundary: role permissions, no rejection API access, version pinning where available, approved data fields, and a kill switch. Policy text alone would be weaker.

The decision record names the business owner, technical owner, review date, approved features, prohibited uses, jurisdictions, evidence, known limitations, monitoring metrics, and change triggers. Approval expires after the pilot.

Triggers include a new model, new job family, new country, threshold changes, automatic messaging, new data fields, material complaints, or unequal outcome signals. The vendor must support incident investigation and communicate relevant changes.