Worked cases: competing stakeholder needs

A hospital wants an AI assistant to rank patients for earlier appointment slots when cancellations occur. The intended benefit is faster care and less staff time spent calling people who cannot attend. Stakeholders include patients, caregivers, clinicians, schedulers, accessibility teams, and hospital leaders.

The first proposal predicts who is likely to accept a short-notice slot. Historical data makes people who live close to the hospital, answer calls quickly, speak the dominant language, and have flexible work appear more “responsive.” Optimizing only for filled slots could repeatedly favor people with more resources. The efficiency metric would be real, but so would the unequal access.

The team separates two questions: clinical urgency and practical availability. Clinicians and existing policy determine urgency; the model may assist only with contact timing among patients within an appropriate priority group. Patients can state preferred channels, language, accessibility needs, transport constraints, and whether they want short-notice offers. Declining one offer does not lower future clinical priority.

The hospital measures appointment fill rate, time to care, missed offers by channel and language, and burden on staff and patients. It provides a staffed route for preference changes and audits whether some groups receive fewer realistic offers. This design does not eliminate every difference. It prevents a convenience prediction from quietly becoming a judgment of who deserves care.

Remaining questions include how much sensitive data the assistant needs and whether patients experience repeated messages as pressure. A limited pilot with clear pause authority is more defensible than immediate hospital-wide automation.

A company offers an AI coach that reviews internal documents for clarity. Employees may benefit from faster feedback, especially those writing in a second language. Managers ask to see individual “communication quality” trends so they can identify training needs.

The same output has different ethical meaning in the two settings. Private coaching supports the employee’s goal. Turning drafts and correction patterns into a performance score changes the purpose, audience, and consequence. Employees may stop using the tool or avoid experimenting if every draft can affect evaluation. Language style may be mistaken for competence, reinforcing cultural preferences unrelated to job performance.

The team keeps coaching content private by default, limits retention, and reports only carefully designed aggregate adoption information to program managers. It prohibits individual employment decisions from using coach interactions. If the company later wants to study training needs, it must propose a separate purpose, involve employee representatives and relevant specialists, minimize data, and create a non-coercive choice.

The assistant explains that its suggestions are optional and may favor conventional writing styles. It lets employees select tone goals and dismiss recommendations. Evaluation includes usefulness across languages and roles, not just agreement with one style guide.

The tradeoff is reduced management visibility. The company accepts that cost because surveillance would undermine the tool’s purpose and employee trust. The boundary is documented so a future analytics request cannot quietly expand the use.

A payment service uses AI to identify unusual transfers. Customers want quick transactions and protection from theft. Fraud teams need useful alerts. Attackers try to learn controls. Customer-support staff handle blocks, and regulators or auditors may require evidence about outcomes.

A highly sensitive detector catches more fraud but blocks more legitimate transfers. Those false positives may fall hardest on travelers, small businesses, shared households, or people with irregular income. Revealing every detection rule could weaken security, yet giving no reason leaves customers unable to correct an error.

The team uses layered controls. Low-risk anomalies prompt a confirmation through a trusted channel. Medium-risk cases pause briefly for trained review. High-risk patterns trigger stronger account protection. The customer receives a useful category such as “new device and unusual recipient,” without a detailed recipe for bypassing the detector. Support staff see verified account context and can escalate rather than improvising.

Metrics include prevented loss, false blocks, time to restore access, repeat blocks after successful verification, and outcomes across relevant customer patterns. The team tests whether its trusted channel is accessible to people who changed phone numbers or use assistive technology. A customer can reach a human without disclosing secrets to a chatbot.

Here transparency is tiered rather than unlimited. The customer gets enough to understand and act; investigators get more detail under controlled access. Accountability remains with the payment service even when a vendor supplies the model.

A city wants a multilingual chatbot to answer questions about permits, waste collection, and emergency housing. The benefit is round-the-clock access. The risks differ by topic. A wrong recycling date is inconvenient; incorrect eligibility guidance may cause a family to miss urgent help.

The city classifies content by consequence. The chatbot can directly answer stable, low-impact questions from approved sources and show citations. For changing or high-impact programs, it summarizes official information, states the date, and routes people to qualified staff. It does not invent an eligibility decision from a conversation.

Residents can use phone and in-person channels. The city tests languages with community participants rather than assuming fluent-looking translations are accurate. It collects only the information required for the current task and does not reuse housing questions for law enforcement or unrelated profiling. Conversation logs have restricted access and short retention.

Accountability is public: a service owner maintains source pages, a language owner reviews important translations, and an operations owner monitors unanswered or escalated questions. The chatbot displays its automated role and offers a correction link. Reports describe coverage, known limitations, and improvement work without exposing private conversations.

The city launches one service category first. This narrows the blast radius and reveals whether residents can actually reach alternatives. A broad launch would create more apparent coverage but less confidence that remedies work.

Across the four cases, responsible choices came from changing the workflow:

• separate a convenience prediction from a high-impact priority decision;
• prevent data collected for personal support from becoming employee surveillance;
• give different levels of explanation to customers and security investigators;
• match automation and human support to the consequence of each information category.

No case relied on model accuracy alone. Each named purpose, stakeholders, unequal burdens, data boundaries, human authority, metrics, alternatives, and a path to correct or pause. Each also accepted a cost: more operations work, less analytics, occasional delay, or a narrower launch.

Those costs are not evidence that ethics blocks innovation. They are the resources required to provide the benefit responsibly. If an organization cannot staff appeals, secure data, or monitor important outcomes, the appropriate scope may be smaller than its technical capability.