What model governance is

Model governance is sometimes mistaken for a policy document or an approval meeting. Those can be parts of it, but governance is the working system around the model. It connects choices made during proposal, design, testing, release, operation, change, and retirement.

A useful governance system includes:

1. Inventory: a record of AI systems, purposes, data, dependencies, versions, users, and status.
2. Classification: a proportionate risk level based on impact, scale, sensitivity, reversibility, and uncertainty.
3. Ownership: a named person accountable for the system and named specialists who contribute evidence or operate controls.
4. Documentation: an understandable record of intended use, prohibited use, performance, limitations, approvals, and known issues.
5. Evaluation and monitoring: evidence before release and signals after release.
6. Change control: rules for deciding which changes require review, testing, approval, or rollback.
7. Incident response: a route to contain harm, notify the right people, recover, and learn.
8. Retirement: a controlled end to access, integrations, data retention, and outstanding obligations.

These parts form a loop. An incident creates a new failure example. That example enters an evaluation set. A change intended to fix it goes through change control. Monitoring checks whether the fix works in production. Documentation records the result. Governance is effective when learning travels around this loop.

The word “model” can hide too much. A deployed AI feature may include a provider model, system prompt, retrieval index, user interface, tools, business rules, human reviewers, and downstream actions. A chatbot can become risky because its knowledge base is stale even when the underlying model has not changed. A ranking tool can become unfair because the applicant population changed. An agent can cause harm because it has excessive permissions.

The practical unit of governance is therefore the AI system in context. Record the model, but also record the surrounding data and workflow. Ask what decisions or actions the whole system enables.

This distinction also prevents responsibility from disappearing into the supply chain. A vendor should provide evidence about its component. The deploying organization still chooses the use, configures access, integrates data, presents outputs, and decides how people can challenge outcomes. Each party governs the choices it controls.

Good governance is not one heavyweight process for every use. A private brainstorming assistant and an automated benefits decision should not face identical controls. Uniform bureaucracy can encourage teams to hide low-risk experiments, while weak controls can expose people to serious harm.

A first-pass risk view considers:

• the consequence of a wrong or manipulated output;
• whether a person’s rights, safety, work, money, or access are affected;
• the sensitivity of input and output data;
• the number of people and speed of operation;
• whether a result is reversible and detectable;
• how much authority the system has;
• whether meaningful human review exists;
• how novel or uncertain the use is.

Lower-risk systems may need a simple inventory entry, basic testing, an owner, and periodic review. Higher-risk systems may need independent validation, formal approval, subgroup testing, strict access controls, continuous monitoring, human decision gates, and a rehearsed incident plan. Proportionate means the strength of the control follows the risk, not the prestige of the project.

No committee, model card, or test suite can prove that an AI system will never fail. Governance creates disciplined decisions under uncertainty. It makes assumptions explicit, requires evidence, preserves a trail, and gives people authority to pause unsafe operation.

Governance also differs from ethics and compliance, though they overlap. Ethics helps a team reason about what it should do and how values conflict. Compliance identifies applicable obligations. Governance turns choices and obligations into ownership, controls, records, and recurring practice. A system can satisfy a checklist yet remain harmful if the checklist asks the wrong questions. It can also have thoughtful intentions yet remain unmanaged if nobody implements them.

For any AI system, sketch eight stages: propose, classify, design, evaluate, approve, operate, change, and retire. At each stage write three things: the decision being made, the person accountable, and the evidence required. Missing entries reveal governance gaps.

For example, “approve” may require the product owner to accept residual risk using evaluation results, privacy review, and an operations plan. “Operate” may require the service owner to review weekly quality signals and respond to alerts. “Change” may require engineering to compare versions and obtain reapproval when a threshold is crossed.

The map should be short enough to use. A beautiful framework that does not affect access, release, monitoring, or response is ceremony rather than control.