Model-change swimlane
| Stage | Owner | Evidence | Stop condition |
|---|---|---|---|
| Detect | Platform owner | Official model catalog or release notice | Unverified model or availability |
| Scope | Legal ops | Affected workflows, users, integrations and data classes | No accountable workflow owner |
| Contract | Privacy/procurement | Terms, retention, region, price and lifecycle | Material term remains Unknown for the proposed data |
| Regression | Subject-matter reviewer | Paired outputs and scored failure classes | Material-error guardrail fails |
| Pilot | Workflow owner | Bounded cohort, logs and support path | Permission, export or cost boundary fails |
| Promote | Named approver | Decision, default change and rollback window | Evidence cannot be reproduced |
Freeze the current baseline
Preserve model ID, product surface, account, region, prompt, sources, tools, permissions, output, total run cost, reviewer corrections and elapsed review time. If the provider uses a moving alias, record the resolved version when the platform exposes it.
Use a legal regression set
Include an ordinary task, a long document, conflicting sources, an adversarial instruction inside a source, a tool failure and a required abstention. Expected results should identify controlling authority, material issues, prohibited actions and reviewer checkpoints. Style cannot compensate for an unsupported authority or missed protection.
Measure completed-task economics
Compare tokens or seats, cache, tool calls, retries, integration cost and reviewer minutes. A lower list price is not a cheaper completed task. Preserve both the mean and the worst material case when volume is small.
Define rollback before promotion
Rollback when the new model introduces an unsupported authority, misses a material issue, expands permissions, changes the output destination, breaks export, exceeds the cost boundary or increases review time beyond the agreed threshold. Keep the prior model and configuration available until the defined observation window closes.
Close the loop in the change ledger
Record source event, affected assets, owner, exact change, baseline, result, decision, unresolved unknowns and next review date. NIST’s voluntary govern-map-measure-manage structure is a useful organizing frame, not a certification or a substitute for the firm’s legal review.
FAQ
Should legal ops adopt a model as soon as it launches?
No. A release should trigger scoping and a controlled regression test. The production default changes only after the agreed guardrails pass.
What belongs in a legal AI regression set?
Use representative and adversarial cases with expected sources, material issues, prohibited actions, tool failures and required abstentions.
When can the prior model be removed?
After the new version passes the pilot and observation window, evidence remains reproducible, and the team has accepted the new rollback and lifecycle plan.
Sources checked
- AI Risk Management Framework, checked 2026-09-03.
- Gemini API models, checked 2026-09-03.
- Fable 5.1 model overview, checked 2026-09-03.
- OpenAI API model catalog, checked 2026-09-03.
Operational information, not legal advice. Verify current terms, account configuration and applicable professional duties before use.