AI Change Impact Assessor
Treat an AI change as a controlled change: pick what changed — model version, system prompt, RAG source, tool permission, embedding model, guardrail, human oversight — and the risk of the system it touches, and get the deterministic reassessment level (none / doc-only / targeted regression / targeted revalidation / full / human approval), whether eval must rerun, and whether prior evidence goes stale. Expanding a permission or weakening oversight always needs human approval. A scoping aid, not a validated assurance system.
OUTPUT
TIME
Limitations — read before you rely on this
- This is a scoping aid, not a validated system, and not a change-control system of record. Reproduce the determination in your own quality system and document the rationale; a tool output is not a change record.
- It assesses one change at a time. Bundled or interacting changes (a new model plus a new retrieval config) can compound in ways a single-change lookup does not capture — assess the combination, not just the parts.
- The base levels and escalation steps are SPEQ judgement, not regulatory allowances. Your quality system may require more for a given change; treat the result as a defensible minimum to adapt, not a ceiling.
- It sizes the reassessment; it does not perform it. Which specific eval cases rerun and which evidence to refresh still require your change-impact analysis against the affected Contexts of Use.
WHAT THIS CALCULATES
The reassessment a change to an AI system requires, treating the change as a controlled change. From what changed — model version, system prompt, RAG source, tool permission, embedding model, guardrail, human oversight — and the risk tier of the system it touches, it returns a deterministic reassessment level (none through human approval), whether the evaluation suite must rerun, and whether prior qualification evidence goes stale. It turns "we tweaked the prompt" into a defensible change-control decision.
THE METHOD
level = ladder[ min( base_rank + tier_escalation, max ) ]; level = human-approval if the change is capability-expanding or oversight-weakening- base_rank
- the change type’s base position on the reassessment ladder (none · doc-only · targeted-regression · targeted-revalidation · full · human-approval)
- tier_escalation
- ladder steps added for the system’s risk tier: low +0, medium +0, high +1, critical +2
- ladder
- the ordered reassessment levels; the index selects the level
- max
- the top of the ladder (human-approval) — escalation is clamped here
- level
- the resulting reassessment level the change requires
Each change type’s base level and its eval-rerun / evidence-stale flags are SPEQ-authored policy expressing the epic’s change-impact flow, not regulatory constants. Two hard rules override the arithmetic: expanding a tool permission or reducing human oversight is pinned at human-approval regardless of tier. Monotonic: a higher tier never lowers the level. An unrecognized change fails safe to full.
THE INPUTS, AND WHAT THEY MEAN
- What changed
- The component that changed, chosen from the controlled AI change types (model version/provider, prompt, template, tool, permission, RAG source, retrieval/chunking/vector store, embedding model, guardrail, workflow, eval threshold, oversight). Each carries its own base impact and evidence consequences.
- Risk tier of the affected system
- The Context-of-Use risk tier of the system the change touches, from the risk classifier. It escalates the reassessment because the same prompt change on a critical decision-support agent matters more than on a low-risk drafting aid.
What reassessment does this AI change require?
Treat an AI change as a controlled change. Pick what changed and the risk of the system it touches, and SPEQ returns the reassessment level, whether the evaluation must rerun, and whether prior evidence goes stale. Expanding a tool permission or weakening human oversight always requires human approval — by rule.
The provider ships a new version of the same model. Behaviour can shift silently — rerun the evaluation regression.
The tier comes from the Context-of-Use risk classification (decision consequence, model influence, data sensitivity, change dynamics). Higher-risk systems escalate the reassessment a given change requires. A SPEQ planning aid, not a validated assurance system.
- Re-run evaluation suite: Yes
- Prior qualification evidence stale: Yes
Model version update is a targeted regression change, escalated to targeted revalidation because it affects a high-risk system.
HOW TO READ THE OUTPUT
- The level is the floor of what to do, not a ceiling. "Targeted regression" means at least rerun the affected eval cases; if your change-impact analysis finds broader effects, do more — the tool sizes the minimum defensible response.
- Capability expansion and oversight reduction are special. Adding a tool, widening a permission, or removing a review step increases what can go wrong, so they demand human approval regardless of the risk tier — the arithmetic cannot talk you out of that.
- Read the two flags together with the level. "Evidence stale = yes" means your prior qualification no longer speaks for the current configuration; leaving stale evidence in place is how an audit finds a validated record that describes a system that no longer exists.
- Risk-decreasing changes are still changes. Reducing a permission or increasing oversight lands low on the ladder, but it is documented, not invisible — confirm nothing depended on what you removed.
WORKED EXAMPLE
The model provider ships a minor version update to the LLM behind a high-risk regulatory copilot. Nothing else changed.
- What changed
- Model version update
- System risk tier
- High
RESULT
A "silent" provider version bump is not silent for assurance: on a high-risk system it escalates one step to targeted revalidation, the evaluation suite must rerun because behaviour can shift, and the prior qualification evidence is now stale. The defensible action is to rerun the bound eval cases against the new version and refresh the evidence before continuing to rely on the copilot.
REGULATORY BASIS
- ICH Q9(R1) — Quality Risk Management
- The risk-based principle behind escalating reassessment with the criticality of the system a change affects.
- ISPE GAMP 5 (2nd ed.) — A Risk-Based Approach to Compliant GxP Computerized Systems
- The change-control and periodic-review expectations for computerized systems that this maps onto AI-specific change types.
- FDA/EMA — Guiding Principles of Good AI Practice in Drug Development (Jan 2026)
- Frames model/lifecycle change management for AI in regulated development; see SPEQ’s Good AI Practice page.