Change Control for AI: Records That Survive an Audit

Regulated industries run formal change management. A change is proposed, assessed for impact, approved, tested, released and recorded, and an auditor can trace any production state back to the approval that authorized it. The process assumes changes are discrete events somebody initiated.

‍

AI systems violate that assumption in three directions at once. Retraining alters behavior without altering code, a provider can revise a hosted model without telling you, and a document added to a retrieval store changes outputs with no deployment at all. An auditor asking to see the change record for a behavior change frequently finds there is none, and that absence is a finding independent of whether anything went wrong.

‍

Why AI Breaks Change Management

‍

The controls are sound and the assumption underneath them has stopped holding. Separating the kinds of change makes the problem tractable, because only one of the three resembles what the process was designed for.

‍

Three Kinds of Change, One of Which You Initiate

‍

Changes you initiate include retraining, prompt revisions, threshold adjustments and tooling updates. Changes you consume include a provider revising or replacing a hosted model, often without notice. Changes that arrive by operation include additions to a retrieval corpus, drift in input distributions and behavior that varies across identical requests because the system is not deterministic.

‍

Existing change control handles the first category adequately. It has no mechanism for the second and treats the third as an incident rather than a change. Programs discovering this during an examination tend to have documented the ten percent of change they controlled. Categorizing systems by how much they change, alongside how much they matter, is what structured categorization should surface before an audit does.

‍

What the Auditor Is Testing

‍

Change management testing samples production states and asks for the authorization behind each one. The question is whether the organization can demonstrate that the system in use is the system that was approved. Where a model version differs from the validated one, the control has failed regardless of whether performance degraded, and the audit tests the record rather than the outcome.

‍

The One Regulator That Solved This

‍

Medical device regulation faced this tension earlier than most sectors, because an adaptive algorithm cleared for market can behave differently in clinical use. The response is worth studying even by organizations with no device in scope.

‍

Description, Protocol, Impact

‍

The Predetermined Change Control Plan, authorized by federal legislation in 2022 and codified in the Food, Drug and Cosmetic Act, lets a manufacturer pre-authorize specific bounded modifications in its original submission so those changes can be deployed later without a new filing. Final guidance specific to AI-enabled device software functions arrived in August 2025 and requires three components. A description of the planned modifications, a protocol governing how each is implemented and validated, and an assessment of the impact on safety and effectiveness.

‍

Bounded Rather Than Open-Ended

‍

The distinction that makes it work is that a plan authorizes specifically described changes made in a specifically described way with specifically described verification. Anything outside the described envelope still requires a new submission, and the most common deficiency is a plan drawn too broadly to constrain anything. Twenty-six devices held authorized plans as of May 2025, so the mechanism is real rather than theoretical, and international regulators have published aligned guiding principles.

‍

Borrowing the Pattern Without a Submission

‍

Nothing about the structure depends on being a device manufacturer. An enterprise operating AI under an internal change management regime can write the same three artifacts and gain the same benefit, which is permission to iterate inside a boundary somebody approved in advance.

‍

Write the Envelope Before the Change

‍

Define which modifications are pre-authorized for a given system, including retraining on refreshed data of the same type, threshold adjustment within a stated range and prompt revisions that do not expand the use case. State the validation each requires before release and the performance bounds outside which the change is rejected. A change inside the envelope becomes a logged event rather than a governance exercise.

‍

Define What Falls Outside It

‍

The envelope only has value if something sits outside. Expanding the intended use, introducing a new data category, changing the model family and removing human review should all trigger full reassessment. Recording those triggers explicitly prevents the argument about whether a particular change was in scope from happening after deployment. Framework obligations align here, since a substantial modification under the EU AI Act reopens conformity questions and the documentation obligations follow the modified system rather than the original one.

‍

Model Risk Functions Already Have Half of This

‍

Institutions running model risk management hold revalidation triggers, materiality tiering and independent challenge, which map onto the protocol component almost directly. What they generally lack is a pre-authorized envelope, so every change routes through full revalidation and the process becomes the reason teams avoid updating models. Borrowing the envelope concept reduces that friction without weakening the control, and defensibility under supervision depends on the boundary being documented rather than generous.

‍

The Changes You Do Not Control

‍

A predetermined plan governs modifications the organization makes. The harder problem for most regulated enterprises is a hosted model changing underneath a validated configuration, which no internal change process can pre-approve because nobody internal initiates it.

‍

Contractual Notice Is the Only Preventive Lever

‍

Advance notification of material model changes, version pinning where the provider offers it, and the right to test a new version before it becomes the default are contractual terms rather than technical controls. Enterprise agreements sometimes offer them and consumer tiers never do, which makes tier selection a change control decision. Negotiating this belongs alongside the other third-party terms that cannot be added later.

‍

Detect What You Cannot Prevent

‍

Where notice is unavailable, the fallback is regression testing on a schedule against a fixed suite, so a behavior change surfaces as a test result rather than as a customer complaint. Recording model version alongside every test run is what makes the comparison possible, and it separates a change in the model from a change in your own configuration. Recording it against the asset rather than a ticket is what an inventory built for reuse makes possible. Continuous approaches carry over here, since periodic testing leaves intervals uncovered in exactly the way an unannounced provider update exploits.

‍

What the Change Record Should Contain

‍

Three fields carry most of the audit value, and their absence is what turns a routine sample into a finding.

‍

• Model Version and Provider: Recorded at the time of every assessment, test and deployment rather than reconstructed later.
• Change Classification: Whether the change sat inside the pre-authorized envelope or triggered reassessment, with the reasoning attached.
• Named Approver: An individual rather than a function, current rather than historical.

‍

Validation evidence tied to the specific version completes the set. A test result that does not identify which version it exercised supports no claim about the version in production, which is the most common documentary weakness in AI change records. Keeping these against the asset rather than in a ticketing system is what makes reconstruction possible, and it is the same property visibility across systems depends on.

‍

Retention Follows the Obligation

‍

Change records inherit the retention period of the regime that governs the system, which for high-risk AI under European rules means the documentation is kept for ten years after the system reaches the market. A change log held in a tool with a shorter retention policy satisfies the process and not the obligation.

‍

Where This Meets Other Regimes

‍

The same envelope serves several obligations, which is the argument for building it once. A substantial modification under European rules reopens conformity assessment. Revised model risk management guidance expects documented revalidation triggers. Quality management standards require controlled change as a clause rather than a practice, and the management system standard treats it as an auditable requirement. Each wants the same three artifacts presented differently, and compliance readiness assessed per requirement is what lets one record answer all of them.

‍

The practical sequencing is to define the envelope per system rather than per framework, then map the envelope to each applicable obligation. Organizations doing the reverse produce three change processes and reconcile them before each audit.

‍

Approve the Boundary, Not Each Change

‍

Change control designed for discrete releases cannot govern systems that change continuously, and the answer is not to abandon the control. Pre-authorizing a bounded set of modifications, stating what falls outside it, recording model version against every assessment and detecting provider-side changes you cannot prevent produces a record that survives a sample. Medical device regulators arrived at that structure first and published it, which makes it available to everyone else. Kovrr's AI Security and Governance Platform keeps version, classification and approver against each asset, so the change history exists before anyone asks for it.

‍

To see whether your AI systems in production match the versions that were assessed, book a demo mapped to your own estate.