AI Assurance

How AI Assurance Works

With AI Assurance, the first question is not:
“Is this AI trustworthy?”
Such a statement would be too general.

Instead, it must first be clarified:

• Which system is being considered?
• For what intended use?
• Under what conditions?
• Which properties are relevant?
• Which risks must be considered?
• Which statements are to be made about the system?
• What evidence can support these statements?

Only then can it be examined how well a statement about the system is actually substantiated.

From Claim to Evidence

A provider could, for example, claim:

Our AI system generates reliable technical documentation.

This statement is initially a claim.
For assurance, it must be clarified what reliable means in this context.

This could result in more specific statements:

Technical statements agree with the cited sources.
Generated commands are valid for the specified software version.
Mandatory work steps are not omitted.
Uncertain statements are identified as such.

For each of these statements, the next question can be:
What evidence do we need to support it?

This creates a chain:

Requirement or claim
        ↓
testable property
        ↓
suitable testing method
        ↓
evidence
        ↓
evaluation
        ↓
justifiable statement

The final step is important.
A test result does not automatically say more than what was actually tested.

Evidence Has Different Strengths

Not every piece of information has the same evidentiary value.
A language model could, for example, check a generated command itself once again and reach the conclusion:

The command is correct.

That is evidence.
Its evidentiary value, however, is limited.
A second model could independently check the same command. This creates additional evidence.
Official documentation could confirm that the option used exists.
A test on the operating system actually intended for use could additionally show that the command works under documented conditions.
These checks answer different questions and have different evidentiary value.
Assurance therefore does not simply require more evidence.
It requires suitable evidence for the respective statement.

Evidence Is Not Proof of Everything

Suppose a generated command was successfully tested on OpenBSD 7.9.
This provides justification for saying:

The command worked under the documented test conditions on the tested OpenBSD 7.9 system.

It does not automatically follow that:

The command works on every OpenBSD version.

Nor:

The command works under every possible configuration.

And certainly not:

All technical statements in the generated article are correct.

Assurance must therefore preserve the boundary between observed evidence and the statement derived from it.

Trust and Trustworthiness

AI Assurance is not intended simply to create trust.
People can trust a system even though that trust is unjustified.
Conversely, they can distrust a well-examined system.
The more important question is therefore whether trust can be justified through suitable evidence.
A system does not become trustworthy by appearing convincing.
Trustworthiness must relate to relevant properties of the system and its intended use.
Assurance attempts to make these properties examinable and the corresponding evidence traceable.

Limitations Are Part of the Result

A good assurance result does not only describe what works.
It also describes what was not examined.

For example:

Examined:
- OpenBSD 7.9
- amd64
- Standard installation
- 120 defined reference cases

Not examined:
- OpenBSD 7.8
- arm64
- customer-specific kernel modifications
- unknown future configurations

These limitations do not weaken the examination.
They make visible which statement the available evidence actually supports.
Assurance that conceals its limitations may create more trust, but not better-justified trust.

AI Evaluation Provides Evidence

AI Evaluation is an important component of AI Assurance.
An evaluation could, for example, show:

97 of 100 test cases met criterion A.

That is a result.
For assurance, further questions must then be asked:

Are these 100 test cases representative?
Which cases are missing?
How were the results evaluated?
What uncertainty does the measurement have?
What are the consequences of the three errors?
Under what conditions was the testing performed?

Evaluation therefore generates evidence.
Assurance considers which statements can be justified by this evidence.

Passing a Benchmark Is Not General Assurance

Benchmarks can provide important information about a model.
A model can, for example, achieve very good results on a test set.
However, this does not automatically mean that it is suitable for a specific use case.
A benchmark can examine different tasks, data, or risks than the actual application.
Assurance must therefore always consider the intended use.
The question is not only: “How good is this model?”
But: “What evidence do we have that this system is suitable for this task under these conditions?”

Assurance Concerns the System, Not Just the Model

An AI model is often only one component of a larger system.
An application can additionally use:

• System prompts
• Context Engineering
• Retrieval systems
• Databases
• external tools
• APIs
• rule sets
• filters
• human approvals
• downstream processing

A highly capable model can produce unreliable results in a poorly designed overall system.
Conversely, suitable controls and checks can limit certain weaknesses of a model.
AI Assurance should therefore consider the system context relevant to the use case.

Assurance Is Not a One-Time Examination

AI systems can change.
A provider can update a model.
A knowledge source can be changed.
A system prompt can be modified.
New tools can be added.
A retrieval system can return different documents.
The intended use can change.
As a result, existing evidence can also lose its evidentiary value.
Assurance must therefore consider the conditions under which the evidence was generated and which changes make renewed examination necessary.

Independence Changes the Evidentiary Value

An organization can examine its own AI system.
That is valuable and often necessary.
For certain statements, however, an independent examination can have additional evidentiary value.
Independence is not a binary state.
An external examiner can, for example, be independent of the development team but use the same reference data.
Two language models from different providers can appear independent and still produce similar errors because of comparable training data or evaluation patterns.

Therefore, the question should not only be: “Who performed the examination?”

But also: “How independent are the information and testing foundations actually used?”

Assurance Encompasses More Than Technical Correctness

Depending on the use, properties beyond technical performance can be relevant.

These can include, for example:

• Safety
• Privacy
• Robustness
• Fairness
• Transparency
• Traceability
• legal requirements
• organizational responsibilities

Which of these must be considered depends on the specific system and its area of use.
AI Assurance is therefore not a single universal testing method.
It is a framework for the question of which properties must be assessed with justification and which evidence is required for this.

Why AI Assurance Has Practical Effects

• Claims become testable: General statements become specific properties.
• Evidence is assigned: It becomes visible which examination actually supports which statement.
• Limitations are preserved: Areas that were not examined are not silently treated as having been examined.
• Evaluation gains context: Test results are considered together with conditions and risks.
• Changes become relevant: Model, context, or system changes can make renewed examination necessary.
• Different testing methods can be combined: Technical tests, documentation, and human assessments can provide different evidence.
• Trust can be better justified: What matters is not whether a system appears convincing, but what traceable evidence is available.

Typical Example

A company wants to use a language model to suggest changes to server configurations.

The claim:

The system generates secure configurations.

is too general for assurance.
Instead, specific properties are defined.

For example:

Suggested options exist in the software version being used.
Existing project-specific security rules are followed.
Changes are not automatically executed in production.
Uncertain or unverifiable assumptions are made visible.

Different checks are used for these properties.
Configuration options are checked against documentation and test systems.
Rules are checked automatically.
Known problem cases are used as repeatable evaluations.
Changes with high impact require human approval.
The results are documented together with the model version, context, test conditions, and known limitations.

The result is then not:

The AI is safe.

But, for example:

Under the described conditions, the defined properties were examined using the documented methods.
For the examined cases, the described evidence is available.
No corresponding statement can be derived from this for configurations that were not examined.

That is less spectacular.
But considerably more robust.

Distinction

• AI Assurance concerns the generation, evaluation, and communication of evidence from which justified statements about relevant properties and the use of an AI system can be derived.
• AI Evaluation is an important component of it. It examines the behavior and results of an AI system using defined criteria.
• Verification and Validation provide more general concepts for examining whether requirements are met and whether a system is suitable for its intended purpose.
• AI Governance encompasses organizational rules, responsibilities, policies, and decision-making structures around AI. Assurance can support governance, but is not identical to it.
• AI Safety concerns the prevention and limitation of harm caused by AI systems. Safety can be a property for which evidence is required as part of AI Assurance.
• Compliance considers adherence to specific legal, regulatory, or normative requirements. Examining and documenting compliance can be part of assurance, but does not automatically cover all properties relevant to a use case.

Further Reading

Related Concepts

→ AI Evaluation
→ Verification and Validation
→ Probabilistic
→ Context Engineering
→ Observability
→ Resilience