The claim is that specified safety measures, such as input filters, output checks or monitoring for misuse, actually ran on the requests a deployed model served.
Rules for deploying AI systems often require such safeguards. A developer's statement that safeguards exist does not show that they ran on every request, or that the version checked is the one in production. Verifying application matters to regulators and to agreements that allow deployment on condition of mitigations.
It is a positive claim, but it inherits the difficulties of verifying the served model and adds more. Safeguards are often separate software components whose configuration can change, and the verifier usually cannot see users' prompts or the provider's systems.
Proposals use trusted execution environments to attest the serving stack, audits run inside such environments, and inspections. Whether a safeguard is effective is a separate question from whether it was applied.
Safeguard application has been shown only in small research prototypes, which can show that a declared safeguard ran, not that it works.
Safeguard attestation (research demonstration, R2) runs the safeguard inside a trusted execution environment, whose hardware signs a measurement of its code with a commitment to each input and response 7. It builds on TEE remote attestation (operational use, R3) and on knowing which model is served. The proof-of-guardrail prototype demonstrates it on CPU enclaves in AWS and calls its guardrail model through an external API 7. In the authors' tests it detected modified guardrail code, attestations and responses 7. Confidential multi-party verification (R2) can limit monitoring to a plan both parties sign 8, and Auditor-in-a-Box demonstrates such plan-scoped monitoring, though its authors state that the demo's user data and plan execution are not actually secure 9.
Every component that influences inference must be covered by the launch measurement 4, and the prototype's attestation is opt-in, so coverage of all traffic is not shown 7. Where hardware trust is unavailable, the sources fall back on inspections, audits and personnel-based layers 1 2 3.
On this page
Why it matters
Rules on deployment often take the form of required mitigations.
- RAND's verification framework gives the example of deployment mitigations specified in terms of input and output data, such as filtering out some kinds of inputs or running oversight checks on outputs 1. Proposals for frontier AI auditing call for rigorous third-party verification of developers' safety and security claims, grounded in deep, secure access to non-public information 3.
- Wasil and colleagues' verification methods for international AI agreements include inspections of AI developers to check that they run authorised code and properly implement model evaluations and safeguards 2.
- A 2026 analysis of privacy-preserving monitoring argues that trusted execution environments could let an auditor check that deployed software follows an agreed policy, with access control enforced so that neither the service provider nor the auditor needs to pair users' identities with their data 4.
Why it is hard
- Knowing what software is serving. With TEEs, hardware attestation can show that specified software is running unaltered, but only for what the launch measurement covers 4. The same analysis notes that every component influencing inference behaviour must be covered, and that incomplete specifications could let an adversary undermine deployment integrity 4. It also notes that whoever holds the hardware's attestation key can produce valid reports, and that side-channel and physical attacks remain concerns 4.
- Confidentiality. Proof-of-guardrail takes the developer's agent as a secret input, so the agent stays private while the attestation shows that the public guardrail ran 7. For evaluations, Attestable Audits runs benchmarks inside TEEs so that results are verifiable while model weights and test data stay confidential, with a prototype tested on Llama-3.1 5 (see evaluation integrity).
- Software changes quickly. Wasil and colleagues note that code can be rapidly modified or hidden, which limits what periodic inspections of developers can show 2. Automated inspection has its own weaknesses: a low-trust system overview notes that inspector agents may be vulnerable to prompt injection and that their classification precision is uncertain 6.
- Application is not effectiveness. Proof-of-guardrail's authors note that an attested guardrail can still err or be jailbroken 7. RAND notes that deciding which technical rules actually achieve policy objectives remains difficult 1. Personnel-based layers such as whistleblower programmes and interviews are proposed for cases where technical evidence is unavailable 1.
The Oxford Martin AI Governance Initiative's report discusses on-chip logs for mobile AI-enabled devices, including weapons 10. Cryptographic commitments could be sent before the underlying operational data is revealed for later neutral analysis 10. Storage practicality and whether centrally stored logs reflect true operations remain unresolved 10.