{
  "schema_version": "1.0.0",
  "rubric_version": "1.0",
  "license": "CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)",
  "record": {
    "id": "C-0006",
    "slug": "safeguards-were-applied",
    "title": "Declared safeguards were applied during inference",
    "aliases": [
      "safeguard verification",
      "deployment mitigations"
    ],
    "status": "draft",
    "last_reviewed": "2026-09-23",
    "review_interval_days": 90,
    "steward": null,
    "provenance": {
      "drafted_by": "ai",
      "reviewed_by": []
    },
    "risk_flags": [],
    "flags": [],
    "one_liner": "Specified safety measures, such as input filters, output checks or monitoring, actually ran on the requests a deployed model served.",
    "summary": "Rules for deploying AI systems often require safeguards: filters on inputs, checks on outputs, or monitoring for misuse. A developer's statement that safeguards exist does not show that they ran on every request, or that the version checked is the one in production. Verifying application matters to regulators and to agreements that allow deployment on condition of mitigations. It is a positive claim, but it inherits the difficulties of verifying the served model and adds more. Safeguards are often separate software components whose configuration can change, and the verifier usually cannot see users' prompts or the provider's systems. Proposals use trusted execution environments to attest the serving stack, audits run inside such environments, and inspections. Whether a safeguard is effective is a separate question from whether it was applied.",
    "claim_class": "positive",
    "editors_synthesis": {
      "assessment": true,
      "markdown": "Safeguard application has been shown only in small research prototypes. Proof-of-guardrail demonstrates [[M-0023|safeguard attestation]] (R2) [[S-1500]]. Auditor-in-a-Box demonstrates plan-scoped monitoring, and its authors state that the demo's user data and plan execution are not actually secure [[S-1504]]. These prototypes can show that a declared safeguard ran, not that it works.\n\nSafeguard attestation runs the safeguard inside a trusted execution environment, whose hardware signs a measurement of its code with a commitment to each input and response [[S-1500]]. It builds on [[M-0008|TEE remote attestation]] (R2) and on knowing which model is served ([[C-0005]]). [[M-0025|Confidential multi-party verification]] (R2) can limit monitoring to a plan both parties sign [[S-1503]].\n\nThe proof-of-guardrail prototype runs on CPU enclaves in AWS and calls its guardrail model through an external API; in the authors' tests it detected modified guardrail code, attestations and responses [[S-1500]].\n\nEvery component that influences inference must be covered by the launch measurement [[S-0014]]. The prototype attests only the responses for which it offers attestation, so coverage of all traffic is not shown [[S-1500]]. Where hardware trust is unavailable, the sources fall back on inspections, audits and personnel-based layers [[S-0062]] [[S-0003]] [[S-0002]].",
      "text": "Safeguard application has been shown only in small research prototypes. Proof-of-guardrail demonstrates safeguard attestation (R2) [S-1500]. Auditor-in-a-Box demonstrates plan-scoped monitoring, and its authors state that the demo's user data and plan execution are not actually secure [S-1504]. These prototypes can show that a declared safeguard ran, not that it works. Safeguard attestation runs the safeguard inside a trusted execution environment, whose hardware signs a measurement of its code with a commitment to each input and response [S-1500]. It builds on TEE remote attestation (R2) and on knowing which model is served (The declared model is the one being served). Confidential multi-party verification (R2) can limit monitoring to a plan both parties sign [S-1503]. The proof-of-guardrail prototype runs on CPU enclaves in AWS and calls its guardrail model through an external API; in the authors' tests it detected modified guardrail code, attestations and responses [S-1500]. Every component that influences inference must be covered by the launch measurement [S-0014]. The prototype attests only the responses for which it offers attestation, so coverage of all traffic is not shown [S-1500]. Where hardware trust is unavailable, the sources fall back on inspections, audits and personnel-based layers [S-0062] [S-0003] [S-0002]."
    },
    "sources": [
      {
        "source": "S-0002",
        "supports": "deployment mitigations specified by inputs and outputs (filters, oversight checks); difficulty of choosing technical rules; whistleblower and interview layers",
        "locator": "Table 3; §4"
      },
      {
        "source": "S-0062",
        "supports": "AI developer inspections to check authorised code and implementation of evaluations and safeguards; software can be quickly modified or hidden",
        "locator": "Access-dependent methods; Table 1"
      },
      {
        "source": "S-0003",
        "supports": "third-party verification of developers' safety and security claims with deep, secure access; AI Assurance Levels",
        "locator": "abstract"
      },
      {
        "source": "S-0014",
        "supports": "TEEs for policy-adherent inference; launch measurement must cover all components; attestation-key holder can forge reports",
        "locator": "main argument; Limitations"
      },
      {
        "source": "S-0009",
        "supports": "TEE-based verifiable benchmarks keeping model IP and datasets confidential; prototype with Llama-3.1",
        "locator": "abstract"
      },
      {
        "source": "S-0012",
        "supports": "property attestation across training and inference on TDX + H100; formal model in Tamarin",
        "locator": "abstract"
      },
      {
        "source": "S-0018",
        "supports": "output cross-checks; inspector agents vulnerable to prompt injection",
        "locator": "architecture; open problems"
      },
      {
        "source": "S-1500",
        "supports": "proof-of-guardrail protocol, prototype on AWS Nitro Enclaves with an external guardrail API, tamper tests, opt-in attestation, jailbreak risk",
        "locator": "abstract; §3; §4.1; Table 1; Appendix A"
      },
      {
        "source": "S-1503",
        "supports": "monitoring scoped to a jointly signed plan and executed in an attested TEE",
        "locator": "abstract"
      },
      {
        "source": "S-1504",
        "supports": "Auditor-in-a-Box demo of plan-scoped monitoring; authors state user data and plan execution are not actually secure and the demo has not been stress-tested by a counterparty",
        "locator": "whole post"
      }
    ],
    "concepts": [
      "K-0006",
      "K-0004",
      "K-0019",
      "K-0005"
    ],
    "order": 6,
    "type": "claim",
    "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/",
    "source_file": "content/claims/safeguards-were-applied.md",
    "flags_all": [
      "ai-drafted"
    ],
    "body_markdown": "## Why it matters\nRules on deployment often take the form of required mitigations.\n\n- **Regulators and auditors.** RAND gives the example of deployment mitigations specified in terms of input and output data, such as filtering out some kinds of inputs or running oversight checks on outputs [[S-0002]]. Proposals for frontier AI auditing call for rigorous third-party verification of developers' safety and security claims, grounded in deep, secure access to non-public information [[S-0003]].\n- **International agreements.** Wasil and colleagues include inspections of AI developers to check that they run authorised code and properly implement model evaluations and safeguards [[S-0062]].\n- **Users' privacy.** A 2026 analysis argues that trusted execution environments could let an auditor check that deployed software follows an agreed policy, with access control enforced so that neither the service provider nor the auditor needs to pair users' identities with their data [[S-0014]].\n\n## Why it is hard\n- **Knowing what software is serving.** With TEEs, hardware attestation can show that specified software is running unaltered, but only for what the launch measurement covers [[S-0014]]. The same analysis notes that every component influencing inference behaviour must be covered, and that incomplete specifications could let an adversary undermine deployment integrity [[S-0014]]. It also notes that whoever holds the hardware's attestation key can produce valid reports, and that side-channel and physical attacks remain concerns [[S-0014]].\n- **Confidentiality.** [[I-0007|Attestable Audits]] runs benchmarks inside TEEs so that results are verifiable while model weights and test data stay confidential, with a prototype tested on Llama-3.1 [[S-0009]]. PAL\\*M defines properties across training and inference and attests them using confidential virtual machines with GPU support [[S-0012]]. Its authors model the protocol formally with the Tamarin prover [[S-0012]].\n- **Software changes quickly.** Wasil and colleagues note that code can be rapidly modified or hidden, which limits what periodic inspections of developers can show [[S-0062]]. Automated inspection has its own weaknesses: a low-trust system overview notes that inspector agents may be vulnerable to prompt injection and that their classification precision is uncertain [[S-0018]].\n- **Application is not effectiveness.** Proof-of-guardrail's authors note that an attested guardrail can still err or be jailbroken [[S-1500]]. RAND notes that deciding which technical rules actually achieve policy objectives remains difficult [[S-0002]]. Personnel-based layers such as whistleblower programmes and interviews are proposed for cases where technical evidence is unavailable [[S-0002]].",
    "body_text": "Why it matters Rules on deployment often take the form of required mitigations. - Regulators and auditors. RAND gives the example of deployment mitigations specified in terms of input and output data, such as filtering out some kinds of inputs or running oversight checks on outputs [S-0002]. Proposals for frontier AI auditing call for rigorous third-party verification of developers' safety and security claims, grounded in deep, secure access to non-public information [S-0003]. - International agreements. Wasil and colleagues include inspections of AI developers to check that they run authorised code and properly implement model evaluations and safeguards [S-0062]. - Users' privacy. A 2026 analysis argues that trusted execution environments could let an auditor check that deployed software follows an agreed policy, with access control enforced so that neither the service provider nor the auditor needs to pair users' identities with their data [S-0014]. Why it is hard - Knowing what software is serving. With TEEs, hardware attestation can show that specified software is running unaltered, but only for what the launch measurement covers [S-0014]. The same analysis notes that every component influencing inference behaviour must be covered, and that incomplete specifications could let an adversary undermine deployment integrity [S-0014]. It also notes that whoever holds the hardware's attestation key can produce valid reports, and that side-channel and physical attacks remain concerns [S-0014]. - Confidentiality. Attestable Audits runs benchmarks inside TEEs so that results are verifiable while model weights and test data stay confidential, with a prototype tested on Llama-3.1 [S-0009]. PALM defines properties across training and inference and attests them using confidential virtual machines with GPU support [S-0012]. Its authors model the protocol formally with the Tamarin prover [S-0012]. - Software changes quickly. Wasil and colleagues note that code can be rapidly modified or hidden, which limits what periodic inspections of developers can show [S-0062]. Automated inspection has its own weaknesses: a low-trust system overview notes that inspector agents may be vulnerable to prompt injection and that their classification precision is uncertain [S-0018]. - Application is not effectiveness. Proof-of-guardrail's authors note that an attested guardrail can still err or be jailbroken [S-1500]. RAND notes that deciding which technical rules actually achieve policy objectives remains difficult [S-0002]. Personnel-based layers such as whistleblower programmes and interviews are proposed for cases where technical evidence is unavailable [S-0002].",
    "addressed_by": [
      {
        "id": "M-0025",
        "title": "Confidential multi-party verification",
        "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/",
        "role": "supporting",
        "note": "Plan-scoped monitoring runs an agreed classifier over private usage records (S-1503)."
      },
      {
        "id": "M-0009",
        "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors",
        "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/",
        "role": "supporting",
        "note": "Could require deployment-time safeguards on approved devices (S-0035)."
      },
      {
        "id": "M-0012",
        "title": "Model identity attestation",
        "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/",
        "role": "supporting",
        "note": "Links an attested evaluation to the model later served (S-0009)."
      },
      {
        "id": "M-0023",
        "title": "Safeguard attestation",
        "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/",
        "role": "primary",
        "note": "Attests that a measured safeguard program (guardrail, filter, monitor) mediated the attested responses; coverage of all traffic is not established."
      },
      {
        "id": "M-0008",
        "title": "TEE remote attestation for AI workloads",
        "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/",
        "role": "supporting",
        "note": "Can attest that measured policy software, such as filters and logging, wrapped the model (S-0014). The evaluation variant is I-0007."
      },
      {
        "id": "M-0004",
        "title": "Zero-knowledge proofs of inference",
        "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/",
        "role": "supporting",
        "note": "Attestable proposes that a proof could show an agreed input classifier was applied; South et al. prove evaluation results."
      },
      {
        "id": "I-0007",
        "title": "Attestable Audits",
        "url": "https://trustbutveri.fyi/implementations/attestable-audits/",
        "role": "supporting",
        "note": "Attests that declared safety benchmarks were run on that model and what they scored; it does not attest runtime safeguards."
      },
      {
        "id": "I-0005",
        "title": "Attestable zero-knowledge inference prover",
        "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/",
        "role": "supporting",
        "note": "Proposed use: showing an agreed input classifier was applied."
      },
      {
        "id": "I-0012",
        "title": "Low-trust AI compute verification system overview",
        "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/",
        "role": "supporting",
        "note": "Screening checks that outputs are free of blacklisted uses, including with inspector agents."
      }
    ],
    "referenced_by": [
      {
        "id": "M-0025",
        "title": "Confidential multi-party verification",
        "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/"
      },
      {
        "id": "M-0009",
        "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors",
        "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/"
      },
      {
        "id": "M-0012",
        "title": "Model identity attestation",
        "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/"
      },
      {
        "id": "M-0023",
        "title": "Safeguard attestation",
        "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/"
      },
      {
        "id": "M-0008",
        "title": "TEE remote attestation for AI workloads",
        "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/"
      },
      {
        "id": "M-0004",
        "title": "Zero-knowledge proofs of inference",
        "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/"
      },
      {
        "id": "I-0007",
        "title": "Attestable Audits",
        "url": "https://trustbutveri.fyi/implementations/attestable-audits/"
      },
      {
        "id": "I-0005",
        "title": "Attestable zero-knowledge inference prover",
        "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/"
      },
      {
        "id": "I-0012",
        "title": "Low-trust AI compute verification system overview",
        "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/"
      }
    ]
  }
}