{
  "schema_version": "1.0.0",
  "rubric_version": "1.0",
  "license": "CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)",
  "record": {
    "id": "C-0009",
    "slug": "weights-have-not-left",
    "title": "Model weights or data have not left the facility",
    "aliases": [
      "no weight exfiltration",
      "data containment"
    ],
    "status": "draft",
    "last_reviewed": "2026-09-23",
    "review_interval_days": 90,
    "steward": null,
    "provenance": {
      "drafted_by": "ai",
      "reviewed_by": []
    },
    "risk_flags": [],
    "flags": [],
    "one_liner": "No copy of specified model weights or sensitive data has left a designated facility through networks, physical media or other channels.",
    "summary": "Agreements and security commitments may require that certain model weights stay inside a secured facility, for example to protect them against theft by well-resourced attackers. Verification itself creates a related concern, since monitoring equipment could leak the prover's secrets. Verifying that weights have not left is a negative claim over every possible channel: network links, physical media, people and physical side channels. Approaches include checking that outgoing traffic is explained by legitimate computation so that hidden data stands out, capping the bandwidth of external links, suppressing side channels, and physical security with inspection. Checking inference outputs against a reference cut the information an attacker could hide in responses to under 0.5% in one test; in an independent study, an attacker who controlled the prompts roughly doubled the leak. Physical and insider channels remain harder to bound.",
    "claim_class": "negative",
    "editors_synthesis": {
      "assessment": true,
      "markdown": "Only the network channel has a demonstrated (R2) bound, and only for inference outputs. Physical side channels rely on proposed (R1) designs, and insiders and physical media on facility security.\n\n[[M-0024|Bounding unexplained information]] (R2) measures how much of a facility's output the declared computation cannot explain. [[M-0001|Sampled recomputation]] (R2) supplies the reference, [[M-0002|deterministic inference]] (R2) would remove the tolerance that hidden data can exploit [[S-0020]], and [[M-0013|network taps]] (R1) record the traffic. Caps on external bandwidth ([[C-0008]]) bound the total that can leave [[S-1508]]. [[M-0022|Side-channel suppression]] (R1) targets physical leakage [[S-0038]], and [[I-0010|RAND's secure inference data centre]] (R1) is designed to protect weights against state-backed attackers [[S-1510]].\n\nChecking inference outputs cut the information an attacker could hide to under 0.5% on a 30-billion-parameter model, at a false-positive rate below 0.01% [[S-0015]]. In an independent study, an attacker who controlled the prompts roughly doubled the leak per token [[S-1507]].\n\nPeople and physical media are left to facility security [[S-1610]] and personnel measures such as whistleblowing [[S-0002]]. RAND's design does not describe how an outside party would verify it [[S-1510]]. The designs aim for a stated bound on leakage, not an absolute guarantee [[S-0018]].",
      "text": "Only the network channel has a demonstrated (R2) bound, and only for inference outputs. Physical side channels rely on proposed (R1) designs, and insiders and physical media on facility security. Bounding unexplained information (R2) measures how much of a facility's output the declared computation cannot explain. Sampled recomputation (R2) supplies the reference, deterministic inference (R2) would remove the tolerance that hidden data can exploit [S-0020], and network taps (R1) record the traffic. Caps on external bandwidth (Communication between compute groups is bounded) bound the total that can leave [S-1508]. Side-channel suppression (R1) targets physical leakage [S-0038], and RAND's secure inference data centre (R1) is designed to protect weights against state-backed attackers [S-1510]. Checking inference outputs cut the information an attacker could hide to under 0.5% on a 30-billion-parameter model, at a false-positive rate below 0.01% [S-0015]. In an independent study, an attacker who controlled the prompts roughly doubled the leak per token [S-1507]. People and physical media are left to facility security [S-1610] and personnel measures such as whistleblowing [S-0002]. RAND's design does not describe how an outside party would verify it [S-1510]. The designs aim for a stated bound on leakage, not an absolute guarantee [S-0018]."
    },
    "sources": [
      {
        "source": "S-1610",
        "supports": "importance of protecting frontier weights; 38 attack vectors; five security levels; adversaries up to nation-states; comprehensive defences",
        "locator": "summary"
      },
      {
        "source": "S-0015",
        "supports": "steganographic exfiltration via inference server responses; security game; <0.5% exfiltratable information at <0.01% FPR on MoE-Qwen-30B; >200x adversary slowdown",
        "locator": "abstract"
      },
      {
        "source": "S-1507",
        "supports": "prompt-controlling adversary roughly doubles bits leaked per token; slowdown falls to 60–118x",
        "locator": "abstract; results"
      },
      {
        "source": "S-0005",
        "supports": "strong security to keep weights in a data centre plus close monitoring",
        "locator": "data-centre security discussion"
      },
      {
        "source": "S-0018",
        "supports": "verifier aims to exfiltrate prover secrets; only commitments leave the facility; egress explainable by ingress; memory wiping; side-channel target",
        "locator": "threat model; architecture; open problems"
      },
      {
        "source": "S-0038",
        "supports": "physical side channels can bypass network monitoring; defences",
        "locator": "channels of concern; defences"
      },
      {
        "source": "S-0002",
        "supports": "confidentiality as protecting models, data and code from theft; whistleblower and interview layers",
        "locator": "§1; §4"
      },
      {
        "source": "S-0031",
        "supports": "active taps scrubbing headers against covert channels on front-end links",
        "locator": "frontend vs backend"
      },
      {
        "source": "S-1508",
        "supports": "egress limits cap what can be stolen",
        "locator": "§5.1"
      },
      {
        "source": "S-0020",
        "supports": "approximate output matching leaves degrees of freedom, including steganography, that covert adversaries can exploit",
        "locator": "abstract"
      },
      {
        "source": "S-1510",
        "supports": "secure inference data centre design against state-backed attackers; no external verification path described",
        "locator": "Summary; ch. 1"
      }
    ],
    "concepts": [
      "K-0022",
      "K-0013",
      "K-0021",
      "K-0014",
      "K-0012"
    ],
    "order": 9,
    "type": "claim",
    "url": "https://trustbutveri.fyi/claims/weights-have-not-left/",
    "source_file": "content/claims/weights-have-not-left.md",
    "flags_all": [
      "ai-drafted"
    ],
    "body_markdown": "## Why it matters\nModel weights are a primary target for theft [[S-1610]].\n\n- **Security of frontier models.** RAND researchers write that protecting frontier models from theft and misuse will become more important as they grow more capable [[S-1610]]. They identify 38 distinct attack vectors, and define five security levels for defending against actors ranging from opportunistic criminals to well-resourced nation-states [[S-1610]].\n- **Agreements.** Scher and Thiergart describe one approach as using strong security to prevent model weights from leaving a data centre, and then monitoring that data centre closely [[S-0005]].\n- **Verification itself.** RAND's framework treats confidentiality, meaning protection of models, data and code from unauthorised access and especially theft, as a requirement on verification [[S-0002]]. A low-trust system overview assumes the verifier may try to exfiltrate the prover's secrets [[S-0018]]. It therefore sends only cryptographic commitments out of the facility, and checks challenged data only in an auditing environment inside the prover's facility [[S-0018]].\n\n## Why it is hard\nThe claim must hold for every channel. The RAND weight-security study's 38 attack vectors span far more than network transfer [[S-1610]].\n\n- **Hidden data in ordinary outputs.** An attacker who controls an inference server could hide weights inside ordinary model responses using steganography [[S-0015]]. Rinberg and colleagues formalise this as a security game and verify outputs against a reference [[S-0015]]. On a 30-billion-parameter mixture-of-experts model, their detector reduced exfiltratable information to under 0.5% at a false-positive rate below 0.01%, which they equate to a slowdown of more than 200 times for an adversary [[S-0015]]. An independent study found that an attacker who chooses the prompts roughly doubles the bits leaked per token, cutting the slowdown to 60–118 times [[S-1507]].\n- **Capping egress.** Rinberg and colleagues note that egress limits cap theft directly: if only 10 GB leaves a data centre, an adversary cannot steal more than 10 GB [[S-1508]]. Bounding a facility's links in this way is the subject of [[C-0008]].\n- **Explaining all egress.** Making all egress traffic explainable by its ingress is an open design question, including which engineering compromises it would require [[S-0018]]. Active network taps on front-end links could scrub headers to reduce covert channels [[S-0031]].\n- **Physical side channels.** These offer routes around network monitoring altogether. Proposed defences include shielding, jamming, inspection and filtering [[S-0038]], with a design target of kilobits per second rather than zero [[S-0018]]. Residual copies in memory are addressed by memory wiping [[S-0018]].\n- **Insiders and physical media.** These are outside the reach of network-level mechanisms. RAND's verification framework includes whistleblower programmes and personnel interviews among its layers [[S-0002]], and the weight-security study treats defence as requiring comprehensive measures rather than isolated ones [[S-1610]].",
    "body_text": "Why it matters Model weights are a primary target for theft [S-1610]. - Security of frontier models. RAND researchers write that protecting frontier models from theft and misuse will become more important as they grow more capable [S-1610]. They identify 38 distinct attack vectors, and define five security levels for defending against actors ranging from opportunistic criminals to well-resourced nation-states [S-1610]. - Agreements. Scher and Thiergart describe one approach as using strong security to prevent model weights from leaving a data centre, and then monitoring that data centre closely [S-0005]. - Verification itself. RAND's framework treats confidentiality, meaning protection of models, data and code from unauthorised access and especially theft, as a requirement on verification [S-0002]. A low-trust system overview assumes the verifier may try to exfiltrate the prover's secrets [S-0018]. It therefore sends only cryptographic commitments out of the facility, and checks challenged data only in an auditing environment inside the prover's facility [S-0018]. Why it is hard The claim must hold for every channel. The RAND weight-security study's 38 attack vectors span far more than network transfer [S-1610]. - Hidden data in ordinary outputs. An attacker who controls an inference server could hide weights inside ordinary model responses using steganography [S-0015]. Rinberg and colleagues formalise this as a security game and verify outputs against a reference [S-0015]. On a 30-billion-parameter mixture-of-experts model, their detector reduced exfiltratable information to under 0.5% at a false-positive rate below 0.01%, which they equate to a slowdown of more than 200 times for an adversary [S-0015]. An independent study found that an attacker who chooses the prompts roughly doubles the bits leaked per token, cutting the slowdown to 60–118 times [S-1507]. - Capping egress. Rinberg and colleagues note that egress limits cap theft directly: if only 10 GB leaves a data centre, an adversary cannot steal more than 10 GB [S-1508]. Bounding a facility's links in this way is the subject of Communication between compute groups is bounded. - Explaining all egress. Making all egress traffic explainable by its ingress is an open design question, including which engineering compromises it would require [S-0018]. Active network taps on front-end links could scrub headers to reduce covert channels [S-0031]. - Physical side channels. These offer routes around network monitoring altogether. Proposed defences include shielding, jamming, inspection and filtering [S-0038], with a design target of kilobits per second rather than zero [S-0018]. Residual copies in memory are addressed by memory wiping [S-0018]. - Insiders and physical media. These are outside the reach of network-level mechanisms. RAND's verification framework includes whistleblower programmes and personnel interviews among its layers [S-0002], and the weight-security study treats defence as requiring comprehensive measures rather than isolated ones [S-1610].",
    "addressed_by": [
      {
        "id": "M-0014",
        "title": "Bandwidth limits and compartmentalization",
        "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/",
        "role": "supporting",
        "note": "A cap on outgoing bandwidth bounds how much weight data can leave a facility in a given time (S-1508)."
      },
      {
        "id": "M-0024",
        "title": "Bounding unexplained information in outputs",
        "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/",
        "role": "primary",
        "note": "Bounds how much weight or other undeclared information can leave in checked outputs; does not close other channels."
      },
      {
        "id": "M-0002",
        "title": "Deterministic and bit-exact inference",
        "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/",
        "role": "supporting",
        "note": "Removes the tolerance margin that steganographic exfiltration could use (S-0020)."
      },
      {
        "id": "M-0012",
        "title": "Model identity attestation",
        "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/",
        "role": "supporting",
        "note": "The recomputation variant limits steganographic weight exfiltration through outputs (S-0015)."
      },
      {
        "id": "M-0013",
        "title": "Network taps and certifiers",
        "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/",
        "role": "supporting",
        "note": "Aims to make covert exfiltration of results through tapped links infeasible (S-1300)."
      },
      {
        "id": "M-0001",
        "title": "Sampled inference recomputation",
        "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/",
        "role": "supporting",
        "note": "Bounds how much information can be hidden steganographically in checked outputs. It is not a stand-alone defence against weight exfiltration (S-0015)."
      },
      {
        "id": "M-0022",
        "title": "Side-channel suppression for isolated facilities",
        "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/",
        "role": "supporting",
        "note": "Supports arguments that weights cannot leave by unmonitored physical routes."
      },
      {
        "id": "I-0002",
        "title": "DiFR (Divergence From Reference)",
        "url": "https://trustbutveri.fyi/implementations/difr/",
        "role": "supporting",
        "note": "Used as the estimator in a weight-exfiltration detection scheme (S-0015)."
      },
      {
        "id": "I-0010",
        "title": "RAND secure inference data center (SIDC) design",
        "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/",
        "role": "primary",
        "note": "Designed to keep weights and inference data inside the facility; the report does not describe how an external party would verify this."
      }
    ],
    "referenced_by": [
      {
        "id": "M-0014",
        "title": "Bandwidth limits and compartmentalization",
        "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/"
      },
      {
        "id": "M-0024",
        "title": "Bounding unexplained information in outputs",
        "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/"
      },
      {
        "id": "M-0002",
        "title": "Deterministic and bit-exact inference",
        "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/"
      },
      {
        "id": "M-0012",
        "title": "Model identity attestation",
        "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/"
      },
      {
        "id": "M-0013",
        "title": "Network taps and certifiers",
        "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/"
      },
      {
        "id": "M-0001",
        "title": "Sampled inference recomputation",
        "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/"
      },
      {
        "id": "M-0022",
        "title": "Side-channel suppression for isolated facilities",
        "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/"
      },
      {
        "id": "I-0002",
        "title": "DiFR (Divergence From Reference)",
        "url": "https://trustbutveri.fyi/implementations/difr/"
      },
      {
        "id": "I-0010",
        "title": "RAND secure inference data center (SIDC) design",
        "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/"
      },
      {
        "id": "C-0008",
        "title": "Communication between compute groups is bounded",
        "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/"
      },
      {
        "id": "O-0210",
        "title": "Intelligence Security Laboratories",
        "url": "https://trustbutveri.fyi/organizations/intelligence-security-laboratories/"
      },
      {
        "id": "O-0200",
        "title": "RAND",
        "url": "https://trustbutveri.fyi/organizations/rand/"
      }
    ]
  }
}