Methodology

Scope

This site maps the technical means of verifying claims about AI hardware and software, such as that compute runs inference rather than training, or that chips are where they are declared to be. Inspections, laws, audits and human intelligence are out of scope, except where a technical mechanism makes them verifiable. It is an educational reference, not an audit, certification or endorsement.

Record types

• Claims are properties someone might want verified. Negative claims, which assert that something is absent, are generally harder to verify than positive ones.
• Mechanisms are generic techniques, such as sampled inference recomputation.
• Implementations are specific prototypes, products or proposed architectures that realise mechanisms.
• Concepts, organizations and sources support the others.

A mechanism or implementation links to each claim it helps verify, in one of two roles. Primary means it is aimed directly at the claim. Supporting means it contributes to verifying the claim, for example as a building block for a primary mechanism or as partial evidence.

Readiness

Each mechanism and implementation has a readiness level from R0 (idea) to R3 (deployment-ready), assessed for its verification use under rubric v1.0. The Readiness levels page defines each level, lists the rules for assigning them, and shows the records at each level.

Properties

Each mechanism and implementation has five properties. They appear in its sidebar and can be used as filters on the Browse page.

Threat model

How far the design trusts the party being checked (the prover).

• Cooperative prover. Assumes the prover is honest but wants to demonstrate it.
• Semi-trusted prover. Tolerates some deviation but relies on parts of the prover's stack being trustworthy.
• Adversarial prover. Designed to hold even if the prover actively tries to cheat, within stated assumptions.

Adversarial evaluation

The strongest published attempt to break it, judged for its verification use.

• None. No published adversarial analysis.
• Analysis. Published security analysis or argument, no practical red-teaming.
• Red-teamed. Attacked in practice by the developers or collaborators.
• Independent red-team. Attacked in practice by a party independent of the developers.

Hardware needed

What hardware it needs beyond what already ships.

• None. Software or cryptography only.
• Existing features. Uses features already present in shipping hardware (e.g. TEEs, counters).
• Retrofit device. Needs an added device or sensor that can be installed on existing infrastructure.
• New chip design. Needs changes to future chip designs.

Prover cooperation

How much the party being checked must take part.

• Required. The prover must actively participate.
• Partial. Needs some access or cooperation, such as installing a device.
• Not required. Works without the prover's cooperation.

Confidentiality

Whether it keeps the prover's weights, data and code from the verifier.

• Preserving. Designed not to reveal weights, data or code to the verifier.
• Partial. Reveals some sensitive information, or protects it only under extra assumptions.
• Revealing. Requires the verifier to see sensitive information.

Flaws and blockers

Flaws. Only published or publicly demonstrated flaws are listed.

• Each is a demonstrated attack, a theoretical argument or an open question.
• Critical severity means the flaw defeats the main claim under the stated threat model.
• Each has a status (open, mitigated or disputed), with any response from the authors.
• Flaws are described as classes of weakness at their sources' level of detail, never as operational instructions.

Blockers. A blocker is an obstacle to the next level or to real use. When the obstacle is another mechanism's immaturity, the blocker links to it.

Citations

Every factual statement cites a public source. Sources are tiered:

Tier	Covers	Use
A	Peer-reviewed papers, standards, official government publications	Any claim
B	Preprints, technical reports, official documentation, public code (pinned to a version)	Any claim, except that a developer's own material about its own system supports only "X reports…" statements, and the record is flagged provider-reported
C	Expert blogs, talks and forum posts by the authors of the work	Descriptions of their own proposals and attributed views, but not "X is broken" or "X is secure"
D	Unpublished documents, link-shared drafts, personal communications, private chats	Never cited

• Exceptional claims, that a mechanism is broken, secure or deployed, need tier A or B sources or public reproducible code.
• Conclusions that no single source states appear only in assessment fields: readiness rationales, flaws, blockers and claims' editors' syntheses.
• Unpublished expertise enters only once published, or later as a signed, dated expert note that can support assessments but not stand-alone facts.

Neutrality

• Neutral voice. Records describe; they don't promote.
• Developers' claims about their own systems are attributed ("X reports…"), and the record is flagged provider-reported.
• Conflicts of interest. Contributors disclose affiliations, and may propose but not approve assessments of their own organization's work.
• Right of reply. A developer can dispute an assessment. The record then shows disputed, with both positions attributed, until an editor responds.

Provenance

Every record was drafted with AI assistance from public sources. An independent AI verifier then checked each factual sentence against its cited source, corrected errors and recorded open questions for the maintainer. The small print at the foot of each record gives the date of that source check and says whether a human expert has reviewed the record. Until experts have reviewed the records, the site footer marks the whole site as a draft.

Governance

• The maintainer is editor-in-chief: they merge changes, own the rubric and settle disputes.
• Editing is by invitation, through reviewed pull requests.
  • Cited factual edits need one approval.
  • Judgment edits (readiness, flaws, blockers and syntheses) need the maintainer's approval and a written rationale.
  • Rubric changes are announced with a comment period before they take effect.
• Freshness. Each record has a review interval, usually 90 days, and shows a review overdue flag when it lapses. The Status page lists overdue records.

Personal information

People appear only as authors of public sources, with names as published. To request a correction or removal, see Contribute.