# AI verification proposal

A proposal built with the Proposal Explorer of the AI Verification Tech Map (https://trustbutveri.fyi/), from its records of 2026-10-08. Interactive version: https://trustbutveri.fyi/explorer/?mechanisms=M-0002&implementations=M-0002:I-0016

How to read it: a claim is something one party wants to verify about another's AI hardware or software. A mechanism is a general technique for verifying claims; it is "aimed at" a claim when that is its direct purpose, and "supporting" when it contributes without being aimed at it. A claim is addressed when a mechanism in the proposal is aimed at it and is not excluded by the filters; addressed does not mean verified, so check that mechanism's readiness and open flaws. Readiness levels R0 to R4 describe one record's public evidence for its assessed use and are never combined. Definitions: https://trustbutveri.fyi/about/methodology/ (roles, properties and flaws) and https://trustbutveri.fyi/about/readiness/ (readiness levels).

## Filters

Filters apply to mechanisms only and describe the setting the proposal is for.

None set. Every mechanism on the map was available.

## Overview

One row per mechanism, read from its record. Open flaws: critical / significant / minor. The last three columns are the editors' reading of what the verifier sees.

| Mechanism | Readiness | Prover | Attack testing | Hardware | Open flaws | Weights | Inputs and outputs | Training data |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Deterministic and bit-exact inference / Batch-invariant inference kernels (Thinking Machines) | R2 | Cooperative | None | None | 0 / 0 / 0 | unspecified | unspecified | unspecified |

## Claims

No claims chosen.

## Mechanisms

### Deterministic and bit-exact inference

Open-source kernels from Thinking Machines Lab that make LLM outputs independent of batch size, adopted in vLLM and SGLang to give reproducible inference. ([Deterministic and bit-exact inference](https://trustbutveri.fyi/mechanisms/deterministic-inference/))

- Assessment: selected implementation [Batch-invariant inference kernels (Thinking Machines)](https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/).
- Readiness: R2 Demonstrated, assessed for exact recomputation of served outputs by a verifier, with a cooperating provider.
- Claims in this proposal: none of them.
- Threat model: cooperative prover. Hardware: none. Prover cooperation: required. Attack testing: none. Category: Cryptographic & computational.
- What the verifier sees: model weights unspecified; inputs and outputs unspecified; training data unspecified. This Explorer has no asset-specific exposure assessment for this implementation. Check its source and deployment assumptions.


## Properties

**No new hardware needed**

- Deterministic and bit-exact inference


## Attack testing

Published attempts to break a system, including those that found failures. Testing history does not establish that open flaws are resolved.


## Limits

**Family finding context**

- Context for Batch-invariant inference kernels (Thinking Machines); applicability depends on the finding's scope. Some kernels remain genuinely nondeterministic (open question, in Deterministic and bit-exact inference; https://trustbutveri.fyi/mechanisms/deterministic-inference/#flaw-1) [9]. The bit-exact work separates kernels that are deterministic but not batch-invariant from truly nondeterministic ones that use atomic functions. Some integer de-quantization kernels use atomic additions and remain nondeterministic, so exact replay needs backends that avoid them.
- Context for Batch-invariant inference kernels (Thinking Machines); applicability depends on the finding's scope. Cross-hardware replay relies on reverse-engineered, closed behaviour (open question, in Deterministic and bit-exact inference; https://trustbutveri.fyi/mechanisms/deterministic-inference/#flaw-2) [9][10]. Emulating one GPU's rounding on another requires reverse-engineering tensor-core arithmetic and modelling proprietary kernel choices. Hawkeye covers a subset of NVIDIA architectures and states that attention and other higher-level operations need further reverse engineering. For the bit-exact emulator, a proprietary Hopper kernel family is an open edge case.


## Possible additions

Mechanisms on the map, not in the proposal, that the records connect to an unaddressed or partly addressed claim, an open flaw or a dependency. Pointers, not recommendations: each brings its own readiness level and flaws, and none is claimed to close a flaw.

None found.



## Dependencies

**Blockers**

- Deterministic and bit-exact inference: Batch invariance costs throughput: on Qwen3-8B the improved deterministic build took 42 s against 26 s for vLLM's default, and SGLang reports an average slowdown of 34.35% on its FlashInfer and FlashAttention 3 backends. (performance & compatibility) [1][3]
- Deterministic and bit-exact inference: Outputs are identical only while the model, inference implementation and device stay fixed, so provider and verifier must run the same stack. (performance & compatibility) [6]


## What the verifier sees

- Model weights: shown by none; depends on the design for none; hidden by none; not involved in none; unspecified for Deterministic and bit-exact inference.
- Inputs and outputs: shown by none; depends on the design for none; hidden by none; not involved in none; unspecified for Deterministic and bit-exact inference.
- Training data: shown by none; depends on the design for none; hidden by none; not involved in none; unspecified for Deterministic and bit-exact inference.

## Implementations

- Deterministic and bit-exact inference: [Batch-invariant inference kernels (Thinking Machines)](https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/) (R2, open-source project); [Verde and RepOps (Gensyn)](https://trustbutveri.fyi/implementations/gensyn-verde-repops/) (R3, product); [Low-trust AI compute verification system overview](https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/) (R1, proposed architecture)

## Sources

1. Defeating Nondeterminism in LLM Inference, H. He & Thinking Machines Lab (2025). https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/
2. thinking-machines-lab/batch_invariant_ops (GitHub repository), Thinking Machines Lab (2025). https://github.com/thinking-machines-lab/batch_invariant_ops
3. Towards Deterministic Inference in SGLang and Reproducible RL Training, The SGLang Team (2025). https://www.lmsys.org/blog/2025-09-22-sglang-deterministic/
4. Batch Invariance (vLLM documentation), vLLM project (2026). https://github.com/vllm-project/vllm/blob/main/docs/features/batch_invariance.md
5. [Feature]: Batch Invariant Feature and Performance Optimization (vLLM issue #27433), vLLM project contributors (2025). https://github.com/vllm-project/vllm/issues/27433
6. DiFR: Inference Verification Despite Nondeterminism, A. Karvonen et al. (2025). https://arxiv.org/abs/2511.20621
7. Verde: Verification via Refereed Delegation for Machine Learning Programs, A. Arun et al. (2025). https://arxiv.org/abs/2502.19405
8. EigenAI: Deterministic Inference, Verifiable Results, D. Ribeiro Alves et al. (2026). https://arxiv.org/abs/2602.00182
9. Bit-Exact AI Inference Verification Without Performance Tradeoffs, N. Cankaya (2026). https://arxiv.org/abs/2606.00279
10. Hawkeye: Reproducing GPU-Level Non-Determinism, E. Badash et al. (2026). https://proceedings.mlsys.org/paper_files/paper/2026/hash/e217c271a57c365a246b0ad39e668ba8-Abstract-Conference.html