Source · Tier B · DocumentationSource · Batch Invariance (vLLM documentation)
Batch Invariance (vLLM documentation)
vLLM project. 2026. vLLM documentation (GitHub, docs/features/batch_invariance.md).
| Original | https://github.com/vllm-project/vllm/blob/main/docs/features/batch_invariance.md |
|---|---|
| Version | Main branch as viewed on 2026-09-25 (GitHub source file; the rendered page at docs.vllm.ai was not read). The feature was described as beta, supported on NVIDIA GPUs of compute capability 8.0 or higher and on Intel XPUs with Triton, and tested on dense and mixture-of-experts models. Commit e0317b28ce (12 October 2026, pull request #52231) added AMD Instinct GPUs (gfx90a, gfx942, gfx950) to the supported hardware. On 2026-10-17 that commit was the file's latest change, and the feature was still described as beta. |
| Accessed | 2026-10-17 |
Cited by
- Deterministic and bit-exact inferenceMechanism
- Batch-invariant inference kernels (Thinking Machines)Implementation