Source · Tier C · Forum / discussionSource · [Feature]: Batch Invariant Feature and Performance Optimization (vLLM issue #27433)
[Feature]: Batch Invariant Feature and Performance Optimization (vLLM issue #27433)
vLLM project contributors. 2025. GitHub (vllm-project/vllm issues).
| Original | https://github.com/vllm-project/vllm/issues/27433 |
|---|---|
| Accessed | 2026-10-17 |
| Note | vLLM developers' tracking issue for batch invariance. Its body states "We have basically support Batch Invariant based on" the Thinking Machines post (S-1009). On 2026-09-25 it listed open work such as performance, NVFP4, AMD testing and speculative decoding. On 2026-10-17 the issue was still open. Its unchecked items were FLASHINFER_MLA support, performance, prefix caching, AMD support, XPU support and speculative decoding, with no NVFP4 item. The AMD item links pull request #52231, which was merged on 2026-10-12. |
Cited by
- Deterministic and bit-exact inferenceMechanism
- Batch-invariant inference kernels (Thinking Machines)Implementation
- Thinking Machines LabOrganization