{
  "schema_version": "1.2.0",
  "rubric_version": "1.1",
  "license": "CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)",
  "record": {
    "id": "O-0214",
    "slug": "thinking-machines-lab",
    "title": "Thinking Machines Lab",
    "aliases": [],
    "status": "published",
    "last_reviewed": "2026-09-25",
    "review_interval_days": 90,
    "steward": null,
    "provenance": {
      "drafted_by": "ai",
      "reviewed_by": [
        "codex-review"
      ]
    },
    "risk_flags": [],
    "flags": [
      "provider-reported"
    ],
    "kind": "company",
    "homepage": "https://thinkingmachines.ai/",
    "one_liner": "An AI research and product company; developer of batch-invariant kernels that make language-model outputs independent of batch size.",
    "sources": [
      {
        "source": "S-3562",
        "supports": "self-description",
        "locator": "homepage"
      },
      {
        "source": "S-1009",
        "supports": "batch size as the main cause of nondeterminism; batch-invariant kernels; Qwen3-235B result (provider-reported)"
      },
      {
        "source": "S-1813",
        "supports": "MIT-licensed batch_invariant_ops library",
        "locator": "README"
      },
      {
        "source": "S-1012",
        "supports": "SGLang's deterministic mode built on the kernels"
      },
      {
        "source": "S-1814",
        "supports": "vLLM developers' statement that batch invariance is based on the post"
      }
    ],
    "type": "organization",
    "url": "https://trustbutveri.fyi/organizations/thinking-machines-lab/",
    "source_file": "content/organizations/thinking-machines-lab.md",
    "flags_all": [
      "provider-reported"
    ],
    "body_markdown": "Thinking Machines Lab describes itself as \"an artificial intelligence research and product company\" [[S-3562]]. Its work relevant here is [[I-0016]]:\n\n- In September 2025 it published batch-invariant kernels for LLM inference, arguing that varying batch sizes are the main reason LLM endpoints give nondeterministic outputs [[S-1009]]. Its MIT-licensed batch_invariant_ops library replaces several PyTorch operations with batch-invariant versions [[S-1813]]. See [[M-0002|deterministic and bit-exact inference]].\n- It reports that, with the kernels, 1,000 temperature-zero completions from Qwen3-235B were identical [[S-1009]]. SGLang built its deterministic mode on the kernels [[S-1012]], and vLLM's developers state that its batch-invariant mode is based on the same work [[S-1814]].",
    "body_text": "Thinking Machines Lab describes itself as \"an artificial intelligence research and product company\" [S-3562]. Its work relevant here is Batch-invariant inference kernels (Thinking Machines): - In September 2025 it published batch-invariant kernels for LLM inference, arguing that varying batch sizes are the main reason LLM endpoints give nondeterministic outputs [S-1009]. Its MIT-licensed batchinvariantops library replaces several PyTorch operations with batch-invariant versions [S-1813]. See deterministic and bit-exact inference. - It reports that, with the kernels, 1,000 temperature-zero completions from Qwen3-235B were identical [S-1009]. SGLang built its deterministic mode on the kernels [S-1012], and vLLM's developers state that its batch-invariant mode is based on the same work [S-1814].",
    "referenced_by": [
      {
        "id": "I-0016",
        "title": "Batch-invariant inference kernels (Thinking Machines)",
        "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/"
      },
      {
        "id": "S-1009",
        "title": "Defeating Nondeterminism in LLM Inference",
        "url": "https://trustbutveri.fyi/sources/he-defeating-nondeterminism-llm-inference/"
      },
      {
        "id": "S-1813",
        "title": "thinking-machines-lab/batch_invariant_ops (GitHub repository)",
        "url": "https://trustbutveri.fyi/sources/thinking-machines-batch-invariant-ops-code/"
      },
      {
        "id": "S-3562",
        "title": "Thinking Machines Lab",
        "url": "https://trustbutveri.fyi/sources/thinking-machines-homepage/"
      }
    ]
  }
}