A field guide to the bit-exact kernel spec language

Same model, same inputs, different bits

An H100 and a Trainium 2 both compute "a bf16 matmul with fp32 accumulation". On long dot products they agree on the exact bits in fewer than one case in a hundred. This book teaches the language we built to say precisely what each chip computes, one idea per chapter.

One dot product, two chipsK = 4096 bf16 · computed live from the probed models
a, b ~ N(0, 1) rounded to bf16

The figure is not a cartoon. It runs the H100 tensor-core model (verified on our H100 against 10⁹ outputs) and the Trn2 PE-array model (verified on Trn2 against 10⁹ outputs) on the same vectors, in exact arithmetic, in your browser. Neither chip is broken. They are both deterministic and both reasonable. They simply make different choices about how to group, align and round, and none of those choices appears in the vendor documentation.

Why a language

The project behind this book asks a practical question: when a model trained on NVIDIA hardware runs on Trainium, does it lose quality because of the hardware's numerics? Answering it needs an experiment in which only the numerics change. That means running NVIDIA's arithmetic, bit for bit, on Trainium, and that in turn needs a precise, checkable description of NVIDIA's arithmetic.

Prose descriptions ("accumulates in fp32") are not precise enough, and the source code of a kernel is not enough either, because the arithmetic lives below the source, in the instructions. So the project defines a small language in which a kernel's output bits are a total, deterministic function of its input bits: every value is an exact dyadic rational, every rounding is explicit, and every reduction is a tree fixed by the shapes. Five principles shape every construct:

  1. No black boxes. Every vendor instruction, including tensor-core MMA and approximate transcendentals, is a macro over a small core.
  2. Exact arithmetic, explicit rounding. A float appears only through round[format, mode, policy].
  3. Algorithm fixed to the bit, schedule free. Trees, rounding points and constants are part of the spec; tiling, threads, memory and pipelining are not.
  4. Launch configuration is not expressible. Grid size, SM count and warp width cannot influence a result because the language cannot name them.
  5. Trust is explicit. Each instruction model carries its provenance, so a spec's trust base is the list of probed models it uses.

Chapters

Ch.TopicThe one idea

How to read this

Each chapter stands alone as a post and builds on the ones before it. Chapters 1–6 teach the language itself with small examples. Chapters 7–9 use it on the two real chips and on real decoder kernels. Chapters 10–11 cover what can be checked and how specs compile across chips. Every figure is live and runs the same models as the project's Python code, re-implemented in JavaScript with exact BigInt arithmetic; reported hardware numbers are always the project's measured ones, labelled with their finding IDs (F1–F8 for Trn2, H1–H3 for H100).

Sources: the execution plan "Bit-Exact Kernel Spec: NVIDIA ↔ Trainium", HANDOVER.md, NOTES.md, docs/diff_table.md, docs/direction_A_design.md, and the spec/, libs/ and lift/ code in the bitexact repository, as of 2026-11-03. H100 matrix-unit semantics build on MMA-Sim (Xie et al., arXiv 2511.10909).