A field guide to the bit-exact kernel spec language
An H100 and a Trainium 2 both compute "a bf16 matmul with fp32 accumulation". On long dot products they agree on the exact bits in fewer than one case in a hundred. This book teaches the language we built to say precisely what each chip computes, one idea per chapter.
The figure is not a cartoon. It runs the H100 tensor-core model (verified on our H100 against 10⁹ outputs) and the Trn2 PE-array model (verified on Trn2 against 10⁹ outputs) on the same vectors, in exact arithmetic, in your browser. Neither chip is broken. They are both deterministic and both reasonable. They simply make different choices about how to group, align and round, and none of those choices appears in the vendor documentation.
The project behind this book asks a practical question: when a model trained on NVIDIA hardware runs on Trainium, does it lose quality because of the hardware's numerics? Answering it needs an experiment in which only the numerics change. That means running NVIDIA's arithmetic, bit for bit, on Trainium, and that in turn needs a precise, checkable description of NVIDIA's arithmetic.
Prose descriptions ("accumulates in fp32") are not precise enough, and the source code of a kernel is not enough either, because the arithmetic lives below the source, in the instructions. So the project defines a small language in which a kernel's output bits are a total, deterministic function of its input bits: every value is an exact dyadic rational, every rounding is explicit, and every reduction is a tree fixed by the shapes. Five principles shape every construct:
round[format, mode, policy].| Ch. | Topic | The one idea |
|---|
Each chapter stands alone as a post and builds on the ones before it. Chapters 1–6 teach the language itself with small examples. Chapters 7–9 use it on the two real chips and on real decoder kernels. Chapters 10–11 cover what can be checked and how specs compile across chips. Every figure is live and runs the same models as the project's Python code, re-implemented in JavaScript with exact BigInt arithmetic; reported hardware numbers are always the project's measured ones, labelled with their finding IDs (F1–F8 for Trn2, H1–H3 for H100).
Sources: the execution plan "Bit-Exact Kernel Spec: NVIDIA ↔ Trainium", HANDOVER.md, NOTES.md, docs/diff_table.md, docs/direction_A_design.md, and the spec/, libs/ and lift/ code in the bitexact repository, as of 2026-11-03. H100 matrix-unit semantics build on MMA-Sim (Xie et al., arXiv 2511.10909).