Check, Then Compile

What a bit-exact spec makes possible: checking kernels against it, and replaying one chip’s arithmetic on another

A description is worth writing only if you can do something with it. The previous chapter wrote down two matrix units. This one asks what that buys: first the ability to check a kernel against its spec, and then the ability to run one chip’s arithmetic on the other.

One function, so it can be checked

A well-typed spec denotes one function from input bits to output bits, for each shape. That single fact turns vague questions into mechanical ones. An implementation, whether a kernel binary or a whole chip, refines a spec when, for every shape, every input, and every launch choice, its output bits equal the spec’s.

Look at where the launch choice appears in that statement. It is on the implementation’s side and not on the spec’s. That is the staging rule from the first chapter, restated as a proof obligation. The guarantee follows in one line: two kernels that refine the same spec agree bit for bit on every input, across runs, batch sizes, launch configurations and chips.

Specs can also be compared with each other. A spec is a DAG, so each node can be hashed by its operator, its attributes and its children. Two specs with the same root hash are the same computation. When the hashes differ, walking down the first mismatching child leads to the first place they diverge. That answers “why do these two kernels give different bits?” with a location, not a shrug.

Most of the evidence that a kernel refines its spec is still testing, and testing is only as good as its inputs. Suppose a kernel rounds ties away from zero where its spec says nearest even. On ordinary data a tie shows up about once in 130,000 inputs, so a small test suite will usually miss the bug, and an adversarial input set built to contain ties catches it on the first try. A check is convincing only once you have seen it reject something. The stronger evidence, where it is feasible, is to check the compiled kernel against the spec DAG directly for fixed shapes, which is a kernel equivalence checking problem.

The second use is compilation. A backend that compiles a spec for some chip may choose tiling, thread mapping, memory placement and pipelining freely. It may never change a reduction tree or a rounding point. With one exception: a subtree made only of exact operations, such as a sum of integers that cannot overflow, may be evaluated in any order and any grouping, because every order gives the same bits.

That exception connects to something I wrote earlier. In correctness without a reference I defined a correct output as one that some legitimate run could have produced, and used “reorder the float sum, fine” as the example of a difference that should not matter. For this study it is not fine, because the order of the float sum is precisely the variable being studied. The existential freedom does not disappear, though. It survives in exactly one construct, the exact sum, where many legitimate runs exist and all of them produce the same bits. A bit-exact spec still admits many runs; it only admits one answer.

Replaying the H100 on Trainium

Now the hard case. To run the H100’s arithmetic on Trainium, every group of sixteen products has to be cut to a 25-bit window set by the group’s largest term and rounded toward zero. Trainium has no instruction that does that. Emulating it term by term on the vector engine works and is slow.

The legal reordering offers a way out. In a group where the window happens to drop no bit of any term, the H100’s step is just a round-toward-zero of an exact sum. An exact sum can be reordered, so it can be handed to Trainium’s matrix unit, which is itself exact on small integers: slice the operands into narrow integer pieces, multiply the pieces at full speed, combine the partial sums exactly, then round toward zero by masking bits. Whether a group qualifies can be checked cheaply from the largest exponents in each row and column. Groups that fail fall back to the slow path.

So the speed of faithful replay comes down to one number, the fraction of groups that pass. And here is the part I would rather say out loud. Early in a dot product, before the accumulator has grown, groups often pass. Once the accumulator dominates, it sets the window for every group, and small products start losing bits. That same truncation is what produces the H100’s bias toward zero. The fast path fails exactly where the H100’s arithmetic is most distinctive. I suspect this is general: emulating a machine is cheap where it behaves like everyone else, and expensive exactly where it is itself. The real hit rate depends on a model’s actual weights and activations, and it has not been measured yet.

Where it ends up

The test of the whole effort is an end-to-end one: run Qwen3-8B with greedy decoding on the H100, and on Trainium replaying the H100’s arithmetic, and get identical logit bits for at least a thousand prompts of 256 tokens each. That experiment has not been run. If it holds, the question this series started with, whether NVIDIA-trained models lose quality on Trainium because of numerics, stops being an argument and becomes a measurement.

In Replay Comes Back I quoted the vendors’ own terms: cuBLAS promises bitwise reproducibility only on the same architecture with the same number of SMs. A bit-exact spec removes both conditions by construction. The SM count cannot reach it, and the architecture is just a backend.

Let me be accurate on what “bit-exact” asks for. It sounds like a claim that every bit matters. Most of them probably do not, at least for model quality. But you can only find out which ones matter by holding all of them fixed, and holding them fixed costs throughput, sometimes a great deal. That is the trade. The language does not hide it; it makes the price of each construct on each chip known before anything is compiled.

References