Accountable by Construction
An evidence tree for AI: commit to everything, check what you choose, and read only what no machine can vouch for
I give an agent a task. Later it says the task is done. How would I know?
Take a small example: write a correct function that sorts an array. In version one, the agent replies “done, tested.” In version two, it writes the code, writes tests, runs them, compares the outputs against what a sorted array should look like, and reports that every check passed.
Almost everyone trusts version two more. But every step was done by the same agent. The tests could be trivial, the run invented, the comparison simply asserted. If the agent can err or lie in version one, it can in every step of version two. So why does version two feel better? The intuition is right, and the reason it is right tells us how to build accountability for AI instead of hoping for it.
Not more trustworthy, more checkable
Humans make mistakes too, and we never fixed that by finding people who don’t. We arranged work so that mistakes leave evidence. Version two does three things version one does not. It splits one claim into smaller ones: “the code is correct” is hard to check, “this test covers the empty array” is easy. Some of those claims are executions, and a computation can be rerun by someone else. And the steps leave artifacts, a test file and a run log, that exist apart from the agent’s account of them.
None of this makes the agent more truthful. It changes what a lie costs. A false “done” costs one sentence. A false version two needs a test file, a log, and outputs that agree with each other and with the code, and survive whatever check I choose to run afterward. Trust does not track how much work the agent did. It tracks how much of that work I can check, and how cheaply.
Commit first, then check
Whatever check I choose afterward is where a Merkle hash tree earns its place. Each node stores the hash of its children, so one short root value commits to the whole structure; change anything and the root changes. Git works this way, and so do the public logs that make web certificates auditable.
Hash every artifact the agent produces, and every claim together with the hashes it rests on. The agent hands over the root hash before I choose what to open. Now a spot check means something: any step the agent faked had to be faked in advance, and every one I open that does not match its commitment is caught. Random sampling becomes an audit with computable odds.
A hash proves that what I see is what was committed, not that it is right. Whether a claim is right depends on what kind of step the node records.
Three kinds of node
Executed nodes record a computation: run these tests on this code, compare these outputs with this oracle. Anyone can replay them from the committed inputs, so their credibility is in effect 100%. What they do not cover is how the computation came to be: the script that runs the tests was written by the agent, so “this script runs these tests” is a judged claim. Running it is not.
Judged nodes record claims no computation settles. Do these tests cover what “sorted” means? Did the agent think of duplicates, the empty array, reversed input? A separate certificate model reads the task, the artifacts, and the claim, and assigns a credibility score. It must be independent of the agent, with different weights and nothing but the committed artifacts to go on: an agent testing its own code shares its own blind spots. And its scores are judgments, not proofs; language models are still unreliable step-level judges (see Does It Work?).
Given nodes are the specification: “output is sorted and is a permutation of the input.” Nobody checks it, because it defines the task. A person has to read it and agree it is what they meant.
The root is the human’s task, children are subtasks, and each node carries a kind and a score. Scores compose by product, so one weak link drags down everything above it.
What the human reads
Nobody reads the whole tree. A human reads the specification and the few nodes with low scores. Everything else is replayed by machines, scored by an independent model, or sampled against a commitment made in advance.
Machine-checked proofs already work this way: you trust a small checker and the statement at the top, not the person who wrote the proof. And every verified system still has a paragraph at the top that someone read and believed. The difference is that an evidence tree is made of steps a person can read, so a failure lands on a named node.
When the tree is too big to read
A sorting tree has a handful of nodes. The tree for porting a large system, or for a week of agent work, has thousands, and “read the low-scoring nodes” can still mean hundreds. Reading the tree has become a task. So treat it like one: give it to an agent, and ask for an evidence tree.
The tempting request is “summarize this tree,” and it is the wrong one. A summary that replaces evidence can lose the one bad node inside “mostly fine.” Ask something narrower: does tree T, committed at root hash h, support claim R, and where are its exceptions?
That question has a useful property. Most of it is computation over committed data. Does every executed node replay? Does every hash check? Which judged nodes score below 0.9? These are programs run over the tree, so in the new tree they are executed nodes, replayable by anyone. What stays judged is small: the exceptions themselves, and the threshold that defines an exception, which belongs to the human. The tree about the tree is mostly executed, and only the exceptions climb.
Repeat it, and each level is smaller than the last, because aggregates are cheap to check and, if the work was good, exceptions are rare. Stop when what remains fits in a person’s attention: the specification and a few nodes no machine could vouch for. Every level points at the one below by hash, so the human can still open any leaf. Nesting shrinks what you must read, not what you can check. Organizations already work this way: an auditor does not read every invoice; she tests the controls, samples, and reports the exceptions.
This also has a limit, and the limit is informative. Nesting converges only when judgment is sparse. If most of the original work is judgment, every level forwards the same uncertain claims upward and nothing shrinks. That is not a failure of the method. It is a measurement: work whose evidence cannot be compressed is work we cannot yet delegate with accountability, and it marks where people still have to do the thinking.
Leaving evidence
Proof-carrying code had the same economics decades ago: the producer builds the evidence, which is expensive, and the consumer checks it, which is cheap. AI makes the producer’s side nearly free, so we should ask for much more of it.
Version two was the right instinct, half-built. Don’t ask whether the agent is trustworthy. Ask for a tree you can hold it to: every artifact committed, every computation replayable, every judgment scored by someone else, and a top small enough to read. Accountability for AI will not come from AI that never errs. It will come from AI that cannot err without leaving evidence.
References
- Ralph C. Merkle. A Digital Signature Based on a Conventional Encryption Function. CRYPTO 1987.
- George C. Necula. Proof-Carrying Code. POPL 1997.
- Ben Laurie, Adam Langley, and Emilia Kasper. Certificate Transparency. RFC 6962, 2013.