AI-Native: Autonomy as Revision Authority

“AI-native” is about to become a word that means nothing. It is at the stage where everyone uses it and nobody has been asked to define it, which is the stage right before it appears on every slide. I’d like to try to fix it to something, while there’s still time to argue.

The instinct is to define it by presence: an AI-native system is one with AI inside it, and the deeper the AI goes, the more native it is. This is the wrong axis, and the previous two posts were mostly an argument for why. A learned cache policy has AI as deep inside as it goes—right on the hot path, in the per-event decision itself. And it revises nothing. The design is fixed. The code is fixed. The weights only move when a human decides to retrain. It sits at the center of the system and changes nothing about it.

Call that arrangement Model-Resident, because it deserves a name that sounds like what it is: something living there. Occupancy is not autonomy.

Grade by revision authority

So use the other axis. Not where does the AI sit but what is the AI allowed to revise? And since the last post gave us levels defined by binding time, we can be exact about it: a system’s grade is the level whose decisions the AI has authority to change.

S3, self-tuning. The AI has authority over L3—the parameters. It watches the system and turns the knobs: buffer sizes, thresholds, batch sizes, timeouts. This is real, this exists, and it is genuinely useful; automated configuration tuning beats hand-tuning routinely. It is also bounded in a way that’s easy to miss: the AI can only choose among behaviors the code already knows how to exhibit. If the right answer requires an eviction rule nobody wrote, no amount of tuning finds it. S3 searches inside a space it cannot enlarge.

S2, self-rewriting. The AI has authority over L2—the implementation. When the workload moves, it doesn’t just retune the heuristic; it writes a new one, in ordinary code, and ships it. This is the level that just became possible, and it is the whole reason this series is worth writing now. Note what it fixes: the artifact in production is still code, still fast, still readable, still revertible. The learning happened upstream and left.

S1, self-architecting. The AI has authority over L1—the design itself. It concludes that the problem isn’t the eviction rule but the fact that there’s a cache here at all, and restructures. As far as I can tell this category is empty. Not “early”—empty. People are beginning to explore the possibility, though—see the effort to define “system intelligence” on the SIGOPS blog.

The numbering runs backwards on purpose: Sn means authority over Ln. Lower number, higher stakes, larger space. And notice what the scheme says about the decade of work in the first post: there is no S4, and there cannot be one. An L4 decision is consumed the instant it’s made. There is nothing there to revise. Runtime machine learning went and sat at the one level of the system where adaptation is not merely hard but undefined, and then the field spent ten years trying to make adaptation happen there anyway—by retraining the model, which is to say, by editing L3 from the outside, by hand.

The definition

Strong-S2 is where I want to plant the flag, and it takes four clauses. An AI-native system is one in which:

The AI holds revision authority over the implementation (L2) and exercises it in a closed loop—observing the deployed system, deciding the current code is no longer the right code, regenerating it, and shipping it. Not a human prompting an agent for a patch. The loop is the system’s own.

It detects when regeneration is not enough and escalates. This is the “strong” part and the hardest engineering in the definition. A system that will rewrite its heuristic forever, sliding further from the design’s assumptions without ever saying so, is worse than one that can’t rewrite at all. Knowing this problem is above my level is the safety property that makes the rest tolerable.

It has authority over the allocation map—α—not just over the decisions. A system that revises code but takes a fixed assignment of who-decides-what from its operators is automated, not autonomous. An AI-native system can decide that this decision should now be made by generated code rather than a tuner, that that one needs to go back to a human. It manages the boundary, not just the interior. Autonomy over the allocation of authority is what separates a tool that adapts from a system that governs itself.

Every move is gated—ρ—with a verified fallback. Nothing takes effect because a model believed in it. Regenerated code passes tests, canaries, shadow traffic; a known-good path stays warm; rollback is fast and automatic. The fallback must be verified rather than merely present, which is the part people skip, and it is the same discipline the systems community already applies to anything it doesn’t trust—you don’t trust the output, you check it.

And the envelope, which is not a clause but a boundary around all of them: L0 and the objectives and invariants of L1 stay human. What the system is for. What “better” means. What must never happen.

An analogy to self-driving cars

AI-native systems have an obvious analogy in self-driving cars, whose autonomy levels are defined by an actual published standard. Most borrowings of that ladder are loose; a few of its lessons transfer cleanly. Here is what transfers.

The famous line is about fallback, not skill. The distinction between the levels people argue over is not how good the driving is. At the lower one, the human is expected to supervise and to handle what the system doesn’t; at the higher one, the human need not supervise but must be ready to take over when asked; higher still, the system itself performs the fallback and nobody expects the human to answer. The axis is who catches it when it fails. That is the same axis as my α—an allocation of responsibility—and its levels are a few named points on it, not a measure of merit.

The envelope is a first-class concept. No serious automated driving system claims to work everywhere; each declares the conditions it was designed for—roads, weather, time of day—and the standard treats the level as meaningless without that declaration. Safety doesn’t come from the system being excellent. It comes from the system knowing the edge of its envelope and having somewhere safe to fall.

And the standard has a hole exactly where mine would be. It requires that a system asking a human to take over does so “within sufficient time” for that human to respond appropriately—and never quantifies “sufficient time.” Anywhere. That is the whole handover problem hiding inside an adjective, and it’s the reason I don’t think escalation is a checkbox. A system that escalates without a time budget has not handed off responsibility; it has merely documented whom to blame. If you build the escalation clause I’m asking for, the number is the design.

The last piece is the one nobody argues about: the car does not choose the destination. Every fight about driving autonomy is about the middle of the stack. No one has ever proposed that the vehicle decide where you should want to go. That level was never contested, because it isn’t a driving problem.

Same here, and this is where the series ends. L0 stays human not because machines can’t be trusted with purpose—I don’t need that argument and I’m not sure it’s true. It stays human because it is the only level in the stack with nothing above it to appeal to. Every level below is an optimization problem handed an objective from the level above. At L0 there is no level above; there is only someone deciding what is worth wanting. Hand that to a machine and you haven’t automated a decision, you’ve deleted the thing the decision was answerable to.

So: let the machine own the maintenance loop, all of it, gated. Let it rewrite the code, retune the knobs, and redraw the map of who decides what. Keep the purpose. That’s not a compromise position. It’s the only division of labor where the word “better” still refers to anything.

References

SAE International. Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles, J3016_202104. (Levels are nominal rather than ordinal, §8.3; operational design domain, §3.21; “sufficient time,” §5.4.)