What Would Make a System AI-Native

For about a decade, my field has been putting machine learning inside systems. Learned index structures that beat B-trees. Learned caches that beat LRU. Learned schedulers, learned congestion control, learned query optimizers. The papers are good—I mean that seriously, not as a setup. The gains are real and the evaluations are thorough.

Now go look at what’s actually running in production, and you find something stranger than failure. The learning shipped. The models mostly didn’t.

Azure predicts VM behavior with trees trained offline and served behind an explicit “no prediction” fallback. Google’s flash-cache admission policy is a knapsack solver. The heuristic that packs Google’s fleet and recovers about 0.7% of its worldwide compute is four lines of arithmetic, produced by an offline search. And the nearest thing to an exception draws the line rather than erasing it: Google’s VM scheduler does consult a learned lifetime model in production—compiled directly into the scheduler’s own binary, nine microseconds a call, at a decision point that fires tens of times a second, not billions. In every case the learning happened, and then something either took the model out before deployment or pinned it down until it behaved like code.

Nobody planned this as a principle. Group after group, in subfield after subfield, discovered it as an implementation detail on the way to a benchmark number, reported it in a methods section, and moved on. The pattern has never been named, and it should be, because it is the answer to the question everyone actually asks about ML for systems—why the papers keep winning and the models—with rare exceptions—keep not shipping.

Here is the claim this series is about. The problem was never modeling. It was placement. Let me elaborate.

The question underneath

The reframe I want to make is small and, I think, load-bearing. We have been asking where does the model go. The better question is when does the decision get frozen, and who is allowed to unfreeze it.

Every decision in a running system was made at some point in time and then held fixed. The architecture was decided once, years ago, and is now the thing nobody touches. The algorithm was decided when someone wrote the code, and changes on the timescale of code review. The tunables were decided at configuration time and change on the timescale of a deploy. And the per-event choices—evict this line, admit this packet, run this thread—are decided fresh, constantly, and consumed instantly.

That ordering is not a taxonomy I invented for tidiness; it’s a real gradient, and it has an exponent on it. As you descend, decisions get more frequent and each one matters less. At the top, a handful of choices that determine everything. At the bottom, billions of choices a second, each nearly weightless. The bottom of that gradient is where we spent ten years putting neural networks.

Once you see the gradient, the diagnosis writes itself. The barriers people list for machine learning in systems—inference cost, silent degradation under workload drift, ugly tail behavior, a training pipeline that someone has to keep alive forever—are not four independent problems. They are four symptoms of one choice: the model runs at runtime. Move the learning earlier and most of them stop being your problem, because what you ship is no longer a model. It’s code.

That is what has actually changed recently, and why I think the question is live again rather than settled. A coding agent doesn’t have to sit in the hot path to help you. It can sit in the development loop, look at your workload, and write an ordinary heuristic—branches and constants, the kind of thing a good engineer would have written if they’d had a month to stare at traces. The artifact you deploy is code you can read, profile, bound, and revert. The learning happened, and then it left. It’s in the git history, not the binary.

Occupancy is not autonomy

That is only half the reframe, and the smaller half. The other half is a question about authority.

When we say a system is “AI-native”—and people are going to say it, with or without a definition, on a great many slides—the instinct is to ask where the AI is. Which component is the model in? How deep does it go? But sitting somewhere is not the same as being in charge of anything. A learned cache policy occupies a decision point. It also revises nothing: the design is fixed, the code is fixed, and the model itself only moves when a human retrains it. It is resident, not autonomous.

So grade systems by what the AI is allowed to revise, not by where it sits. A system where the AI adjusts parameters is doing something real but modest. A system where the AI regenerates its own implementation—rewrites the heuristic when the workload moves, notices when the rewrite isn’t enough, and ships it through a gate that can roll it back—is a categorically different thing. And a system where the AI revises its own design is, as far as I can tell, an empty category today. Worth naming anyway, so we can be honest that it’s empty.

Stated that way, the last decade acquires an uncomfortable shape. Runtime machine learning put the model at the one level where revision isn’t even defined—a per-event choice is consumed the instant it’s made; there’s nothing there to adapt. The adaptation people wanted was always happening one or two levels up, in the code and the configuration, done by humans, slowly.

What the series is

Three posts, in the order it takes to explain what “AI-Native” should technically mean:

ML for systems—why a decade of strong results shipped everything but the model, and why four of the five standard barriers are the same barrier wearing different clothes.
Binding time—the four levels of decision making in systems, and why only compiled code can occupy the bottom one.
Revision authority—a definition of AI-native I’m willing to defend, and what stays human forever.

And a word about where this is going, so nobody has to read three posts to find out whether I’m proposing that the model run the datacenter. I’m not. The argument ends with an envelope: the purpose of the system, the objectives, the invariants that must not be violated, and the meaning of the word “better” stay with people. Not because machines can’t be trusted with them—that’s a separate argument—but because those are the only decisions in the stack with nothing above them to appeal to. Everything below is an optimization problem. The top is not. That’s the one level that isn’t a level; it’s the thing that makes all the others answerable.