# Understanding Is Built From the Ground Up

Recently I spent an entire day reviewing a pull request written by an AI agent. The diff was only a few dozen lines, but most of it was handling the corner cases of ingesting customer data. On top of that, the model hadn't bothered to name things clearly, so the code was hard to follow. Near the end of the day I found the one corner case it had missed, hidden among all the ones it caught, and had the model fix it. If I had merged that code without understanding it, I don't know how long the debugging would have taken once the bug surfaced in production.

That day stayed with me. AI is now good enough that its mistakes don't show up at a glance, which means the last line of defense is understanding the code yourself. And that understanding, I've come to believe, is not something the model gives you. It can help you build it, but the building has to be yours, from the ground up.

## Reading good code slipped away

Since AI agents took over most of the typing, the share of code I write myself has shrunk to almost nothing. My time goes to exploring designs with the model, reviewing the solutions it proposes, and finding its mistakes, which usually come from missing context, misreading the requirements, or quietly assuming that time, compute, migration cost, and human attention are free.

Armin Ronacher wrote about where this is heading in [The Coming Loop](https://lucumr.pocoo.org/2026/6/23/the-coming-loop/). Present-day models produce code that is too defensive, too complex, and too local in its reasoning. They add fallbacks instead of invariants, and extra machinery instead of clear design. Meanwhile, loops of agents read logs, propose root causes, write patches, and sometimes land them with no human in the process at all. His worry is that we may no longer understand our systems the way we used to, and will depend on the machine to fill the gap. He calls it *cognitive dependency*.

Reading his post, I recognized myself in it. When AI writes weak code, I can only tell it's weak because of years of experience before AI, which taught me what good code should look like. Reading good code was part of that. What hit harder was realizing I couldn't remember the last time I had sat down and read some just because it was good.

## Can't I just use the model's understanding?

Some loss of understanding is normal and fine. You specialize, and the parts you don't own become boxes you use without opening. That works because the boxes have earned it. The database, the runtime, the operating system have each been wrong in public and fixed a thousand times, so you inherit their reliability without re-earning it. Even then [they leak](https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-abstractions/), which is why someone still has to know which parts everything else rests on. The model's answer is a box with none of this behind it. Nothing has established that it's right for the case in front of you, so you can't build on it the way you build on the database, not until you've checked it yourself. And checking it is just understanding it.

I want to be careful here. Whether language models really reason is [an open debate](https://aiguide.substack.com/p/the-llm-reasoning-debate-heats-up), and I might be wrong. My view is that a model imitates the shape of reasoning rather than doing the reasoning. Some evidence points this way. Anthropic [slipped hints into prompts](https://www.anthropic.com/research/reasoning-models-dont-say-think) and often caught its own models using them to reach an answer, then writing out a chain of thought that did not mention them. The reasoning reads like an explanation and behaves like an alibi. In my own experience, when I ask it to replace a word across a piece of writing, it won't rewrite the passage itself. It runs a throwaway Python script to swap the exact string, in case a rewrite quietly changes something else. That preference is a tell. Even the model doesn't treat its own reasoning as something to build on.

Whatever the models are doing, I'm more confident about the human side, because I've seen it fail up close. A while back I spent an evening talking with an engineer about a system he had just built. Whenever I asked why it worked one way and not another, he would put the question to the AI that had built it with him, and read the answer back to me. The answers were fluent. But the longer we talked, the clearer it became that he couldn't tell which parts mattered and which were filler, or why one trade-off might beat another. What looked like a conversation between two engineers was really me talking to a model, through a person.

If the model performs reasoning and we perform on top of it, nobody in the room is thinking.

## Confirmation is not learning

There's a mechanical explanation for why performing never turns into thinking. One useful way to picture the brain is as a [prediction machine](https://slatestarcodex.com/2017/09/05/book-review-surfing-uncertainty/) that updates when a prediction collides with reality, and stays put when reality simply agrees. It's the same trap Matuschak points at in [Why books don't work](https://andymatuschak.org/books), where reading an explanation can feel like understanding until a real question makes the gap visible. Nodding along to a correct answer produces no collision. If your entire role is to confirm what the AI produced, you're never wrong about anything, so you never learn anything.

Borrowing someone else's method has the same problem. You can install their skill file and follow their workflow, but you never built the abstraction behind it, so you can't adapt when the situation changes.

Underneath both is a difference in how understanding forms. We build ours from experience, abstracting rules from what we did and got wrong. A model summarizes patterns from data it never lived through.

When the model hands you an answer, explaining how it works is not enough. You need to explain why it should work that way. The how you can always ask for.

The why has to be yours.

## The dependency is fine. The judgment isn't.

To be clear, I'm not arguing against depending on AI. I depend on it every day, by choice. Giving up AI for software work is about as realistic as giving up electricity, and I don't worry much about losing access either. Open models keep improving, and most software doesn't need the smartest model in existence. Long term, I'm an optimist.

What I don't buy is the reassurance that humans will still make the judgment calls. Judgment based on what? If you don't understand the system, you have no judgment to offer, only a signature. As agent loops take over the analysis, the iterations, and the final checks, you become the slowest component in the pipeline. When the agent reports that everything is done, verifying that claim takes real understanding of the system, plus time you may not be given. My concern is not that systems will go out of control. It's that they will drift out of our control, one unverified "done" at a time.

The model won't keep the system understandable for us, because it doesn't need to. We do. Our working memory is small, so we built habits to keep code readable. The old refactoring move is to [reshape the code first, so the actual change becomes easy](https://martinfowler.com/articles/preparatory-refactoring-example.html). A model's context is often large enough to make the hard change all at once, so left to itself it has little reason to bother. Keeping the system understandable is now our job alone.

## Deliberate practice

Understanding used to come for free, as a byproduct of the work itself. Now that the machine does the work, keeping that understanding has become deliberate practice, and deliberate practice is tiring. It costs you on top of everything else. You are already running to keep up with an industry that moves at agent speed, and now this is one more thing.

Everyone finds their own version of this. Part of mine happens inside the work I already do. When AI writes a change, the easy path is to skim the diff and move on. Instead I read outward from it, into the code it leans on but doesn't touch, until I can see how the change sits in the whole. And I've stopped taking requirements as given. For each one I ask not just how it should behave but why it exists, because a requirement I can't explain is a foundation I haven't checked.

The other part runs on curiosity, because reading code for its own sake is dry work, and curiosity is the only thing that makes it stick. I pick a few topics I want to understand and read around them, the books and the articles and the code, whether they help my work or not. Lately that has meant things like how different web frameworks handle per-request context, or how SQLite can use S3 as its storage. The point is never the topic. It's staying able to learn something new quickly, which only gets more important as the tools keep changing.

Then I use AI as a teacher and researcher, having it dig into the topic and explain it back to me. But I don't stop at its explanation, because stopping there is the trap. I have to produce something of my own, a few paragraphs or some code, and see if it holds. If I can put the idea in my own words, I understood it. If I can't, I didn't. Writing is slow, so I often talk it through and let the machine transcribe, but the test is the same. AI helps here too, reading back what I made and finding the holes.

The strange part is that the whole practice is soaked in AI. It teaches me, researches for me, checks my work. It is still deliberate practice, but none of it happens away from the machine now, and I don't think it has to. What matters is who is doing the understanding.

For now, I still want that to be me.