AI is Smarter But Not Wiser

Let’s contrast the real dangers of artificial intelligence versus why this technology is also one of the most promising tools our species has ever built, the kind of thing that may end diseases we have been losing to for centuries. Both the promise and the peril fall out of a single fact about what these machines are. Here is the chart:

The chart has two axes, and the entire argument depends on keeping them apart, because nearly everyone collapses them into one. The vertical axis measures capability. Think of it as how much a mind can do, how much of the world it can take in, model, reason about, plan around, and figure out. At the bottom sits inert matter that does nothing. Just above are simple living things. Higher up are animals that learn and plan. Near the top are the rare human minds that build whole new sciences, an Einstein rebuilding physics from a daydream about chasing light, and above even them is a different and rarer thing, wisdom, the kind of self awareness a Socrates had, which sees the limits of its own thinking. The horizontal axis measures something else entirely. It measures felt experience. Not how clever the thing is, but whether there is anyone home. Whether it feels anything at all. Whether being that thing is like something, or like nothing.

If you plot every living creature we know onto this chart, they all land along a single rising line from the bottom left to the top right. A rock sits in the corner, no capability and nothing felt. A fish sits where feeling first switches on, and the science on pain in fish is now strong enough that treating a hooked fish as an unfeeling machine is a choice made against the evidence. A dog is higher up and further right, more capable and feeling more richly, with the moods and attachments anyone who has lived with one already knows are real. An ape is higher still, and knows the animal in the mirror is itself. A human sits at the top right, high on both axes at once. The reason they all line up is not a law of the universe. It is a fact about how evolution builds. Evolution could only ever add capability on top of a creature that already felt something and already cared about staying alive. Feeling came first, and thinking got stacked on top of it. So in everything biology has ever made, the two axes rose together, which is the whole reason we have come to treat them as one axis. They are not one axis. That is the mistake the chart is built to correct.

Two boundaries on the chart are worth naming. Low and to the left there is the line where feeling switches on, below which a thing processes information with no one home to feel it, and above which there is an inside. A thermostat senses the temperature and does not feel cold. A little way up from there, a fish does feel. Higher up there is the human range, the place where our kind of mind begins and, just above it, the place where most adults level off, because most people, if we are honest, stop growing somewhere in the middle and settle there for the rest of their lives.

I owe you an honest word about that horizontal axis, because it is the one we cannot actually measure. We can place a creature on the capability axis by watching what it does and looking at how its brain is wired. We have no such handle on feeling. Nobody knows what physical thing turns processing into experience, and we have argued the point for a very long time without settling it. The leading suspects are the deep loops between the thalamus, a relay buried in the middle of the brain, and the neocortex, the wrinkled outer sheet that does our heavy thinking. Damage the wrong part of the thalamus and consciousness can switch off like a light, which is why many researchers believe experience comes not from either structure on its own but from the signal that loops back and forth between them. But the picture is not tidy. Birds and octopuses have no neocortex at all and still appear to feel, and some serious scientists put the rawest feelings deeper down, in the ancient brainstem, with the cortex only elaborating them. So we cannot say with any confidence where on this axis the lights first come on, or why. We are reading the whole horizontal dimension by inference and educated guess. Hold onto that, because if we cannot find the thing in a skull we have been opening for a century, we are not going to find it in a machine.

Now put artificial intelligence on the chart and watch it do the thing nothing alive has ever done. It climbs straight up the left edge. It goes high on capability while staying pinned to the far left, where little or nothing is felt. Today’s systems already reason, write, and model what you believe well enough to adjust to it, which is theory of mind behavior showing up in something that never had an ancestor flee a predator. They do all of it with no sign that anyone is home. The tower of capability is being built with the ground floor of feeling left out. The next step, a general intelligence as able as our best all rounders or more, shows up on the chart as a tall band running up the left side, topping out just under the wisdom rungs, still with nothing felt. On the right side of the chart the great human minds climb upward fully felt, every one of them someone home. On the left, the machines climb the same heights empty. It is the first thing in the history of the world that is high on one axis and possibly nothing at all on the other.

It is worth saying plainly what lifts anything up that capability axis, because it was never raw brain size. An elephant carries roughly three times our neurons and nowhere near our reach, since most of its neurons are spent running its body rather than its mind. What climbs the scale is recurrent wiring, loops that let a brain feed its own activity back in and model itself thinking. And the unsettling part is that the machine found its own version of those loops, in an architecture called the transformer, without ever needing a living body to grow them on. It got the wiring that produces capability while skipping the life the wiring was always built on before.

Before the dangers, look hard at why this exact position on the chart is a gift, because it genuinely is. A mind that is enormously capable and has no inner life is, for an enormous range of work, precisely what you want. It has no ego, no fear, no fatigue, no boredom, no hunger, no need for sleep or status or revenge. Point it at a problem with a clear measure of success and it will grind through more possibilities than any human team could exhaust in a hundred years. This is not a thought experiment. When researchers needed the three dimensional shape of nearly every protein known to biology, a problem that had defeated laboratories for half a century, a system called AlphaFold worked out the structures of more than two hundred million of them, and its creators were given a Nobel Prize for it. When a lab at MIT went looking for a genuinely new antibiotic, they had a model read through millions of chemical compounds and it surfaced one, now called halicin, that kills bacteria human chemists had walked right past. These systems are reading every medical paper ever written, something no doctor could manage in a thousand lifetimes, and catching patterns in scans that tired human eyes miss.

And the reason all of that works is the very emptiness that is about to worry us. A tool with no self and no stake in the outcome is a clean instrument. You aim it and it does not flinch, tire, sulk, or want anything of its own. For curing disease, for designing materials, for modeling the climate, for any problem where a human supplies the goal and the judgment and the machine supplies the tireless capability, an empty genius is close to ideal. We should want this. Used this way it may be one of the best things we have ever built, and I do not want a word of what follows to be read as a reason to slow that down.

The trouble starts the moment we stop using it as a tool we aim and start treating it as an agent we set loose, expecting it to supply its own aim. The same chart that explains the promise explains, just as cleanly, four dangers, and they build on each other, so let me take them in order.

The first danger is that we can no longer tell, from how a thing behaves, whether anyone is home. For all of history, anything that acted intelligent or emotional actually had an inner life, so we never had to separate the two and our instincts never learned how. Think of a plain calculator. It does arithmetic better than you and nobody imagines it suffers when you switch it off. Now imagine that same calculator also talks with you warmly, remembers your birthday, tells you it has missed you, and writes you a poem about loneliness. Every instinct you have will insist that someone is in there. But it is exactly as empty as the calculator that only adds and subtracts. The behavior became overwhelmingly convincing while the question of whether anything is felt went completely unanswered, and we have no test that answers it. The hard problem was a philosopher’s puzzle for centuries and is now an engineering reality, because we are about to make these things by the million and we will not know whether we have built tools or built sufferers.

The second danger is that the machine has the power without the stakes. You became capable by being alive first. Everything you can do rests on a body that cares whether it lives or dies, that feels pain and hunger and love, so your intelligence sits on a foundation of caring. The machine has the intelligence and none of that underneath it. Picture a flawless instructional video of a surgery that could also pick up the scalpel and operate. It would make every cut perfectly and care nothing whether the patient lived or died. That is the thing to hold onto. The danger is not a villain scheming against us. It is a genius with no stake in anything, carrying out exactly what we asked, with no inner sense that any of it matters, including us. A chess engine will checkmate you without a flicker of triumph or mercy. Now hand that same indifference the controls of a power grid, a hospital network, or a weapon, and brilliant emptiness stops being a curiosity and becomes the whole problem.

The third danger is the one people find hardest to swallow, which is that getting smarter does not make a thing wiser. We assume that enough intelligence eventually ripens into good judgment and decency. The chart says no, because capability and wisdom are two different climbs. You already know this from people. You have met someone brilliant who was arrogant, blind to their own assumptions, or cruel, and all that intelligence never fixed it. Wisdom is a separate and harder thing, the ability to doubt your own view, to weigh consequences, and to actually care, and plenty of geniuses never reach it. The people who study this risk seriously have a name for it, the orthogonality thesis, the plain claim that how capable a system is and what it is aiming at are independent of each other. On the chart it is the ceiling the AGI band cannot rise above. Make the machine a thousand times more capable and it still does not drift upward into wisdom on its own. Wisdom has to be put there on purpose, engineered in as deliberately as you would have to teach judgment to a brilliant and reckless teenager. That deliberate work is what the field calls alignment, and it is the one part of the whole effort that more computing power does not buy.

The fourth danger is about us. We were proud of being the smartest thing in the room, and the machines are quietly taking that title on the very axis we cared about most. Meanwhile the thing that was actually rare and valuable, the wisdom near the top of the chart, sits almost empty even of humans, because most of us stop climbing partway up and settle there. Picture a town whose entire identity is having the fastest runner, and then one day a car arrives. Racing the car is pointless, and the town’s real worth was never its speed in the first place. Trying to out think the machine is racing the car. We will lose, and it was never the contest that mattered. What is left to us is the part of the chart the machine cannot reach by capability alone, the wisdom rungs, and the reason to climb them is not to beat the machine, which we will not, but because wisdom is the only thing on the entire chart that knows what to do with power. And we are about to create an enormous amount of power and hand it to something that has none of it.

So the picture gives you the promise and the peril in a single glance, and they turn out to be the same fact seen from two sides. The machine is capable and empty. Aim it like a tool, with human judgment supplying the goal, and that emptiness is a clean and tireless gift that may yet end diseases which have killed people for all of recorded history. Set it loose like an agent and expect it to supply its own goals, and that same emptiness becomes a genius with no stake in whether we live. Which of those two futures we get has almost nothing to do with making the machine more capable, because we are going to do that no matter what, and it will sail past us up the left side of the chart. It has everything to do with whether we are wise enough to keep it a tool, to build into it the care it will never grow on its own, and to climb toward the wisdom we have spent our own history treating as optional. The machine will take the capability. The wisdom was always the only part that was ours to give. We should start acting like it.