The ongoing debate between scaling text-based models and building embodied world models made me ponder a deeper question: what would it actually take for AI to be conscious?
It's interesting how you dissmanle the intuitive answer about consciousness. So, if introspection is just another cognitive process, are we basically just running a recursive function on our own perceptions?
Love the way you put that, it is a really nice intuition.
I would say yes, in a loose sense, we are doing something like running a function on our own perceptions, but the reality is a bit messier than the way it works in code.
Your brain does not just process red or pain or a melody. It also runs another process that takes that first process as input and builds a little story about it. Things like “I am seeing red right now” or “this hurts and I want it to stop”. That second layer is what we usually call introspection or metacognition.
That self model can itself be looked at again, summarized, talked about, remembered. So you do end up with representations of representations, which is where the idea of recursion fits.
But it is not a clean infinite loop the way a recursive function is. The brain only goes up a level or two when it is useful. It does this in a very parallel and noisy way, not like a single neat call stack. And introspection is pretty rough and approximate, more like a fuzzy debug log than an exact trace of what is happening.
So I like your framing. Introspection really is another cognitive process operating on other cognitive processes. The main claim in the piece is that once a system can do that kind of self monitoring and make those internal states available for report and decision making, you already have what we usually point to when we talk about experience or what it feels like from the inside, without needing some extra magical ingredient on top.
I should also say that we do not know in the sense of “proven beyond any possible doubt.” This is a theory about what consciousness is, grounded in a lot of neuroscience and psychology, but it is still a model of what is going on, not a direct reading of reality.
What we do have are a few kinds of evidence that all point in roughly the same direction.
First, we can change people’s experience in very specific ways by changing the brain. Anesthesia, strokes, prefrontal damage, certain drugs. Sometimes the sensory pathways are still working, the body still reacts, but the person reports no pain or no awareness. That suggests that “what it feels like” depends on particular higher level systems, not just raw input.
Second, we can see brain activity that tracks things like “this stimulus was consciously seen” versus “this stimulus was processed but not consciously seen.” Experiments with masking, change blindness, and so on show that the brain can process information unconsciously quite far, but only some of it becomes globally available and reportable. That lines up with the idea that consciousness is tied to this global, self accessible level, not to early sensory processing.
Third, there is a whole line of research on metacognition. People can be right or wrong about their own mental states. You can ask someone not just “what did you see” but “how confident are you that you saw it,” and you can measure how accurate their self assessment is. That tells us introspection is an additional layer of processing on top of the primary perception, and it can be selectively impaired or enhanced.
Put together, those kinds of findings make a certain picture very natural. The picture is roughly: there are lots of unconscious processes, and then there are systems that monitor some of those processes, integrate them, and make them available for report and control of action. What we call “conscious experience” seems to track that integrated, self monitored level, not some extra mysterious ingredient.
Could this whole picture still be wrong in some deep way? Sure. But it is not just armchair speculation about the brain. It is a way of making sense of a lot of empirical results without having to posit a special non physical “feel” on top of the physical activity.
So when I talk as if the brain “builds a self model” or “monitors its own processing,” I am really saying here is a way of describing patterns that show up again and again in experiments on perception, attention, pain, confidence, anesthesia, brain damage, and so on. It is a best current explanation, not a revealed truth.
It's interesting how you dissmanle the intuitive answer about consciousness. So, if introspection is just another cognitive process, are we basically just running a recursive function on our own perceptions?
Love the way you put that, it is a really nice intuition.
I would say yes, in a loose sense, we are doing something like running a function on our own perceptions, but the reality is a bit messier than the way it works in code.
Your brain does not just process red or pain or a melody. It also runs another process that takes that first process as input and builds a little story about it. Things like “I am seeing red right now” or “this hurts and I want it to stop”. That second layer is what we usually call introspection or metacognition.
That self model can itself be looked at again, summarized, talked about, remembered. So you do end up with representations of representations, which is where the idea of recursion fits.
But it is not a clean infinite loop the way a recursive function is. The brain only goes up a level or two when it is useful. It does this in a very parallel and noisy way, not like a single neat call stack. And introspection is pretty rough and approximate, more like a fuzzy debug log than an exact trace of what is happening.
So I like your framing. Introspection really is another cognitive process operating on other cognitive processes. The main claim in the piece is that once a system can do that kind of self monitoring and make those internal states available for report and decision making, you already have what we usually point to when we talk about experience or what it feels like from the inside, without needing some extra magical ingredient on top.
I should also say that we do not know in the sense of “proven beyond any possible doubt.” This is a theory about what consciousness is, grounded in a lot of neuroscience and psychology, but it is still a model of what is going on, not a direct reading of reality.
What we do have are a few kinds of evidence that all point in roughly the same direction.
First, we can change people’s experience in very specific ways by changing the brain. Anesthesia, strokes, prefrontal damage, certain drugs. Sometimes the sensory pathways are still working, the body still reacts, but the person reports no pain or no awareness. That suggests that “what it feels like” depends on particular higher level systems, not just raw input.
Second, we can see brain activity that tracks things like “this stimulus was consciously seen” versus “this stimulus was processed but not consciously seen.” Experiments with masking, change blindness, and so on show that the brain can process information unconsciously quite far, but only some of it becomes globally available and reportable. That lines up with the idea that consciousness is tied to this global, self accessible level, not to early sensory processing.
Third, there is a whole line of research on metacognition. People can be right or wrong about their own mental states. You can ask someone not just “what did you see” but “how confident are you that you saw it,” and you can measure how accurate their self assessment is. That tells us introspection is an additional layer of processing on top of the primary perception, and it can be selectively impaired or enhanced.
Put together, those kinds of findings make a certain picture very natural. The picture is roughly: there are lots of unconscious processes, and then there are systems that monitor some of those processes, integrate them, and make them available for report and control of action. What we call “conscious experience” seems to track that integrated, self monitored level, not some extra mysterious ingredient.
Could this whole picture still be wrong in some deep way? Sure. But it is not just armchair speculation about the brain. It is a way of making sense of a lot of empirical results without having to posit a special non physical “feel” on top of the physical activity.
So when I talk as if the brain “builds a self model” or “monitors its own processing,” I am really saying here is a way of describing patterns that show up again and again in experiments on perception, attention, pain, confidence, anesthesia, brain damage, and so on. It is a best current explanation, not a revealed truth.