There’s a poetic idea hidden inside modern neuroscience: the “lights” didn’t switch on all at once. Something like consciousness likely emerged in layers—first as sensing, then as integration, then as memory, prediction, self-models, and finally the reflective ability to notice your own mind.
That matters right now because the AGI race is essentially the engineering version of that evolutionary story: building systems that don’t just generate answers, but navigate messy environments, maintain goals, update beliefs, recover from surprises, and keep working when the world changes.
We can’t yet point to a single, universally accepted scientific definition of consciousness. But we can track the capabilities that evolution kept selecting—and those capabilities map surprisingly well to what “true autonomy” in AI requires.
The first step wasn’t “thinking”—it was sensing + acting
Before brains, there was adaptive behaviour.
A classic example is E. coli chemotaxis: bacteria move toward nutrients (such as sugars) by sensing gradients and adjusting the rotation of their flagella. That’s not random motion—it’s information guiding action.
This is the earliest “proto-architecture” that matters for both biology and AI:
- Sensors (receptors)
- State (internal chemistry / internal variables)
- Policy (how to act given state)
- Feedback (did it get better or worse?)
Even if there’s “nobody home” in a bacterium, evolution is already building the machinery that later becomes cognition: control loops that reduce danger and increase survival.
AGI parallel: the moment you build an agent that must reliably pursue goals under uncertainty (rather than predict text), you recreate this same pattern: perception → internal state → action → feedback.
Integration: when lots of signals must become “one situation”
As organisms got more complex, the world stopped being a single gradient and became a multi-stream problem: light, motion, temperature, threats, and social cues.
That’s where nervous systems change the game—especially once you have networks that can integrate signals across space and time (rather than isolated reflexes). The evolutionary timing and origins are debated, but early nerve nets (e.g., cnidarians) and later centralised structures represent a step-change in integration capacity.
AGI parallel: today’s AI often excels at one channel at a time (text, vision, control). The hard part is deep integration: a unified internal state that binds perception, memory, goals, and action—without becoming brittle.
Memory: the bridge from reaction to learning
A system that only reacts is limited. Memory lets behaviour change based on what happened before.
In the brain, long-term memory is strongly tied to synaptic plasticity (often summarised as “cells that fire together wire together”), a foundational concept associated with Hebb’s work.
Once memory exists, new abilities emerge:
- learning associations (“that smell predicts food”)
- recognising places and contexts
- building stable preferences and aversions
- updating strategies after failure
AGI parallel: memory is not just a database. For autonomy, memory must be action-relevant: it changes the policy, not just the prompt.
Prediction: why many neuroscientists think experience feels “model-like”
A major modern view is that brains don’t passively record reality—they actively predict sensory input and update those predictions when they’re wrong (predictive processing / related ideas). Anil Seth is one well-known voice connecting perception and conscious experience to controlled hallucination-like prediction constrained by sensory data.
Prediction matters because it turns intelligence into a simulation engine:
- “What happens if I do X?”
- “What’s likely true given noisy inputs?”
- “What should I attend to next?”
AGI parallel: the leap from “good answers” to “robust autonomy” is basically the leap from pattern completion to world modelling + counterfactual simulation.
Selfhood: the world model starts including “me”
At some point, organisms benefit from modelling not only the environment but also their own bodies, limits, and viewpoints.
One perspective (e.g., Graziano’s attention schema theory) argues that consciousness relates to simplified internal models of attention and self-relevant processing.
Regardless of which theory you favour, self-models unlock:
- boundary management (self vs. other)
- stable goals (needs, drives, “preferences”)
- social reasoning (what others might do)
AGI parallel: if you want an agent to operate safely and reliably, it needs an explicit (or implicit) model of:
- what it is trying to do,
- what it can and cannot do,
- what it knows vs. what it doesn’t know,
- What actions change the external world?
Metacognition: knowing when you don’t know
Humans don’t just think—we can notice how well we’re thinking.
In animals, metacognition is tested indirectly (e.g., uncertainty monitoring / opt-out paradigms). The evidence is mixed across species and methods, but the broader point holds: self-monitoring improves decisions when the world is ambiguous.
AGI parallel: autonomy without self-monitoring becomes:
- overconfident tool use,
- compounding errors,
- failure to ask for help,
- brittle planning.
A system that can say “I’m uncertain—run a check” is closer to the evolutionary advantage that reflective cognition provides.
Consciousness is not “one brain shape”: convergent solutions show up
A significant shift in science has been acknowledging that complex conscious-like capacities are not limited to humans (or even mammals).
- The Cambridge Declaration on Consciousness (2012) stated that evidence indicates many non-human animals (including birds and octopuses) have the neurological substrates for conscious experience.
- The New York Declaration on Animal Consciousness (2024) further argued that there is strong scientific support for conscious experience in a range of animals, and a realistic possibility in more (including some invertebrates).
Why this matters: Evolution appears to find multiple architectures that deliver similar functions (perception, planning, flexible control).
AGI parallel: don’t bet everything on one “magic module.” Nature’s lesson is: if the function is valuable, multiple implementations will emerge.
Three major theories (and what each implies for AI)
No theory has “won,” but three frameworks show up constantly in modern debate:
Global Neuronal Workspace Theory (GNWT)
Conscious access emerges when information becomes globally available (“broadcast”) across many systems (attention, memory, decision, language).
AI implication: autonomy improves when perception isn’t trapped in silos—agents need a shared workspace (a unified state that downstream tools can use).
Recurrent Processing Theory (RPT)
Conscious perception depends on recurrent/feedback processing rather than purely feedforward sweeps.
AI implication: the path to robust reasoning may require more iterative loops (refinement, verification, re-entrant perception), not just deeper feedforward stacks.
Integrated Information Theory (IIT)
Consciousness corresponds (roughly) to how much a system integrates information into irreducible unified states (often discussed using Φ / “phi”).
AI implication: even massive computation may not yield unified “experience-like” states unless the architecture supports deep integration (though IIT’s claims are controversial).
Active inference: the cleanest bridge between “life” and “agents”
If you want one framework that naturally links biology to autonomous AI, it’s the free energy principle / active inference tradition associated with Karl Friston: organisms (and potentially agents) act to reduce prediction error/surprise by updating beliefs and taking actions that make their predictions come true.
Why it’s relevant to AGI:
- It treats perception + action as one loop
- It emphasises stability under uncertainty
- It encourages explicit modelling of what the agent expects and values
This is extremely close to what we actually want when we say:
“AI that truly knows how to reason and adapt autonomously in an ever-changing world.”
The AGI race, reframed as evolution’s checklist
Evolution didn’t “aim” for consciousness. It aimed for survival in messy environments—and kept rediscovering that the following bundle is a decisive advantage:
1) A world model (not just pattern completion)
- causal structure (what leads to what)
- object permanence/state tracking
- counterfactual simulation
2) Memory that changes behaviour
- episodic-like traces (“what happened last time”)
- skill/policy learning
- compression into useful abstractions
3) Integration + a shared workspace
- unify multimodal signals
- Keep goals consistent across tasks
- coordinate tools and sub-systems
4) Recurrence (think-looping, not single-pass)
- re-checking perception
- Revising plans after new evidence
- stabilising beliefs across time
5) Self-monitoring and uncertainty tracking
- calibrated confidence
- “Stop and verify”
- ask-for-help triggers
6) Grounded action in the real world
- tool use
- safe exploration
- recovery behaviours when plans fail
If you’re building AI agents, you can treat this as your autonomy readiness model.
Closing thought
The “consciousness story” is really a story about adaptive control: better models, better integration, better self-monitoring—again and again—because reality is volatile.
That’s also the core of the AGI race.
If AGI arrives, it likely won’t look like a single breakthrough. It will look like a system that can stay coherent while the world changes—and keep updating itself without falling apart.
