CTM-AI: Evolving Multi-Agent Intelligence Inspired by a Model of Consciousness
summary
The gist
CTM-AI presents an early blueprint for implementing a general AI system by combining the Conscious Turing Machine with foundation models to bridge the gap between consciousness theory and practical…
In short
CTM-AI combines a model of consciousness with foundation models to create a general AI system. It uses specialized processors, limited memory, and iterative loops for selective attention and communication. This architecture achieves state-of-the-art results in multimodal perception and tool use by dynamically coordinating agents.
Key concepts
- Conscious Turing Machine (CTM)
- A formal model of consciousness inspired by Alan Turing's computation theory. It provides the abstract blueprint for how a general AI system should be structured, defining components like processors and memory workspaces to mimic conscious processing.
- Up-tree Competition
- A mechanism used in CTM-AI to select which information chunk enters the limited short-term memory workspace. This competition ensures that only the most relevant or important information is prioritized for processing, enforcing selective attention across the system.
- Iterative Loop
- The core inference process involving four stages: chunk inference, up-tree competition, down-tree broadcast, and link formation. This loop allows the system to continuously refine its understanding by exchanging information between processors over multiple cycles.
- Link Fusion
- The process where relevant processors form links and exchange information during unconscious communication. This exchange enriches long-term memories by integrating diverse pieces of data into a more comprehensive and nuanced representation.
Terminology used across episodes
This episode discusses
- CTM-AI: Evolving Multi-Agent Intelligence Inspired by a Model of Consciousness · Paper Radio
- GPT-4 Technical Report
- Qwen3-VL Technical Report
- Consciousness in Artificial Intelligence: Insights from the Science of Consciousness
- AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- QoQ-Med: Building Multimodal Clinical Foundation Models with Domain-Aware GRPO Training
- StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models
- Towards a Science of Scaling Agent Systems
- PuzzleWorld: A Benchmark for Multimodal, Open-Ended Reasoning in Puzzlehunts
- A Unified Multi-Agent Framework for Universal Multimodal Understanding and Generation
- Agent-Omni: Test-Time Multimodal Reasoning via Model Coordination for Understanding Anything
- VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents
- AlphaEvolve: A coding agent for scientific and algorithmic discovery
- ChatDev: Communicative Agents for Software Development
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
- Agent Laboratory: Using LLM Agents as Research Assistants
- Mixture-of-Agents Enhances Large Language Model Capabilities
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Qwen3-Omni Technical Report
- A Survey of AI Agent Protocols
The paper
CTM-AI: Evolving Multi-Agent Intelligence Inspired by a Model of Consciousness · Read on arXiv
Haofei Yu, Yining Zhao, Lenore Blum, Manuel Blum, Paul Pu Liang
University of Illinois Urbana–Champaign · Carnegie Mellon University · Massachusetts Institute of Technology
Despite remarkable advances, today's AI systems remain narrow in scope, falling short of the adaptive and continual intelligence that characterizes human capabilities. This gap has fueled longstanding debates about whether AI might one day achieve human-like generality or even consciousness. Flipping the debate, we present a concrete blueprint for a multi-agent AI system inspired by a formal machine model of consciousness called the Conscious Turing Machine (CTM). Together, CTM-AI contains a large collection of powerful processors ranging from specialized experts (e.g., language models, multimodal models, tools, and APIs) to unspecialized general-purpose learners poised to develop their own expertise. Crucially, for whatever problem must be dealt with, information from many processors is competitively selected, integrated, and exchanged appropriately to solve the task. Feedback mechanisms in CTM-AI enable the system to iteratively evolve its internal connections and communication. CTM-AI achieves state-of-the-art accuracy on MUStARD (72.28) and UR-FUNNY (72.13), outperforming multimodal and multi-agent frameworks. On tool-using and agentic tasks, CTM-AI achieves 10+ points of improvement on StableToolBench and WebArena-Lite. Overall, CTM-AI offers a principled and testable multi-agent AI system inspired by a model of consciousness. The code is available at https://github.com/consciousness-lab/ctm-ai.
Transcript
Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.
Ines: We're looking at the paper: "CTM-AI: Evolving Multi-Agent Intelligence Inspired by a Model of Consciousness".
Marcus: Here's the gist of the paper. CTM-AI presents an early blueprint for implementing a general AI system by combining the Conscious Turing Machine with foundation models to bridge the gap between consciousness theory and practical AI architecture This work demonstrates that CTM-AI achieves state-of-the-art accuracy on MUStARD (72.28) and UR-FUNNY (72.13), outperforming multimodal and multi-agent frameworks Pruning redundant reasoning tokens The paper introduces CTM-AI, which combines the Conscious Turing Machine with foundation models to create a general AI system inspired by consciousness theory The CTM is defined as a simple and formal model of consciousness inspired by Alan Turing’s model of computation and Bernard Baars’ theater model of consciousness CTM-AI operationalizes this abstract architecture using modern AI technologies, defining multiple specialized processors, a limited capacity short-term memory workspace enforcing selective attention via up-tree competition, a global broadcast of information via a down-tree from the workspace to all processors, and the formation of links between relevant processors over time Key contributions include serving as the first practical instantiation of the CTM, yielding a highly modular and decentralized multi-agent architecture free from rigid central orchestrators, and integrating reasoning and agentic flexibility How the training signal is built The core inference process of CTM-AI involves an iterative loop defined by four stages: LTM processor chunk inference, Up-tree competition into STM, Down-tree broadcast, and Link formation and fusion Step 1 involves all K LTM processors running in parallel to produce chunks with a content gist h it summarizing its findings, a set of follow-up queries q it for potential cross-processor consultation, and a self-assessed score s it The chunk score s i t is decomposed into three interpretable sub-scores: weight, intensity, and mood Step 2 uses an up-tree competition to select the final chunk that goes into STM's limited-capacity workspace based on the chunk scores Step 3 involves a down-tree broadcast where the winning chunk is distributed to all LTM processors causing them to pay conscious attention Step 4 enables unconscious communication where LTM processors form links and exchange information, leading to link fusion which enriches long-term memories Where the gains come from Quantitative results show CTM-AI achieves state-of-the-art or competitive results across multimodal perception, tool use, and agentic tasks On multimodal perception tasks like MUStARD and UR-FUNNY, CTM-AI matches and slightly outperforms the strongest baselines on MUStARD and UR-FUNNY For tool use on StableToolBench, CTM-AI yields the most substantial gains, improving over the strongest single-model baseline by up to 16.8 points on multi-tool scenarios Furthermore, CTM-AI generalizes to tool-use and agentic tasks yielding an absolute improvement of >10% in pass rate on both StableToolBench and WebArenaLite The paper also demonstrates that the iterative loop is the most critical mechanism for performance, with the iterative loop emerging as the most critical (-6.7 F1) mechanism Ablation on CTM-AI dynamics Ablation studies confirm a clear hierarchy of importance among CTM-AI mechanisms The iterative loop emerges as the most critical (-6.7 F1), followed by up-tree competition (-5.6), link fusion (-3.9), and down-tree broadcast (-3.5) Scaling the iterations from 1 to 4 yields continuous performance gains across both datasets, with MUStARD improving from 65.0 to 73.8 The analysis also shows that link formation behaviors differ between datasets, as UR-FUNNY forms an average of 2.6 links at four iterations compared to just 1.5 for MUStARD RQ2: As an inference-time scaling method, how does CTM-AI compare with the reasoning paradigm?
Ines: CTM-AI and reasoning are complementary paradigms where applying reasoning models improves baseline performance but falls short of CTM-AI when the backbone model is fixed Pairing Qwen3-8B-think with CTM-AI yields an additional +16.8 to +27.3 point improvement on StableToolBench compared to using Qwen3-8B-think alone, confirming that reasoning deepens deliberation within individual processors while CTM-AI broadens coordination across them This decentralized architecture supports the flexible integration of reasoning, allowing practitioners to selectively inject advanced reasoning capabilities into specific modalities A.
Marcus: The Use of Large Language Models (LLMs) The implementation uses LLMs as a writing assistant and utilizes CodePilot and Claude Code to help code faster, with all AI-generated content manually checked The backbone model selected for most processors is Gemini-2.5-flash-lite because it supports audio, vision, and text inputs while being relatively small-scale B.
Ines: Artifact Details The paper utilizes several models including GPT-4o (OpenAI, 2024), OpenAI o3 (OpenAI, 2025), Gemini-2.5-flash-lite (Comanici et al., 2025), and Qwen3 variants like Qwen3-VL-8B and Qwen3-Omni-flash Evaluation datasets include MUStARD, URFunny, WebArena-Lite, and StableToolBench C.
Title and authors: Ines: So we're looking at this paper today called "CTM-AI: Evolving Multi-Agent Intelligence Inspired by a Model of Consciousness". It’s trying to build a practical blueprint for a general AI system by mixing the formal model of consciousness, the Conscious Turing Machine, with foundation models.
Marcus: Yeah, that's what it does. Basically, they take this abstract idea from consciousness theory and try to make it work with today's AI tools. It’s trying to bridge that gap between the theory and what we can actually build right now.
Yuki: From a population genetics perspective, I think the focus on a general system is interesting because it suggests we aren't just optimizing narrow tasks anymore, but something more adaptive. It asks if this architecture could support broader forms of intelligence across different domains.
Ines: Exactly. The core idea is they operationalize this consciousness model using multiple specialized processors, short-term memory workspaces with selective attention through up-tree competition, and a global broadcast mechanism from the workspace to all processors.
Marcus: That decentralized structure is what caught my eye because it’s trying to move away from those rigid central orchestrators we see in current agentic systems. It aims for a highly modular architecture where you can just add or remove specialized processors without breaking the whole thing.
Yuki: I wonder if that decentralization actually translates into robust coordination when you're dealing with complex, real-world problems that require deep, long-term understanding of the world.
Ines: The mechanism they describe for how information flows is really about this iterative loop: chunk inference, up-tree competition into short term memory workspace, down-tree broadcast, and then link formation between processors over time.
Marcus: That link formation part is key because it enables unconscious communication between processors to integrate their knowledge into higher-order multimodal information. It’s not just about passing data along a line; it's about them learning from each other in a more integrated way.
Yuki: So, when you look at the results they show, they claim CTM-AI achieves state of the art accuracy on MUStARD and URFUNNY datasets. I want to know what that means for genuine perception capabilities.
Ines: It means they are matching or slightly beating the strongest baselines on those multimodal perception tasks, achieving scores like seventy-two point something on MUStARD and seventy-two point something on URFUNNY.
Title and authors: Marcus: And it's not just about the raw accuracy; they also showed significant gains in tool use. On StableToolBench, CTM-AI improved over the strongest single model baseline by up to sixteen point something in multi-tool scenarios.
Yuki: That suggests that this architecture is actually helping the system handle complex, multi-step tasks with tools much better than what we see now when we just use a single large model.
Ines: And they also showed generalization to tool use and agentic tasks, yielding an absolute improvement of over ten point something in pass rate on both StableToolBench and WebArenaLite.
Marcus: The iterative loop itself is highlighted as the most critical mechanism for performance, with it emerging as the most critical factor at negative six point seven F1. That tells us that how often this loop runs really dictates how good the system gets.
Yuki: If we look at those ablation studies on iteration count, scaling from one to four iterations gives continuous performance gains across both datasets; MUStARD goes up from sixty five point zero to seventy three point eight.
Ines: And they found that the link formation behavior actually differs between the datasets; for example, URFUNNY forms an average of two point six links at four iterations, compared to just one point five for MUStARD.
Marcus: That difference suggests that the nature of the interaction between processors changes depending on what kind of task you're asking them to solve.
Yuki: So, before we move on, how does this architecture actually compare to using a reasoning model alongside a fixed backbone model? I always wonder if this is just another way to layer reasoning on top.
Ines: Well, they show that pairing Qwen3 eight point eight with CTM-AI yields an additional improvement of sixteen point eight to twenty seven point three on StableToolBench compared to using just Qwen3 eight point eight alone.
Marcus: That points toward the idea that CTM-AI and reasoning models operate along different axes for coordination, which is a big detail. It’s not just one or the other; it's how they work together across those dimensions.
Yuki: I see how that fits with the larger picture of complex systems where you need both deep individual deliberation and broad coordination to get a good result.
Title and authors: Ines: The paper also mentioned using large language models as a writing assistant, but the backbone processor is Gemini two point five flash lite because it handles audio, vision, and text inputs while still being relatively small-scale.
Marcus: And when you look at their efficiency analysis, they found that most tasks resolve within one to three iterations. The performance boost from adding more iterations is directly proportional to an increase in the average number of interprocessor links formed.
Yuki: That suggests that the system dynamically allocates more processing power only when it actually needs it to improve communication density, which sounds quite efficient.
Ines: They also found that the optimal balance for link formation threshold, called eta, is somewhere between zero point four and zero point seven for both datasets.
Marcus: So, a few final thoughts before we wrap up on this CTM-AI blueprint: the architecture is modular because of those competition mechanisms. The results show it performs well across perception, tool use, and agentic tasks.
Yuki: I think the self-reported scores are also important because they showed that the distribution stabilizes and converges by iteration three, meaning that mechanism continues to be a driver for its internal dynamics.
Ines: So we've seen how this formal model of consciousness translates into a decentralized system free from rigid central orchestrators, and it achieves high accuracy on perception tasks like MUStARD and URFUNNY.
Marcus: And the tool use improvements on StableToolBench really show that this dynamic prioritization of processors leads to substantial gains when dealing with multi-tool scenarios.
Yuki: Overall, CTM-AI seems to be a blueprint for how we might build AI that handles complex tasks through flexible coordination rather than just brute force scaling.
Ines: That's the main point, and it points toward building systems that are fundamentally more capable in affective learning and decision making.
Marcus: We've covered the architecture, the performance gains on perception and tools, how iteration count affects performance, and how reasoning models can enhance it. This paper on CTM-AI really shows a path forward for architecturally inspired AI.
Yuki: It’s definitely something to keep watching as we think about building more adaptive intelligence in these systems.
Ines: We'll leave you there with the CTM-AI blueprint, an early attempt to translate consciousness theory into a practical, working system for general AI.
The paper's summary: Ines: So we're looking at how they’re trying to build this whole system by taking that formal model of consciousness, the Conscious Turing Machine, and mixing it with foundation models to create a general AI architecture.
Marcus: Right. It’s basically trying to bridge that gap between the abstract theory of consciousness and what we can actually build in practice right now.
Yuki: From my perspective as someone who looks at population genetics, I see the focus on a general system suggesting we aren't just optimizing for one narrow task anymore, but something that could be more adaptable across different kinds of intelligence.
Ines: Exactly. They’re operationalizing this by using multiple specialized processors, a limited short-term memory workspace that uses up-tree competition for selective attention, and a global broadcast from the workspace to all processors.
Marcus: That decentralized structure is what interests me because it’s moving away from those rigid central controllers we see in current agent systems. It aims for something modular where you can just add or remove specialized processors without breaking the whole thing.
Yuki: I wonder if that decentralization actually translates into robust coordination when you're dealing with really complex, real-world problems that need deep, long-term understanding of the world.
Ines: The core process they describe is this iterative loop: chunk inference, up-tree competition into short term memory workspace, down-tree broadcast, and then link formation between processors over time.
Marcus: That link formation part is key because it allows unconscious communication between processors to integrate their knowledge into a higher level of multimodal understanding. It’s not just about passing data along a line; it’s about them learning from each other in a more integrated way.
Yuki: So when you look at the results they show, they claim this CTM-AI achieves top accuracy on perception tasks like MUStARD and URFUNNY datasets. I want to know what that means for genuine perception capabilities.
Ines: It means they are matching or slightly beating the strongest baselines on those multimodal perception tasks, hitting scores like seventy-two point something on MUStARD and seventy-two point something on URFUNNY.
The paper's summary: Marcus: And it's not just about raw accuracy; they also showed huge gains in tool use. On StableToolBench, CTM-AI improved over the strongest single model baseline by up to sixteen point something in multi-tool scenarios.
Yuki: That suggests this architecture is helping the system handle complex, multi-step tasks with tools much better than what we see now when we just use one big model.
Ines: They also showed generalization to tool use and agentic tasks, yielding an absolute improvement of over ten point something in pass rate on both StableToolBench and WebArenaLite.
Marcus: The iterative loop itself is highlighted as the most critical mechanism for performance, emerging as the most important factor at negative six point seven F1. That tells us how often this loop runs really dictates how good the system gets.
Yuki: If we look at those studies on iteration count, scaling from one to four iterations gives continuous performance gains across both datasets; MUStARD goes up from sixty five point zero to seventy three point eight.
Ines: And they found that the link formation behavior actually changes between the datasets; for example, URFUNNY forms an average of two point six links at four iterations, compared to just one point five for MUStARD.
Marcus: That difference suggests the way processors interact changes depending on what kind of task you're asking them to solve.
Yuki: So before we move on, how does this architecture compare to using a reasoning model alongside a fixed backbone model? I always wonder if this is just another way to layer reasoning on top.
Ines: They show that pairing Qwen three point eight with CTM-AI yields an additional improvement of sixteen point eight to twenty seven point three on StableToolBench compared to just Qwen three point eight alone.
Marcus: That points toward the idea that CTM-AI and reasoning models operate along different axes for coordination, which is a big detail. It’s not just one or the other; it's how they work together across those dimensions.
The paper's summary: Yuki: I see how that fits with the larger picture of complex systems where you need both deep individual deliberation and broad coordination to get a good result.
Ines: The paper also mentioned using large language models as a writing assistant, but the main backbone processor is Gemini two point five flash lite because it handles audio, vision, and text inputs while still being relatively small-scale.
Marcus: And when you look at their efficiency analysis, they found most tasks resolve within one to three iterations. The performance boost from adding more iterations is directly proportional to an increase in the average number of interprocessor links formed.
Yuki: That suggests the system dynamically allocates more processing power only when it actually needs it to improve communication density, which sounds pretty efficient.
Ines: They also found that the optimal balance for link formation threshold, called eta, is somewhere between zero point four and zero point seven for both datasets.
Marcus: So before we wrap up on this CTM-AI blueprint: the architecture is modular because of those competition mechanisms. The results show it performs well across perception, tool use, and agentic tasks.
Yuki: I think the self-reported scores are also important because they showed that the distribution stabilizes and converges by iteration three, meaning that mechanism continues to be a driver for its internal dynamics.
Ines: So we've seen how this formal model of consciousness translates into a decentralized system free from rigid central orchestrators, and it achieves high accuracy on perception tasks like MUStARD and URFUNNY.
Marcus: And the tool use improvements on StableToolBench really show that that dynamic prioritization of processors leads to substantial gains when dealing with multi-tool scenarios.
Yuki: Overall, CTM-AI seems to be a blueprint for how we might build AI that handles complex tasks through flexible coordination rather than just brute force scaling.
Ines: That's the main point, and it points toward building systems that are fundamentally more capable in affective learning and decision making.
The paper's improvements: Ines: We're talking about how they suggest making CTM-AI better by focusing on these specific mechanisms instead of just running the system once and moving on.
Marcus: Right, so instead of just looking at the final score, they point to things like scaling up those iterative loops because that loop is actually the most critical part of the whole process at negative six point seven F1.
Yuki: Scaling those iterations means if you increase them from one to four, MUStARD performance goes up from sixty five point zero to seventy three point eight. That shows that more iterations help improve the system’s grasp of the task.
Ines: And it's not just about getting a higher score; they show that increasing those iterations directly leads to a bigger average number of interprocessor links formed, which is what actually drives the performance boost.
Marcus: That means we don't just want more computation; we want that extra computation to create better communication between the processors.
Yuki: So if you look at link formation, they found that for URFUNNY, it forms an average of two point six links at four iterations compared to only one point five when you run it once. That shows how the interaction changes depending on the task.
Ines: That’s interesting because it suggests the system learns a different way to coordinate when dealing with humor versus perception tasks.
Marcus: And they also looked at those self-reported scores and found that the distribution stabilizes and converges by iteration three, meaning that self-reporting keeps playing a crucial role in driving what's happening inside the AI.
Yuki: So, for someone listening who doesn't know all this detail, it just means CTM-AI isn't just a static system; it’s something that actively refines its own internal structure over time.
Ines: That’s right. It’s adapting and getting denser in its communication graph without needing a manual reset or parameter update.
Marcus: It sounds like the goal is to build this kind of intelligence that improves by simply interacting with the world through these iterative refinement steps rather than just being pre-programmed.
Yuki: That takes us nicely into how they compared this to using reasoning models on top of a fixed backbone model, because that’s another way to think about enhancing the system's capability.
Conclusion: Ines: So we've covered how this CTM-AI system uses that formal model of consciousness to build a decentralized architecture using up-tree competition and down-tree broadcast, and I want to wrap up by looking at what all this means for the next generation of AI.
Marcus: Yeah, it’s essentially showing a path toward building general AI systems that are structured around cognitive principles rather than just brute force scaling.
Yuki: From a population genetics standpoint, this suggests that we need architectures that support diverse coordination across different modalities and tasks, which is essential for understanding complex species behavior.
Ines: Exactly. The work on CTM-AI: Evolving Multi-Agent Intelligence Inspired by a Model of Consciousness shows how you can translate consciousness theory into something practical for general AI.
Marcus: And the results on MUStARD and URFUNNY, plus the gains in tool use, show that this approach is actually getting state of the art performance without needing a single massive model to do everything.
Yuki: It changes how we think about what an agent needs to be capable of; it’s not just about bigger data sets anymore, but how that data is processed and coordinated.
Ines: The way they handle those iterative loops, where the system refines its internal structure through link formation and memory fusion, shows a mechanism for continuous self-improvement.
Marcus: It means we might see AI systems that adapt over time without needing explicit parameter updates from a human engineer every single month.
Yuki: So what it changes for someone just listening to the show is that the focus shifts from just training massive models to designing these dynamic, coordinated architectures.
Ines: That's right. It points toward building systems that are fundamentally more capable in areas like affective learning and decision-making because they have this built-in mechanism for deeper deliberation.
Marcus: It’s a solid blueprint for how we can start thinking about building more flexible, multi-agent systems that actually coordinate effectively across different specialized components.
Yuki: I'm curious to see if this kind of dynamic coordination holds up when we apply it to even more complex biological or social systems in the future.
More episodes
- 2610.33881-Sensorimotor behavior from an untrained brain-and-nerve-cord connectome
- 2610.40344-Morphological Features of Murine Pulmonary Arterial Networks for Diagnosing Vascular Disease
- Daily Summary for 2026-11-19
- 2609.10193-Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets
- 2610.38236-Generative discovery of chemical reaction networks
- 2507.11176-An Interpretable AI framework Quantifying Traditional Chinese Medicine Principles Towards Enhancing and Integrating with Modern Biomedicine
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- 2504.03590-Towards a Unified Framework for Determining Conformational Ensembles of Disordered Proteins
- 2603.15217-A multiscale discrete-to-continuum framework for structured population models