What does it mean to understand language?

arXiv:2511.19757 · cs.CL · Submitted 2025-11-24 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "What does it mean to understand language?".

Jane: Here's the gist of the paper. Language understanding entails not just extracting the surface-level meaning of the linguistic input, but constructing rich mental models of the situation it describes The paper proposes that because processing within the brain’s core language system is fundamentally limited, deeply understanding language requires exporting information from the language system to other brain regions that compute perceptual and motor representations, construct mental models, and store our world knowledge and autobiographical memories This framework suggests that deep understanding necessitates exporting information from the core language system to other cognitive and neural systems The Core Language System's Role in Understanding The core language system is specifically engaged in understanding (and producing) language, and not in other nonlinguistic cognitive processes fMRI research over the last 15 years has identified and characterized the brain’s core language system with new precision During comprehension, the language system’s goal is to extract information from word sequences, which requires recognizing familiar words and larger constructions and figuring out how they go together within the sentence However, we argue that this representation is shallow: it is derived solely from our knowledge of language statistics, akin to representations in early versions of text-only large language models Furthermore, the representations in the language system lack sensitivity to real-world plausibility Mechanisms for Deep Understanding Deep understanding is supported by extra-linguistic systems that can be engaged both by linguistic and nonlinguistic inputs These extra-linguistic systems can augment the shallow representations constructed by the core language system with perceptual or motor features, translate them into a format compatible with probabilistic programing engines or symbolic problem solvers, or transform them in some other way The paper identifies several candidate exportation destinations including regions specialized for thinking about the contents of someone else’s or one’s own mind, regions specialized for scene understanding and navigation, regions that construct mental models of the physical world, regions that represent and process amodal semantic information, perceptual and motor regions, and regions that integrate information into a coherent representation of the situation that is being described Specific Exportation Destinations The paper details several brain regions that support deep language understanding, such as those involved in theory of mind, intuitive physics, navigation and scene understanding For instance, theory of mind involves the right temporo-parietal junction (rTPJ) and the broader Theory of Mind network Intuitive physics involves regions of the parietal and frontal cortex in the "Physics Network" Navigation and scene understanding involve areas like the parahippocampal place area (PPA), occipital place area (OPA), and retrosplenial cortex (RSC) Perceptual regions can be engaged when understanding verbal descriptions of visual scenes, such as faces or bodies Motor representations are also implicated when understanding action words, linking to motor cortex regions Episodic and semantic knowledge is accessed through the Default Network A (DN-A) for episodic memory and amodal semantic regions for general conceptual knowledge Factors Influencing Exportation The likelihood of information exportation depends on several factors, including the language system's memory limitations, traits, states, and goals of the comprehender Specifically, input that extends beyond a sentence must be passed to other systems to construct temporally extended contexts Furthermore, the language system must be able to extract a representation abstract enough to interface with downstream systems The process of exportation can occur through two main possibilities: "routing," where information is selectively directed, or "broadcasting," where representations are sent to all possible downstream systems The paper concludes that deep language understanding requires exporting information from the brain’s core language system to other systems that build mental models of the minds, objects, and places described in the linguistic input Caveats include the fact that strong empirical evidence for exportation is restricted to a few systems and methodological limitations restrict conclusions from past work Future research should focus on how language representations are "translated" into a format other brain regions can process and what specific factors influence whether and when information is exported Outstanding questions remain regarding the extent to which everyday language understanding takes place within the core language system The paper proposes a broad research program to advance a richer theory of what language understanding entails cognitively, neurally, and computationally Glossary Exportation: The process by which information delivered through language is transferred out of our core language system to one or more other brain regions for further processing Shallow understanding: Building a representation of linguistic input based solely on knowledge of patterns of language use Deep understanding: Building rich mental models of the people, objects, places, and situations described in a language input Situation model: A representation of the ‘situation’ that is being described by a language input Core language system: A network of frontal and temporal brain regions that are highly specialized for language processing, strongly functionally and anatomically interconnected, and causality important in comprehending and producing language Routing: One hypothesis for how information is exported from the language system to downstream systems, whereby information is selectively directed Broadcasting: Another hypothesis for how information is exported to downstream systems, whereby information is sent from the core language system to all possible downstream systems Acknowledgments We thank Andrea de Varda, Ted Gibson, Steve Piantadosi, Rebecca Saxe, Thomas Clark, and members of the Kanwisher lab for helpful discussion and comments on the manuscript We also thank Emalie McMahon for help with the brain maps for the figure CC was supported by the Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University EF was partially supported by U01 award NS121471 from NINDS, funds from the Simons Foundation awarded to the Simons Center for the Social Brain at MIT, from the McGovern Institute for Brain Research, and MIT’s Quest Initiative NK was partially supported by funds from the Simons Foundation awarded to the Simons Center for the Social Brain at MIT, from the McGovern Institute for Brain Research, and MIT’s Quest Initiative References Zhang, Y., Kauf, C., Levy, R.

Tom: P., Gibson, E.

Jane: (2025).

Tom: Comparative illusions are evidence of rational inference in language comprehension.

Paper summary: Tom: So, to summarize "What does it mean to understand language?", the main thesis is that deep understanding requires exporting information from our core language system to other brain regions that compute perceptual and motor representations, construct mental models of the situation described, and store world knowledge and autobiographical memories.

Jane: They review the existing evidence for this hypothesis and argue that progress in cognitive neuroscience suggests this export mechanism is necessary.

Lu: The paper sets up a framework where the core language system is engaged in processing linguistic forms, but these representations are shallow because they are derived only from knowledge of language statistics, sort of like early versions of text-only large language models.

Meng: So what's the problem with that? If it’s just statistics, how can we get anything meaningful out of it?

Tom: The paper points out that these core representations lack sensitivity to real-world plausibility. They aren't really connected to actual experience.

Jane: And they suggest that deep understanding is supported by extra-linguistic systems that can augment those shallow representations with perceptual or motor features, or translate them into formats like probabilistic programming engines.

Lu: They identify several candidate exportation destinations, including regions specialized for thinking about someone's mind, scene understanding and navigation areas, regions that build mental models of the physical world, and amodal semantic information processing.

Conclusion: Tom: So, looking at "What does it mean to understand language?", the main implication is that if we want a real grasp of what someone is saying, we can't rely only on the core language system doing all the heavy lifting.

Jane: It means deep understanding isn't just about parsing grammar or finding word patterns; it’s about connecting those words to things happening in the real world, like knowing how an object moves or what a person is thinking.

Lu: The paper suggests that exporting information allows for building those rich mental models of people, objects, and places described in the language input.

Meng: So if we think about this from a practical standpoint, it means AI systems that want to be truly competent at understanding language need more than just massive text data; they need those connections to other sensory or physical representations.

Tom: Exactly. The authors suggest future research should focus on how language representations get translated into a format that other brain regions can actually process and what specific factors cause that information to be exported when it is.

Jane: It sets up a broad research program, suggesting we need to look at this from cognitive, neural, and computational angles to advance our theory of language understanding.

Lu: They also flag a limitation in their work: the strong empirical evidence for this exportation is currently restricted to only a few specific systems.

Kempner Institute for the Study of Natural & Artificial Intelligence at Harvard University · School of Psychology at Georgia Institute of Technology · Department of Brain and Cognitive Sciences at Massachusetts Institute of Technology

cs.CL

Submitted: 2025-11-24

Updated: 2026-11-11

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 89/100

The gist: Language understanding entails not just extracting the surface-level meaning of the linguistic input, but constructing rich mental models of the situation it describes The paper proposes that because…

Key concepts

Core Language System
This is the brain network highly specialized for processing language itself. It extracts information from word sequences based on language statistics but lacks sensitivity to real-world plausibility or context outside of linguistic patterns.
Exportation
The process where information generated by the core language system is transferred out to other brain regions. This transfer enriches the initial linguistic representation with perceptual features, motor data, or knowledge from other systems.
Shallow Understanding
This is the basic level of language comprehension where a system only recognizes words and how they fit together in a sentence structure. It relies solely on patterns learned from language use and ignores real-world context.
Deep Understanding
This involves constructing rich mental models of people, objects, and situations described in language. It is achieved when the core system exports information to systems that build models of minds, places, and physical realities.

Terminology

Summary

Language understanding entails not just extracting the surface-level meaning of the linguistic input, but constructing rich mental models of the situation it describes The paper proposes that because processing within the brain’s core language system is fundamentally limited, deeply understanding language requires exporting information from the language system to other brain regions that compute perceptual and motor representations, construct mental models, and store our world knowledge and autobiographical memories This framework suggests that deep understanding necessitates exporting information from the core language system to other cognitive and neural systems

The Core Language System's Role in Understanding

The core language system is specifically engaged in understanding (and producing) language, and not in other nonlinguistic cognitive processes fMRI research over the last 15 years has identified and characterized the brain’s core language system with new precision During comprehension, the language system’s goal is to extract information from word sequences, which requires recognizing familiar words and larger constructions and figuring out how they go together within the sentence However, we argue that this representation is shallow: it is derived solely from our knowledge of language statistics, akin to representations in early versions of text-only large language models Furthermore, the representations in the language system lack sensitivity to real-world plausibility

Mechanisms for Deep Understanding

Deep understanding is supported by extra-linguistic systems that can be engaged both by linguistic and nonlinguistic inputs These extra-linguistic systems can augment the shallow representations constructed by the core language system with perceptual or motor features, translate them into a format compatible with probabilistic programing engines or symbolic problem solvers, or transform them in some other way The paper identifies several candidate exportation destinations including regions specialized for thinking about the contents of someone else’s or one’s own mind, regions specialized for scene understanding and navigation, regions that construct mental models of the physical world, regions that represent and process amodal semantic information, perceptual and motor regions, and regions that integrate information into a coherent representation of the situation that is being described

Specific Exportation Destinations

The paper details several brain regions that support deep language understanding, such as those involved in theory of mind, intuitive physics, navigation and scene understanding For instance, theory of mind involves the right temporo-parietal junction (rTPJ) and the broader Theory of Mind network Intuitive physics involves regions of the parietal and frontal cortex in the Physics Network Navigation and scene understanding involve areas like the parahippocampal place area (PPA), occipital place area (OPA), and retrosplenial cortex (RSC) Perceptual regions can be engaged when understanding verbal descriptions of visual scenes, such as faces or bodies Motor representations are also implicated when understanding action words, linking to motor cortex regions Episodic and semantic knowledge is accessed through the Default Network A (DN-A) for episodic memory and amodal semantic regions for general conceptual knowledge

Factors Influencing Exportation

The likelihood of information exportation depends on several factors, including the language system's memory limitations, traits, states, and goals of the comprehender Specifically, input that extends beyond a sentence must be passed to other systems to construct temporally extended contexts Furthermore, the language system must be able to extract a representation abstract enough to interface with downstream systems The process of exportation can occur through two main possibilities: routing, where information is selectively directed, or broadcasting, where representations are sent to all possible downstream systems

Conclusion and Future Directions

The paper concludes that deep language understanding requires exporting information from the brain’s core language system to other systems that build mental models of the minds, objects, and places described in the linguistic input Caveats include the fact that strong empirical evidence for exportation is restricted to a few systems and methodological limitations restrict conclusions from past work Future research should focus on how language representations are translated into a format other brain regions can process and what specific factors influence whether and when information is exported Outstanding questions remain regarding the extent to which everyday language understanding takes place within the core language system The paper proposes a broad research program to advance a richer theory of what language understanding entails cognitively, neurally, and computationally

Glossary

Exportation: The process by which information delivered through language is transferred out of our core language system to one or more other brain regions for further processing Shallow understanding: Building a representation of linguistic input based solely on knowledge of patterns of language use Deep understanding: Building rich mental models of the people, objects, places, and situations described in a language input Situation model: A representation of the ‘situation’ that is being described by a language input Core language system: A network of frontal and temporal brain regions that are highly specialized for language processing, strongly functionally and anatomically interconnected, and causality important in comprehending and producing language Routing: One hypothesis for how information is exported from the language system to downstream systems, whereby information is selectively directed Broadcasting: Another hypothesis for how information is exported to downstream systems, whereby information is sent from the core language system to all possible downstream systems

Acknowledgments

We thank Andrea de Varda, Ted Gibson, Steve Piantadosi, Rebecca Saxe, Thomas Clark, and members of the Kanwisher lab for helpful discussion and comments on the manuscript We also thank Emalie McMahon for help with the brain maps for the figure CC was supported by the Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University EF was partially supported by U01 award NS121471 from NINDS, funds from the Simons Foundation awarded to the Simons Center for the Social Brain at MIT, from the McGovern Institute for Brain Research, and MIT’s Quest Initiative NK was partially supported by funds from the Simons Foundation awarded to the Simons Center for the Social Brain at MIT, from the McGovern Institute for Brain Research, and MIT’s Quest Initiative

References

Zhang, Y., Kauf, C., Levy, R. P., Gibson, E. (2025). Comparative illusions are evidence of rational inference in language comprehension. p. 5

Where the paper says it

Inferences of the engagement of specific mental processes from activation in particular anatomical locations (or "reverse inference" [137]) are strongest for brain areas that are functionally specific. For example, engagement of the FFA during language understanding would provide strong evidence that a face percept has been constructed, whereas engagement of anterior cingulate cortex would be consistent with numerous disparate cognitive processes.

Journal of Experimental Psychology: General, 154(10), 2752-2771 Bransford, J. p. 0
Where the paper says it

Colton Casto Kempner Institute for the Study of Natural & Artificial Intelligence Harvard University Cambridge, MA 02138 ccasto@mit. edu Anna Ivanova School of Psychology Georgia Institute of Technology Atlanta, GA 30332 a.

D., Barclay, J.R., Franks, J.J. (1972). Sentence memory: A constructive versus interpretive approach. p. 8
Where the paper says it

We briefly highlight three possible factors. First, because the language system is memory-limited and only processes short segments of language [100,108-109], input that extends beyond a sentence or so must be passed to other systems that can construct temporally extended contexts (such as the DN-A and the Theory of Mind networks [100]).

Cognitive Psychology, 3, 193-209 Johnson-Laird, P. p. 0
Where the paper says it

Colton Casto Kempner Institute for the Study of Natural & Artificial Intelligence Harvard University Cambridge, MA 02138 ccasto@mit. edu Anna Ivanova School of Psychology Georgia Institute of Technology Atlanta, GA 30332 a.

N. (1983). Mental models: Towards a cognitive science of language, inference, and consciousness (Cambridge, MA: Harvard University Press) van Dijk, T. p. 0
Where the paper says it

WHAT DOES IT MEAN TO UNDERSTAND LANGUAGE? Colton Casto Kempner Institute for the Study of Natural & Artificial Intelligence Harvard University Cambridge, MA 02138 ccasto@mit.

A., Kintsch, W. (1983). Strategies in discourse comprehension (New York: Academic Press) Bower, G. p. 4
Where the paper says it

This is not to say that the representations constructed by the language system are not useful or powerful. As noted above, language statistics reflect many aspects of the structure of the world, and—as we have learned from LLMs—training on language statistics alone is sufficient to produce grammatically well-formed and semantically coherent discourse, answer questions, and carry out simple conversations [50-51].

H., Morrow, D.G. (1990). Mental models in narrative comprehension. p. 9
Where the paper says it

Concluding remarks In this opinion piece, we have proposed that deep language understanding requires exporting information from the brain’s core language system to other systems that build mental models of the minds, objects, and places described in the linguistic input, that store our memories and word knowledge, and that contain our perceptual and motor representations. Some evidence for this idea already exists, but our goal here is to outline a conceptual framework for thinking about language comprehension in the brain beyond the parsing of sentences and construction of shallow approximations of linguistic meaning carried out in the language network, and to outline many testable hypotheses, as we tried to do above.

Science, 247(4938), 44-8 Gernsbacher, M.A. (1990). Language comprehension as structure building (Hillsdale, NJ: Erlbaum) Graesser, A. p. 1
Where the paper says it

And are you now linking the meaning of this sentence to the previous one, to the unfolding structure of the argument in this article, and to thoughts you have had previously about language processing? In this opinion piece, we argue that a deep understanding of language, which entails building mental models and rich representations of meaning that connect to our broader knowledge of the world, requires the exportation of information from the brain’s core language system to other cognitive and neural systems that can build models of what we are hearing or reading.

C., Singer, M., Trabasso, T. (1994). Constructing inferences during narrative text comprehension. p. 0
Where the paper says it

edu Nancy Kanwisher Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA 02139 ngk@mit. edu November 26, 2025 ABSTRACT Language understanding entails not just extracting the surface-level meaning of the linguistic input, but constructing rich mental models of the situation it describes.

Psychological Review, 101(3), 371-95 Zwaan, R. p. 3
Where the paper says it

Many past studies have reported neural responses in cortical regions outside the classic language areas when people process words, sentences, or entire stories (e. , [68,72-73,79-80,86,94,23,96,101,104]).

A., Radvansky, G.A. (1998). Situation models in language comprehension and memory. p. 0
Where the paper says it

edu Nancy Kanwisher Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA 02139 ngk@mit. edu November 26, 2025 ABSTRACT Language understanding entails not just extracting the surface-level meaning of the linguistic input, but constructing rich mental models of the situation it describes.

Psychological Bulletin, 123(2), 162-85 Jackendoff, R. p. 3
Where the paper says it

Some have interpreted these findings as evidence that no single part of the brain is specialized for language processing but rather “the entire brain is at work” (e. , [123-124]).

S., Erk, K.E. (2025). Toward a Deeper Lexical Semantics. p. 2
Where the paper says it

Areas outside the language system support deep understanding B. The core language system supports shallow language understanding theory of mind navigation intuitive physics amodal semantics perception and motor situation modeling she thought that he loved her… Sid went to the kitchen… the bouncy ball rolled off the table… cats are common household pets… his dark brown eyes sparkled in the light… She thought that he loved her. p. 2

Where the paper says it

Areas outside the language system support deep understanding B. The core language system supports shallow language understanding theory of mind navigation intuitive physics amodal semantics perception and motor situation modeling she thought that he loved her… Sid went to the kitchen… the bouncy ball rolled off the table… cats are common household pets… his dark brown eyes sparkled in the light… She thought that he loved her.

Topics in Cognitive Science Fedorenko, E. p. 0
Where the paper says it

ivanova@gatech. edu Evelina Fedorenko Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA 02139 evelina9@mit.

, Ivanova, A. p. 0
Where the paper says it

Colton Casto Kempner Institute for the Study of Natural & Artificial Intelligence Harvard University Cambridge, MA 02138 ccasto@mit. edu Anna Ivanova School of Psychology Georgia Institute of Technology Atlanta, GA 30332 a.

A., Regev, T.I., The language network as a natural kind within the broader landscape of the human brain Kanwisher, N. p. 0
Where the paper says it

edu Evelina Fedorenko Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA 02139 evelina9@mit. edu Nancy Kanwisher Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA 02139 ngk@mit.

p. 6
Where the paper says it

Another kind of mental model concerns the map of space we use to navigate around in the world. Extensive work in human cognitive neuroscience has identified key brain regions underlying our navigational ability (see [19] for review), including the parahippocampal place area (PPA, [12]), the occipital place area (OPA, [70]), and retrosplenial cortex (RSC, or “medial place area”, MPA [12,71]).

, McDermott, J., Chun, M. (1997). The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for the Perception of Faces. p. 1
Where the paper says it

A key finding from this research is that the core language system is specifically engaged in understanding (and producing) language, and not in other nonlinguistic cognitive processes (see [10] for review). Further, many other cortical regions show remarkable specificity for other mental functions, including regions in the ventral visual pathway selectively engaged in perception of faces, bodies, and scenes [11-13], regions of auditory cortex selective for the perception of speech sounds and music [14-15], the right temporo-parietal junction (rTPJ), which is selectively engaged in thinking about what others are thinking [16], and the other regions comprising the theory of mind network [17-18], and multiple regions engaged in visually-guided navigation [19] and in reaching and grasping [20].

Journal of Neuroscience, 17, 4302-4311 Norman-Haignere, S. p. 0
Where the paper says it

Here we propose that because processing within the brain’s core language system is fundamentally limited, deeply understanding language requires exporting information from the language system to other brain regions that compute perceptual and motor representations, construct mental models, and store our world knowledge and autobiographical memories. We review the existing evidence for this hypothesis, and argue that recent progress in cognitive neuroscience provides both the conceptual foundation and the methods to directly test it, thus opening up a new strategy to reveal what it means, cognitively and neurally, to understand language.

, Kanwisher, N. p. 0
Where the paper says it

edu Evelina Fedorenko Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA 02139 evelina9@mit. edu Nancy Kanwisher Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA 02139 ngk@mit.

p. 0
Where the paper says it

edu Evelina Fedorenko Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA 02139 evelina9@mit. edu Nancy Kanwisher Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA 02139 ngk@mit.

p. 0
Where the paper says it

edu Evelina Fedorenko Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA 02139 evelina9@mit. edu Nancy Kanwisher Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA 02139 ngk@mit.

, McDermott, J. (2015). Distinct Cortical Pathways for Music and Speech Revealed by Hypothesis-Free Voxel Decomposition. p. 1
Where the paper says it

A key finding from this research is that the core language system is specifically engaged in understanding (and producing) language, and not in other nonlinguistic cognitive processes (see [10] for review). Further, many other cortical regions show remarkable specificity for other mental functions, including regions in the ventral visual pathway selectively engaged in perception of faces, bodies, and scenes [11-13], regions of auditory cortex selective for the perception of speech sounds and music [14-15], the right temporo-parietal junction (rTPJ), which is selectively engaged in thinking about what others are thinking [16], and the other regions comprising the theory of mind network [17-18], and multiple regions engaged in visually-guided navigation [19] and in reaching and grasping [20].

Neuron, 88(6), 1281-96 Overath, T., McDermott, J.H., Zarate, J.M., Poeppel, D. (2015) The cortical analysis of speech-specific temporal structure revealed by responses to sound quilts. p. 5
Where the paper says it

Because the anatomical location of functional regions varies across participants, standard group analyses blur the responses of each region with its anatomical neighbors. Thus, a precise measure of the engagement of a given functionally-specific cortical region requires first identifying that region within individual brains with a standard ‘localizer’ task.

Nature Neuroscience, 18(6), 903-11 Saxe, R. p. 1
Where the paper says it

Although this long tradition of behavioral work has provided many clues about the representations extracted during language processing, the precise nature of these representations, and the conditions under which they are constructed during language understanding, has remained elusive. Cognitive neuroscience has much to offer to these questions, by providing both a rich taxonomy of mental processes that selectively engage specific brain regions, and the methods to directly measure that recruitment during language understanding.

, Kanwisher, N. (2003). People thinking about thinking people: The role of the temporo-parietal junction in theory of mind. p. 1
Where the paper says it

A key finding from this research is that the core language system is specifically engaged in understanding (and producing) language, and not in other nonlinguistic cognitive processes (see [10] for review). Further, many other cortical regions show remarkable specificity for other mental functions, including regions in the ventral visual pathway selectively engaged in perception of faces, bodies, and scenes [11-13], regions of auditory cortex selective for the perception of speech sounds and music [14-15], the right temporo-parietal junction (rTPJ), which is selectively engaged in thinking about what others are thinking [16], and the other regions comprising the theory of mind network [17-18], and multiple regions engaged in visually-guided navigation [19] and in reaching and grasping [20].

NeuroImage, 19, 1835-1842 DiNicola, L.M., Braga, R.M., Buckner, R.L., Parallel distributed networks dissociate episodic and social functions within the individual Deen, B. p. 1
Where the paper says it

Further, many other cortical regions show remarkable specificity for other mental functions, including regions in the ventral visual pathway selectively engaged in perception of faces, bodies, and scenes [11-13], regions of auditory cortex selective for the perception of speech sounds and music [14-15], the right temporo-parietal junction (rTPJ), which is selectively engaged in thinking about what others are thinking [16], and the other regions comprising the theory of mind network [17-18], and multiple regions engaged in visually-guided navigation [19] and in reaching and grasping [20]. Ongoing research is investigating the precise function of many other brain regions, including those engaged in intuitive physical reasoning [21], the perception of social agents and their actions [22], amodal semantic processing [23], episodic retrieval and self-projection [24,17], as the domain-general processing characteristic of the ‘Multiple-Demand’ system [25] that is engaged by a broad range of cognitively demanding tasks.

, Freiwald, W. (2022). Parallel systems for social and spatial reasoning within the cortical apex Epstein, R. p. 3
Where the paper says it

, Does the input involve spatial reasoning? Does the input describe a person or social interaction?

A., Baker, C.I. (2019). Scene Perception in the Human Brain. p. 1
Where the paper says it

Further, many other cortical regions show remarkable specificity for other mental functions, including regions in the ventral visual pathway selectively engaged in perception of faces, bodies, and scenes [11-13], regions of auditory cortex selective for the perception of speech sounds and music [14-15], the right temporo-parietal junction (rTPJ), which is selectively engaged in thinking about what others are thinking [16], and the other regions comprising the theory of mind network [17-18], and multiple regions engaged in visually-guided navigation [19] and in reaching and grasping [20]. Ongoing research is investigating the precise function of many other brain regions, including those engaged in intuitive physical reasoning [21], the perception of social agents and their actions [22], amodal semantic processing [23], episodic retrieval and self-projection [24,17], as the domain-general processing characteristic of the ‘Multiple-Demand’ system [25] that is engaged by a broad range of cognitively demanding tasks.

Annual Review of Vision Science, 5, 373-397 Culham, J. p. 0
Where the paper says it

Here we propose that because processing within the brain’s core language system is fundamentally limited, deeply understanding language requires exporting information from the language system to other brain regions that compute perceptual and motor representations, construct mental models, and store our world knowledge and autobiographical memories. We review the existing evidence for this hypothesis, and argue that recent progress in cognitive neuroscience provides both the conceptual foundation and the methods to directly test it, thus opening up a new strategy to reveal what it means, cognitively and neurally, to understand language.

C., Valyear, K.F. (2006). Human parietal cortex in action. p. 3
Where the paper says it

[72] for a similar study from the same authors that did), at a coarse level, their results appear to be largely consistent with the known functional organization of the human brain. For example, the authors found that brain responses near the parahippocampal place area (PPA) and retrosplenial cortex (RSC) were most strongly predicted by ratings of whether the input mentioned a specific location, and that responses near the right temporo-parietal junction (rTPJ) were most strongly predicted by whether the input described a social interaction [73].

Current Opinion in Neurobiology, 16(2):205-12 Fischer, J. p. 7
Where the paper says it

[86] reported that the emotional valence of the words in short passages from Harry Potter modulated responses in the amygdala. These findings provide suggestive evidence for the exportation hypothesis although i) current evidence suggests that few if any brain regions are selectively engaged by emotional processing (e.

, Tenenbaum, J., Kanwisher, N. (2016). The functional neuroanatomy of intuitive physical inference. p. 6
Where the paper says it

In many studies of the ToM network, participants are asked to read a short passage and then choose the correct completion of the story, which requires inferring the beliefs of a character in the passage. The ToM network responds strongly during this task (and the response is locked to the part of the passage that talks about mental states), but not during a logically similar task in which the passage requires an inference about a physical representation [16].

PNAS, 113(34), E5072-81 Hauk, O. p. 7
Where the paper says it

Early work on "embodied" representations argued that motor representations are necessarily engaged for understanding the meanings of action words. For example, Hauk et al.

, Johnsrude, I., Pulvermüller, F. (2004). Somatotopic representation of action words in human motor and premotor cortex. p. 7
Where the paper says it

[54] famously reported that passive reading of words describing actions of the face (“lick”), arm (“pick”) or leg (“kick”) activated regions of motor cortex overlapping with or adjacent to those activated by corresponding actual movements (see also [79]). Although these early studies faced several methodological limitations (see Box 3 for how these questions can be tested more rigorously), a more recent intracranial study found cross-decoding of neural responses from action words to videos depicting those actions [80], more strongly supporting the exportation of action-related meanings.

Neuron, 41(2), 301-7 Nakamura, K., Kawashima, R., Sato, N., Nakamura, A., Sugiura, M., Kato, T., Hatano, K., Ito, K., Fukuda, H., Schormann, T., Zilles, K. (2000). Functional delineation of the human occipito-temporal areas related to face and scene processing—a PET study O’Craven, K. p. 7
Where the paper says it

[54] famously reported that passive reading of words describing actions of the face (“lick”), arm (“pick”) or leg (“kick”) activated regions of motor cortex overlapping with or adjacent to those activated by corresponding actual movements (see also [79]). Although these early studies faced several methodological limitations (see Box 3 for how these questions can be tested more rigorously), a more recent intracranial study found cross-decoding of neural responses from action words to videos depicting those actions [80], more strongly supporting the exportation of action-related meanings.

Kanwisher, N. (2000). Mental imagery of faces and places activates corresponding stimulusspecific brain regions. p. 6
Where the paper says it

An engaging novel or story can conjure up vivid mental imagery. According to the exportation hypothesis, this is possible due to exporting information to corresponding brain areas.

Journal of Cognitive Neuroscience, 12, 1013-1023

--- Page 6 ---

A. The language system exports information to other, non-language-selective brain areas during comprehension C. p. 2

Where the paper says it

The language system exports information to other, non-language-selective brain areas during comprehension C. Areas outside the language system support deep understanding B.

Areas outside the language system support deep understanding B. p. 2
Where the paper says it

The language system exports information to other, non-language-selective brain areas during comprehension C. Areas outside the language system support deep understanding B.

The core language system supports shallow language understanding theory of mind navigation intuitive physics amodal semantics perception and motor situation modeling she thought that he loved her… Sid went to the kitchen… the bouncy ball rolled off the table… cats are common household pets… his dark brown eyes sparkled in the light… She thought that he loved her. p. 2
Where the paper says it

Areas outside the language system support deep understanding B. The core language system supports shallow language understanding theory of mind navigation intuitive physics amodal semantics perception and motor situation modeling she thought that he loved her… Sid went to the kitchen… the bouncy ball rolled off the table… cats are common household pets… his dark brown eyes sparkled in the light… She thought that he loved her.

It processes linguistic forms. p. 2
Where the paper says it

The core language system supports shallow language understanding theory of mind navigation intuitive physics amodal semantics perception and motor situation modeling she thought that he loved her… Sid went to the kitchen… the bouncy ball rolled off the table… cats are common household pets… his dark brown eyes sparkled in the light… She thought that he loved her. It processes linguistic forms.

Meg breezed through security before arriving at her gate… It constructs a form-independent representation. p. 2
Where the paper says it

It processes linguistic forms. Meg breezed through security before arriving at her gate… It constructs a form-independent representation.

Paraphrase Sentence Translation = They enrich/augment language representations. p. 2
Where the paper says it

Meg breezed through security before arriving at her gate… It constructs a form-independent representation. Paraphrase Sentence Translation = They enrich/augment language representations.

She thought that he loved her. They process linguistic and non-linguistic input

--- Page 7 ---

Figure 1: Hypothesis: Exportation of Information from the Core Language System to Other Brain Regions Is Necessary for Deep Understanding A) The language system exports information to extra-linguistic systems that support deep understanding of linguistic content Candidate exportation destinations include regions specialized for thinking about the contents of someone else’s or one’s own mind [16], regions specialized for scene understanding and navigation [19], regions that construct mental models of the physical world, regions that represent and process amodal semantic information, perceptual and motor regions (e.g., [13- 20]), and regions that integrate information into a coherent representation of the situation that is being described These exportation destination regions are visualized with broad masks delineating where the relevant functional regions are found; although these masks may overlap, the functional regions within individuals are distinct B) The core language system supports shallow language understanding entails recognizing individual words and how they go together within the structure of the sentence (i. p. 2

Where the paper says it

, [11-13]), and regions that integrate information into a coherent representation of the situation that is being described [95-96]. These exportation destination regions are visualized with broad masks delineating where the relevant functional regions are found; although these masks may overlap, the functional regions within individuals are distinct.

e., parsing the surface form of the sentence), and then constructing a compositional representation of the sentence’s meaning, perhaps represented in a high-dimensional vector space Importantly, these shallow representations are only defined with respect to other linguistic elements and patterns of their use that we have encountered during our lifetime of experience with language, making no contact with other aspects of our lived experience C) Extra-linguistic systems support deep language understanding They can be engaged both by linguistic and nonlinguistic inputs, which is why we do not consider them part of the language system During language understanding these regions may augment the shallow representations constructed by the core language system with perceptual or motor features, translate them into a format compatible with probabilistic programing engines or symbolic problem solvers, or transform them in some other way

--- Page 8 ---

How much of language understanding occurs within the core language system? fMRI research over the last 15 years has identified and characterized the brain’s core language system with new precision Damage to these regions leads to linguistic deficits All regions of the language network show sensitivity to structure at the sublexical, word, and sentence level During comprehension, the language system’s goal is to extract information from word sequences, which requires recognizing familiar words and larger constructions and figuring out how they go together within the sentence The end product is a representation that captures both structural and co-occurrence-based information encoded in a sentence Several lines of evidence suggest that this representation is quite abstract, no longer tied to the particular words For example, representations appear to be similar across sentence paraphrases and across languages Furthermore, the representations constructed within the language system share some features with representations extracted from meaningful visual inputs

--- Page 9 ---

Critically, although abstract, we argue that these meaning-approximating representations are shallow: they are derived solely from our knowledge of language statistics, akin to representations in early versions of text-only large language models Even severe damage to these brain areas can leave conceptual processing intact, suggesting that whatever representations and computations they support cannot be critical to world knowledge and reasoning Furthermore, the representations in the language system lack sensitivity to real-world plausibility

--- Page 10 ---

Do any AI systems understand language deeply? If so, then we might expect them to show signatures of exportation Several recent studies have attempted to explicitly build exportation into AI systems by augmenting LLMs with diverse extralinguistic systems: vision models, physics engines, formal logic provers, theory of mind engines, memory augmentations, and many more It is important to note that exportation obviously has to happen when linguistic inputs call for an overt response or behavior

How the training signal is built

The paper discusses how AI systems can mimic human-like language understanding by augmenting LLMs with diverse extralinguistic systems such as vision models, physics engines, formal logic provers, theory of mind engines, memory augmentations, and many more Other studies have explored architectures that allow for functional specializations to be learned over the course of training There is also some evidence that individual units within a standard LLM can become functionally specialized AI systems which do have exportation (whether neuro-symbolic or neural-net-only) can serve as useful model organisms for human-like language understanding

Where the gains come from

The gains in AI models come from augmenting LLMs with diverse extralinguistic systems to achieve functional competence tasks The ability of these models to predict brain responses across stories is better when they have more powerful architectures AI systems which do have exportation can serve as useful model organisms for human-like language understanding …

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Improve AI to construct rich mental models by exporting information from its core language system to regions specialized for thinking about the contents of someone else’s or one’s own mind. This allows the AI to build rich representations of meaning that connect to our broader knowledge of the world, enabling it to move beyond surface-level meaning.

  2. Enhance AI's ability to handle complex situations by leveraging exportation to regions specialized for navigation and scene understanding. The system can then construct mental models of what we are hearing or reading, allowing it to integrate linguistic input with perceptual features like spatial locations and objects.

  3. Develop AI systems capable of intuitive reasoning by exporting information to regions that construct mental models of the physical world. This enables the AI to predict what will happen next, and plan actions based on verbal descriptions of physical scenarios.

  4. Augment language processing with theory of mind capabilities by engaging regions like the right temporo-parietal junction when processing linguistic input about mental states. The system can then build mental models of someone else’s mind, allowing it to infer beliefs even without explicit questions.

  5. Integrate episodic and semantic knowledge into language understanding by exporting information to networks representing episodic retrieval and self-projection and amodal semantic processing. This allows the AI to use its own lived experience or general world knowledge when interpreting language, leading to deep understanding.

  6. Implement a mechanism for dynamic context management by utilizing exportation pathways that allow the system to pass information between the language system and networks like the default network. This enables it to construct mental models of what we are hearing or reading over long contexts, rather than just processing short segments.

  7. Improve robustness against linguistic ambiguity by incorporating mechanisms that allow for routing information rather than simple broadcasting. This allows the system to selectively direct (e.g., by the language system or a mediating brain region) to the most appropriate downstream system based on linguistic context, increasing accuracy when meaning is not immediately obvious.

  8. Increase adaptability by allowing exportation pathways that are sensitive to comprehender traits and goals. The AI can be designed so that the likelihood of exportation in the latter case [listening after diagnosis] is much higher, allowing it to adjust its depth of understanding based on its current state or knowledge level about the topic.

Abstract

Language understanding entails not just extracting the surface-level meaning of the linguistic input, but constructing rich mental models of the situation it describes. Here we propose that because processing within the brain's core language system is fundamentally limited, deeply understanding language requires exporting information from the language system to other brain regions that construct mental models of minds, objects, and places, and that compute perceptual and motor representations. Building on earlier theorizing and empirical work, we argue that recent progress in cognitive neuroscience provides both the conceptual foundation and the methods to directly evaluate this hypothesis, thus opening up a new strategy to reveal what it means, cognitively and neurally, to understand language.

Related papers