TOPol: Reconstructing and Interpreting Multidimensional Semantic Polarity Fields
cs.CL
Submitted: 2025-10-29
Updated: 2026-10-31
Comments: 16 pages, 8 figures and 5 tables
License: http://creativecommons.org/licenses/by/4.0/
The gist: Polarity in natural language processing has long been operationalized as a single axis: the sentiment construct.
Terminology
Abstract
Polarity in natural language processing has long been operationalized as a single axis: the sentiment construct. We challenge this one-dimensional view by asking what a topic's documents are bipolar about, without specifying an answer in advance. TOPol inverts the usual concept-direction paradigm: rather than mapping a named construct to one direction, it maps a topic to the full set of statistically reproducible bipolar semantic axes its documents support in a transformer's residual stream, and asks only afterwards which of them correspond to a known construct. These axes need not be evaluative. The method is a recursive, orthogonal-by-construction bipartition of each topic's document cloud, with a candidate axis promoted to a finding only if it survives one pre-specified split-half stability gate. We calibrate against a known answer first: on 9,986 Amazon reviews (14 discovered topics) in Gemma 3 4B, the star-rating difference-in-means vector serves as a ruler known by construction to be sentiment. Discovery blindly re-derives the sentiment axis in every topic (cosine 0.71-0.97) without ever seeing the rating label, while the same gate certifies 543 axes, of which 106 are near-orthogonal to sentiment yet equally reproducible; each topic supports 33-45 mutually orthogonal axes, and sentiment axes resolve in later layers than the rest. Differential vocabulary, an internals-grounded natural-language decoder and vocabulary-based clustering agree on what the non-sentiment axes contrast; clustering by shared vocabulary surfaces 99 families, 9 recurring across topics. Direction similarity to the ruler predicts an independent sentiment estimate only weakly (r=0.243). Polarity is better treated as a property of a topic's representational geometry, with a measurable rank and a per-axis depth, than as one universal scalar.
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data