Isotropy Cliffs: The Geometric Signature of Decision-Making in Large Language Models
cs.AI
Submitted: 2026-08-01
Updated: 2026-11-11
Terminology
Sources
- Emergence of a High-Dimensional Abstraction Phase in Language Transformers
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Persistent Topological Features in Large Language Models
- Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models
- The Llama 3 Herd of Models
- Effect of Selection Format on LLM Performance
- Mistral 7B
- Geometry of Decision Making in Language Models
- Spectral Insights into Data-Oblivious Critical Layers in Large Language Models
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
- Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering
- All-but-the-Top: Simple and Effective Postprocessing for Word Representations
- Navigating Prompt Complexity for Zero-Shot Classification: A Study of Large Language Models in Computational Social Science
- Semantic Consistency for Assuring Reliability of Large Language Models
- The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models
- Leveraging Large Language Models for Multiple Choice Question Answering
- From Topology to Retrieval: Decoding Embedding Spaces with Unified Signatures
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting
- All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality
- Redundancy, Isotropy, and Intrinsic Dimensionality of Prompt-based Text Embeddings
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents