RepBench: Compiling Benchmarks into Capability Representations for Large Language Models
Yanshi Li, Xueru Bai, Shuman Liu, Long Zhang
cs.CL, cs.AI
Submitted: 2026-07-30
Comments: 22 pages, 8 figures, with appendices. Yanshi Li and Xueru Bai contributed equally
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Toward universal steering and monitoring of AI models
- Predicting Where Steering Vectors Succeed
- Beyond Demonstrations: Dynamic Vector Construction from Latent Representations
- Activation Reward Models for Few-Shot Model Alignment
- Latent Performance Profiling of Large Language Models
- What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal
- Decomposing Theory of Mind: How Emotional Processing Mediates ToM Abilities in LLMs
- Convergent Evolution: How Different Language Models Learn Similar Number Representations
- Towards Intrinsic Interpretability of Large Language Models:A Survey of Design Principles and Architectures
- Verbalizable Representations Form a Global Workspace in Language Models
- Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs
- Segment-Level Coherence for Robust Harmful Intent Probing in LLMs
- Loss Landscape Degeneracy and Stagewise Development in Transformers
- Cognitive Foundations for Reasoning and Their Manifestation in LLMs
- Is This Just Fantasy? Language Model Representations Reflect Human Judgments of Event Plausibility
- Decomposing and Steering Functional Metacognition in Large Language Models
- Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry
- Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
- Understanding and Enforcing Weight Disentanglement in Task Arithmetic
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering