Where in the Input Does a One-Pass Decision Encoder Look?
cs.CL, cs.AI
Submitted: 2026-11-12
Updated: 2026-11-12
Code: https://github.com/NandhaKishorM/laya
Terminology
Sources
- Lost in the Middle: How Language Models Use Long Contexts
- Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
- Revisiting Transformer-based Models for Long Document Classification
- Longformer: The Long-Document Transformer
- Hierarchical Transformers for Long Document Classification
- DocBERT: BERT for Document Classification
- Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models
- AnyJev Technical Report
- TypedBench: A Benchmark for Calibration, Framing Sensitivity, and Cost in System One Decision Models
- Joint or Per-Option? Compact Encoders for Closed-Set Decisions from 17 Million to 1 Billion Parameters
- Focused Transformer: Contrastive Training for Context Scaling
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data