WavePrune: One period is often enough for RoPE
cs.CL, cs.SD
Submitted: 2026-10-03
Updated: 2026-10-03
Code: https://github.com/ducatiqwq/WavePrune
Sources
- Program Synthesis with Large Language Models
- Longformer: The Long-Document Transformer
- Evaluating Large Language Models Trained on Code
- Extending Context Window of Large Language Models via Positional Interpolation
- Generating Long Sequences with Sparse Transformers
- LongNet: Scaling Transformers to 1,000,000,000 Tokens
- RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably
- The Llama 3 Herd of Models
- ATFlash: Per-RoPE-Wavelength Attention Windows for Compute/Memory-Efficient LLM Inference
- CoPE: Clipped RoPE as A Scalable Free Lunch for Long Context LLMs
- Ministral 3
- Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
- Gemma 3 Technical Report
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering