Scratchpad Patching: Decoupling Compute from Patch Size in Byte-Level Language Models
cs.CL, cs.LG
Submitted: 2026-05-10
Updated: 2026-11-06
Code: https://github.com/microsoft/guidance
Terminology
Sources
- MAGNET: Improving the Multilingual Fairness of Language Models with Adaptive Gradient-Based Tokenization
- Program Synthesis with Large Language Models
- Large Concept Models: Language Modeling in a Sentence Representation Space
- Conditional Computation in Neural Networks for faster models
- Evaluating Large Language Models Trained on Code
- Bridging the Gap for Tokenizer-Free Language Models
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- No Language Left Behind: Scaling Human-Centered Machine Translation
- Getting the most out of your tokenizer for pre-training and domain adaptation
- Improving Language Understanding from Screenshots
- Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
- Better & Faster Large Language Models via Multi-token Prediction
- Gemini: A Family of Highly Capable Multimodal Models
- Generating Sequences With Recurrent Neural Networks
- Adaptive Computation Time for Recurrent Neural Networks
- OLMES: A Standard for Language Model Evaluations
- DC-DiT: Adaptive Compute and Elastic Inference for Visual Generation via Dynamic Chunking
- ConceptMoE: Adaptive Token-to-Concept Compression for Implicit Compute Allocation
- Dynamic Chunking for End-to-End Hierarchical Sequence Modeling
- Perceiver IO: A General Architecture for Structured Inputs & Outputs
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data