Lattice Deduction Transformers
cs.LG, cs.AI, cs.LO
Submitted: 2026-05-09
Updated: 2026-11-03
Code: https://github.com/chenglou/sotaku
License: http://creativecommons.org/licenses/by/4.0/
The gist: We introduce the Lattice Deduction Transformer (LDT), a recurrent transformer that approximates logically sound deduction by projecting its latent state through a lattice between forward passes.
Terminology
Abstract
We introduce the Lattice Deduction Transformer (LDT), a recurrent transformer that approximates logically sound deduction by projecting its latent state through a lattice between forward passes. We train on-policy in a process that mirrors deduction in a search-based constraint solver and supervise training via a domain-agnostic approximation of the set of solution candidates based on abstract interpretation. An 800 K-parameter LDT outperforms prior small recurrent reasoners on Sudoku-Extreme with under 2% of their estimated training FLOPs, solving 99.999% of the 422, 786 test puzzles. The model is nearly always sound, in that it gives a wrong answer for 4 of 1.27 M puzzle evaluations, and otherwise returns a correct answer or abstains. A 1.8 M-parameter LDT variant reaches 99.8% accuracy on Maze-Hard. Our code is available at github.com/lcrh/lattice-deduction-transformers.
Sources
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis