Lattice Deduction Transformers

arXiv:2605.08605 · cs.LG, cs.AI, cs.LO · Submitted 2026-05-09 · Read on arXiv

cs.LG, cs.AI, cs.LO

Submitted: 2026-05-09

Updated: 2026-11-03

Code: https://github.com/chenglou/sotaku

License: http://creativecommons.org/licenses/by/4.0/

The gist: We introduce the Lattice Deduction Transformer (LDT), a recurrent transformer that approximates logically sound deduction by projecting its latent state through a lattice between forward passes.

Terminology

Abstract

We introduce the Lattice Deduction Transformer (LDT), a recurrent transformer that approximates logically sound deduction by projecting its latent state through a lattice between forward passes. We train on-policy in a process that mirrors deduction in a search-based constraint solver and supervise training via a domain-agnostic approximation of the set of solution candidates based on abstract interpretation. An 800 K-parameter LDT outperforms prior small recurrent reasoners on Sudoku-Extreme with under 2% of their estimated training FLOPs, solving 99.999% of the 422, 786 test puzzles. The model is nearly always sound, in that it gives a wrong answer for 4 of 1.27 M puzzle evaluations, and otherwise returns a correct answer or abstains. A 1.8 M-parameter LDT variant reaches 99.8% accuracy on Maze-Hard. Our code is available at github.com/lcrh/lattice-deduction-transformers.

Sources

Related papers