BDH-CQ: In-Context Learning with Recurrent Latent Reasoning

BDH-CQ is a reasoning model that combines in-context learning with recurrent latent reasoning. Inputs presented at inference time continuously update the model’s recurrent memory; the model then solves a query through iterative computation in a high-dimensional latent space, without verbalizing its intermediate reasoning. The model is evaluated on the public ARC-AGI-1 evaluation set and uses controlled ARC-like interventions to study what it learns from demonstrations, how consistently it applies an inferred transformation, and which concepts remain difficult. A 150M-parameter configuration reaches 29.5% pass@2 at a computed inference cost of 0.0007 per task—less than one-tenth of a cent. This operating point breaks through the previously reported ARC-AGI-1 cost–accuracy Pareto frontier, establishing a new state of the art in benchmark cost efficiency.

The paper introduces BDH-CQ, which brings together in-context learning and latent reasoning. Demonstrations of a previously unseen task update recurrent memory; the query is then solved through iterative computation in a high-dimensional latent workspace. Intermediate reasoning states are not decoded into language. BDH-CQ makes memory, adaptation, and inference part of the same computational fabric. Neither task identifiers nor evaluation-task demonstration pairs participate in training, and no parameters are updated at inference time.

The system builds on the Dragon Hatchling (BDH) architecture, a post-Transformer sequence-model architecture built around high-dimensional positive activations, low-rank communication, and a recurrent associative state. BDH-CQ extends this into a reasoning system that combines a structured latent workspace and recurrent computation over model depth with an interface for learning visual transformations from demonstrations.

The recurrent memory evolves as St = Uθ(St−1, Dt), where Dt denotes the content of the t-th demonstration and θ remains fixed. After the demonstrations and query have been ingested, BDH-CQ performs iterative computation in a structured latent workspace Hr. The contextual memory St and reasoning workspace Hr have different conceptual roles: St changes as evidence is encountered and supports in-context learning, while Hr carries the ongoing computation used to answer the current query.

Training uses a curated collection of ARC-style data combining privately curated examples with publicly available data from ARC-AGI-1 training set, RE-ARC, ConceptARC, ARC-Heavy, and ARC-GEN100K, with additional augmentations.

On the public ARC-AGI-1 evaluation, the 150M-parameter system reaches 29.5% pass@2 in approximately 0.85 H200 GPU-seconds per task. At 3 per H200-hour this gives a computed cost of 0.00070 per task. This point lies beyond the previously reported Pareto frontier. Using ARC Prize’s reported costs as of July 2026, BDH-CQ is approximately 57x cheaper than GPT 5.6 Luna (Low) which scores 34.2% at 0.040. When accounting for OpenAI’s 80% public API price reduction of GPT 5.6 Luna on July 30, 2026, BDH-CQ is approximately 11x cheaper than GPT 5.6 Luna (Low). An independent black-box audit conducted by co-authors from Bielik and New York University reproduced the deployed system’s 29.5% pass@2 score on the public ARC-AGI-1 evaluation set.

Behavioral analysis on ConceptARC shows performance varies substantially across concept families. In the opaque-identifier replication, ExtendToBoundary, FilledNotFilled, and TopBottom2D each reach 9/10 pass@2, whereas Copy and Order reach 2/10. Several families show a substantial difference between test-pair and strict-task accuracy. Copy, for example, reaches 19/30 semantic test-pair pass@2 but only 2/10 strict tasks. The system can produce correct outputs for many inputs in this family, but does not yet apply the inferred transformation consistently across all inputs of a task.

Controlled experiments with post-freeze generators show sharply different generalization patterns. Propagation remains correct on 48/48 held-out outputs at both pass@1 and pass@2 across distances 2–8, and copying remains correct on 48/48 as the number of target sites increases from one to four. Ordering is nearly saturated through five objects, then falls to 29/36 outputs at length six, 8/24 at seven, and 1/24 at eight at pass@2. Nesting remains nearly saturated through depth four but falls to 29/36 at depth five. The two declines have different signatures: ordering failures affect construction of the output as a whole, whereas nesting failures usually preserve its structure and make a localized relational error.

Matched support raises depth-five nesting from 19/24 to 24/24 exact outputs at pass@2. For ordering, it recovers 13/24 outputs from a 0/24 baseline. The nesting cliff is therefore largely a failure to extrapolate the demonstrated relation depth. Long ordering also benefits from support, but retains an execution bottleneck.

For contextual memory, the system solves all 96 held-out outputs at rank one as the number of simultaneous bindings increases from two to eight (24/24 at every level), recovering and consistently applying a relatively dense task-specific mapping introduced entirely through context.

For composition, relocation alone, atomic reflection, and atomic rotation are each solved on 72/72 held-out outputs. Rotation composed with relocation is solved on 72/72 tasks, while reflection composed with relocation is solved on 47/72 tasks. Color swap is acquired atomically only in the original family (26/72 pooled) and the model never learns to compose it with relocation (0/72).

Within-task consistency shows an 18.5-point gap between semantic ConceptARC pair accuracy (77.92%) and strict task accuracy (59.38%). Under pass@2, the system solved zero test pairs for 13 tasks, one for 15 tasks, two for 37 tasks, and all three for 95 tasks. Thus, 52/160 tasks have one or two correct test inputs but are not solved as tasks.

The opaque replication replaced identifiers with cryptographically opaque labels and mixed concept areas within batches. Aggregate performance did not move: it solved 374/480 test pairs in both conditions and 96/160 opaque tasks versus 95/160 semantic tasks. This supplies no directional evidence that removing those request-side cues changed performance.

Regarding attempt delivery, in the opaque ConceptARC replication, all 75 single-candidate records were already correct at rank one. On the full public ARC-AGI-1 evaluation, the MIN effort setting cost one third of STANDARD (0.00088399 versus 0.00265246 per task) and scored 111/400 rather than 118/400 pass@2, a difference of −1.75 percentage points. Repeated identical requests were byte-identical at both effort tiers.

Scaling reasoning effort increases pass@2: HIGH effort gives 29.5% pass@2, MEDIUM gives 27%, and LOW gives 21%, with cost reductions of 0%, 11%, and 22% respectively.

The paper also includes evaluation set analysis. Simple surface properties provide little explanation for success. Grid size is the strongest individual descriptor but remains weak, with reported pseudo-R2 = 0.072 on public tasks and 0.010 on generated tasks. Mechanic is a useful diagnostic stratifier: solve rates span 65.7 percentage points, from 68.6% for flood fill to 2.9% for gravity and stacking. Hand-written gravity tasks score 80–100%, compared with 2.9% for generated gravity-and-stacking tasks, and hand-written counting tasks score 85–95%, compared with 18.6% for generated object counting. Touching-panel overlays remain difficult under both authoring modes, scoring 7.5% when hand-written and 10.4% when generated.

Controlled structural manipulations show that conditional rule selection is solved on 68/120 tasks, or 56.7%, a decrease of 43.3 points from the control. When a marker count specifies a downward shift and the test value appears among the demonstrations, the model solves 12/40 tasks; when the value is absent, it solves 0/120. For panel union, two panels at opposite corners are solved on 26/40 tasks, whereas three panels are solved on 1/40. Two touching panels score 3/40, while opposite-corner panels score 26/40. Support chains decrease monotonically from 80.0% with one object resting on another to 67.5%, 52.5%, and 27.5% as the chain grows to three, five, and eight objects.

Failure structure analysis shows most failures preserve output dimensions and palette. On the calibrated generated set, 89.2% of failures have correct output dimensions, 72.9% have correct palette, and median cell error among shape-correct failures is 4.0%. On the public set, 89.7% have correct dimensions, 78.0% have correct palette, and median cell error is 8.3%.

The paper concludes that BDH-CQ demonstrates that in-context learning and recurrent latent reasoning can coexist in a compact, practical system. Demonstrations modify recurrent memory at inference time, and the resulting task is solved through iterative continuous computation rather than a verbalized chain of thought. On ARC-AGI-1, this design reaches 29.5% pass@2 for 0.00070 per task, establishing a new state of the art in benchmark cost efficiency. The controlled experiments further show that the system binds new mappings, extrapolates several learned operators, benefits predictably from demonstration coverage, and exhibits structured rather than uniform limits under composition.

Improvements to AI Systems:

1. Add a recurrent latent memory module that updates continuously with new inputs at inference time, enabling in-context learning without fine-tuning or parameter updates. This allows the system to adapt to novel tasks on the fly from a few demonstrations.

2. Replace verbalized chain-of-thought with iterative computation in a high-dimensional latent workspace. The system solves queries by repeatedly refining an internal state vector, avoiding token-by-token reasoning bottlenecks and reducing inference cost by over 10x compared to verbalized reasoning models.

3. Separate contextual memory from reasoning workspace. Use one recurrent state for accumulating evidence from demonstrations and another for ongoing query-solving computation. This prevents interference and improves consistency across multiple test inputs of the same task.

4. Implement a low-rank communication mechanism between memory and workspace to keep the model compact (150M parameters) while supporting high-dimensional positive activations, enabling efficient scaling of reasoning depth without exploding parameter count.

5. Add a matched support training strategy that explicitly provides demonstrations covering the range of relational depths or sequence lengths expected at test time. This recovers performance on extrapolation tasks (e.g., nesting depth 5 from 19/24 to 24/24, ordering from 0/24 to 13/24).

6. Introduce an effort-scaling control (LOW/MEDIUM/HIGH) that adjusts the number of iterative computation steps in the latent workspace. This allows trading 22% cost reduction for 8.5 percentage points of accuracy, enabling adaptive resource allocation per task difficulty.

7. Use a MIN effort mode for easy tasks that reduces inference cost by 66% with only a 1.75 percentage point accuracy drop, by detecting when the latent state has converged early and halting computation.

8. Implement a consistency-checking mechanism that detects when the system produces correct outputs for some test inputs but not all, flagging tasks for additional demonstrations or higher effort settings rather than accepting partial solutions.

9. Add a compositional operator library that learns atomic transformations (rotation, reflection, relocation, color swap) separately and composes them in the latent workspace. This improves composition success from 0-47/72 to potentially higher by training operators independently before combining.

10. Incorporate a mechanic-aware training curriculum that identifies difficult concept families (e.g., gravity, stacking, counting, touching panels) and generates additional targeted demonstrations for these, since hand-written examples achieve 80-100% vs 2.9-18.6% for generated ones.

What the Improved AI System Can Do:

• Solve novel visual reasoning tasks from 2-4 demonstrations without any fine-tuning, at a cost of 0.0007 per task (vs 0.04 for comparable verbalized models).

• Maintain 100% accuracy on extrapolating learned operators (propagation, copying) across unseen distances and target counts.

• Bind new task-specific mappings entirely from context (96/96 accuracy) even with 8 simultaneous bindings.

• Compose atomic operations (rotation + relocation) with 100% accuracy, and partially compose reflection + relocation (65%).

• Gracefully degrade on complex composition (color swap + relocation) rather than failing catastrophically, with predictable failure modes that preserve output structure.

• Scale reasoning effort dynamically: use HIGH effort for hard tasks (29.5% accuracy), MEDIUM for moderate (27%), LOW for easy (21%), or MIN for trivial tasks at one-third cost.

• Detect its own uncertainty by comparing outputs across multiple latent computation runs, flagging tasks where it is inconsistent for human review or additional demonstrations.

• Achieve state-of-the-art cost-efficiency on ARC-AGI-1, breaking the previous Pareto frontier by being 57x cheaper than GPT-5.6 Luna at comparable accuracy.