Macaron-V1: Towards Open Continual Learning with Self-Improvement and Mixture-of-LoRA

Macaron-V1 is an open agent-model family for experiential intelligence, defined as the ability to learn from experience accumulated in a real environment and to keep learning after deployment. It is organized around two system goals: Adaptation, pursued through recursive improvement of versioned model-harness pairs, and Collaboration, pursued via the Mixture-of-LoRA (MoL) architecture. The flagship Macaron-V1-Venti combines a 744B GLM-5.2 base with four LoRA specialists for chat, agent, coding, and GenUI; the Qwen3.6-based Macaron-V1-Tall (50B) uses the same design for local deployment.

The MoL architecture keeps a large base model frozen and layers specialist LoRA adapters on top, with adapter selection decided per user turn by the entry adapter L0's own reasoning under a constrained-decoding grammar. The routing loop has three stages: Route (L0 classifies the request into one of four canonical labels under a 24-token budget), Answer (the chosen specialist responds from its own conversation view), and Summary (the specialist emits a 192-token summary stored server-side). Routing accuracy is 99.12% on a 6,448-sample trace for Venti and 99.04% for Tall. The routing and summary hops cost 0.54s and 0.97s on Venti, about 32% of the three-hop total. Per-adapter conversation views are rebuilt deterministically from an append-only timeline, enabling emergent KV cache reuse. The deployment study shows MoL stores about 774.8B logical parameters versus 2.976T for a replicated-base layout, a 74.0% reduction in stored parameter values.

The algorithm combines Model-Harness Co-design and a recursive self-improvement loop. The harness includes UI4A, a component-native GenUI harness where the agent writes ordinary frontend code under runtime-enforced boundaries; a REPL agent harness with executable composition and validated reuse; and the versioned Harness Context Protocol (HCP) contract. MindForge, the agentic RL framework, runs a three-stage loop of task discovery, trajectory expansion, and linked model/configuration updates. In the expansion stage, adaptive search over HCP-carried harness changes reaches cumulative coverage of 122/122 selected base-failure tasks, while the stronger of two full-set single-configuration sweeps reaches 11/122. The model remains frozen throughout this experiment; no optimizer step is taken.

The supporting infrastructure includes MinT, a post-training platform managing adapter revisions over a resident base; LongStraw, a response-only long-context execution stack with reported multi-million-token operating points; and stability techniques for sparse MoE and DSA base models, including R3 rollout routing replay, DSA integration fixes, and IcePop-style token masking.

Evaluation covers Personal Intelligence (Macaron ChatBench, Macaron LivingBench), agent, coding, and GenUI tracks. Macaron-V1-Venti records 58.3 on ChatBench and 64.0 on LivingBench. On UI4A-Bench, it records a Final Score of 87.8, compared with 75.9 for Opus 4.8 and 72.1 for GPT-5.5. On the external suite, it leads TerminalBench 2.1 at 87.6 but trails on VitaBench (60.0 vs Qwen 3.7 Max at 61.2), ClawGym (77.7 vs GPT-5.5 at 82.5), SWE-Verified (85.6 vs Opus 4.8 at 88.6), DeepSWE (58.4 vs GPT-5.5 at 70.0), and SWE Atlas QnA (49.5 vs Opus 4.8 at 57.3). Macaron-V1-Tall is higher than its Qwen3.6 35B-A3B base on all seven reported benchmarks, with differences ranging from 1.3 points on LivingBench to 25.4 on UI4A-Bench.

The paper concludes that the current results validate the system and document execution checks, while compounding gains from continual learning and collective intelligence remain open empirical questions. Limitations include the lack of cross-generation lift measurement, narrow scenario richness, evaluation scope and provenance issues, data governance and safety scope gaps, and the fact that collective intelligence has not yet been demonstrated.

Improvements to AI Systems:

1. Implement a Mixture-of-LoRA (MoL) routing architecture with constrained-decoding grammar for per-turn adapter selection. The improved system uses a small entry adapter (L0) to classify each user request into one of four specialist categories (chat, agent, coding, GenUI) under a 24-token budget, then routes to a frozen large base model with the appropriate LoRA adapter. This achieves 99%+ routing accuracy and reduces stored parameters by 74% (774.8B vs 2.976T) compared to replicated-base systems, enabling deployment of a 744B-parameter model on a single GPU cluster without loading multiple full models.

2. Add a three-stage routing loop (Route → Answer → Summary) with deterministic per-adapter conversation views rebuilt from an append-only timeline. The improved system stores all interactions in a single timeline, rebuilds each specialist's context deterministically, and emits a 192-token summary after each turn. This enables emergent KV cache reuse across turns, cutting routing and summary overhead to 0.54s and 0.97s respectively (32% of total latency), allowing real-time multi-specialist collaboration without redundant context recomputation.

3. Integrate a recursive self-improvement loop via Model-Harness Co-design with versioned Harness Context Protocol (HCP). The improved system runs a three-stage agentic RL loop (task discovery, trajectory expansion, linked model/config updates) where the harness itself is versioned and mutable. During expansion, adaptive search over HCP-carried harness changes reaches 122/122 base-failure tasks (vs 11/122 for single-configuration sweeps), while the model remains frozen—proving that capability gains can come from harness evolution alone, not just weight updates.

4. Deploy component-native GenUI harness (UI4A) where the agent writes ordinary frontend code under runtime-enforced boundaries. The improved system lets the agent generate React/Vue components directly, with sandboxed execution and validation. On UI4A-Bench, this achieves a Final Score of 87.8 vs 75.9 (Opus 4.8) and 72.1 (GPT-5.5), enabling autonomous UI generation that is 12-15 points more reliable than current frontier models for interactive web app creation.

5. Use a response-only long-context execution stack (LongStraw) for multi-million-token operating points. The improved system processes extremely long contexts (millions of tokens) without full-attention recomputation, enabling tasks like whole-repository code analysis, long-horizon agent trajectories, and multi-session memory persistence—capabilities that current systems fail on due to context window limits.

6. Apply stability techniques for sparse MoE and DSA base models: R3 rollout routing replay, DSA integration fixes, and IcePop-style token masking. The improved system trains and deploys sparse expert models without routing collapse or instability, allowing efficient scaling (e.g., 744B base with only active expert computation) while maintaining training stability—critical for continual learning after deployment.

What the Improved AI System Can Do:

• Run a 744B-parameter model locally or on modest hardware (via MoL) with specialist-level performance in chat, coding, agentic tasks, and UI generation simultaneously, without loading multiple full models.

• Continuously improve after deployment by evolving its harness (not just weights) through HCP, fixing its own failure cases (122/122) without any gradient updates—enabling self-repair in production.

• Generate production-quality frontend code (87.8 UI4A score) that executes safely under runtime boundaries, usable for autonomous web app prototyping, dashboard generation, and interactive tool building.

• Maintain coherent multi-turn conversations across different specialists (chat, coding, agent) with per-adapter memory and sub-second routing, so a user can switch from a coding question to a UI request seamlessly.

• Process and reason over millions of tokens (LongStraw) for tasks like analyzing entire codebases, long-form document synthesis, or multi-day agent missions with persistent memory.

• Achieve higher benchmark scores than its base model on all tasks (e.g., +25.4 on UI4A-Bench for the 50B Tall variant) without retraining the base—demonstrating that adapter-based specialization is a drop-in upgrade for existing open-weight models.

Macaron-V1 is an open agent-model family for experiential intelligence: learning from experience in real environments and continuing to learn after deployment. It is organized around two system goals. Adaptation is pursued through recursive improvement of versioned model-harness pairs, where experience from one configuration is evaluated under an external contract and used to construct its successor. Collaboration is pursued via the Mixture-of-LoRA (MoL) architecture that freezes a base model, composes specialist LoRA adapters, and selects one LoRA per user turn. The flagship Macaron-V1-Venti combines a 744B GLM-5.2 base with four LoRAs for chat, agent, coding, and GenUI; the Qwen3.6-based Macaron-V1-Tall (50B) uses the same design for local deployment. This report presents Macaron-V1 as a co-designed system spanning architecture, algorithms, and infrastructure. The MoL architecture supports continual learning through extensible LoRA specialists. The algorithm combines Model-Harness Co-design and recursive self-improvement loop, including the UI4A component-native GenUI harness, a stateful action substrate, versioned HCP contract, and the agentic RL framework MindForge. The supporting infrastructure includes the post-training platform MinT, the long-context RL method LongStraw, and stability techniques for sparse MoE and DSA base models. We evaluate Macaron-V1 on Personal Intelligence, GenUI, and general capability benchmarks against frontier baselines. Our results validate the current system, while compounding gains from continual learning and collective intelligence remain open questions.