PRAXIS: Learning Dynamics of Self-Improving Models with Symbolic Archives

arXiv:2610.11803 · cs.LG · Submitted 2026-10-08 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "PRAXIS: Learning Dynamics of Self-Improving Models with Symbolic Archives".

Jane: The gist: PRAXIS introduces a co-evolutionary framework that models self-improving learning as an interacting dynamical process between generators, learners,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So Jane, we’re looking at PRAXIS today, which is this whole framework for learning systems that can actually improve themselves by changing what data they see and how they optimize things. It seems to tackle a problem where the environment keeps changing outside of just standard learning assumptions.

Jane: That’s right, Tom. The core idea here is modeling the generator, the learner, and symbolic archives as these interacting dynamical processes that evolve together in a closed loop. It’s about handling those nonstationary objectives where things aren't staying constant for the learner to settle on a stable point.

Lu: What I find fascinating is how they set up this coupled state with theta sub t, st, and phi sub t, where theta sub t are the Learner parameters, st are Archive scores, and phi sub t parameterizes a Generator distribution qt <ref:2610.11803#pg6>. It shows how everything is tied together from the start.

Meng: From an engineering standpoint, I want to know how this coupling works in practice. If you’re running this system, what does that mean for the computational overhead? Does it add a lot of complexity or is it manageable?

Lalam: I think what's really compelling is how the symbolic archive isn't just sitting there; it actively influences learning through those time-varying weights wj appearing in Psi <ref:2610.11803#pg5>. It makes the learner pay attention to what the archive thinks is important.

Tom: Exactly, Lalam. And this leads us right into how they manage objective drift, which is a big one for self-improving systems. The paper proves that KL-constrained generator updates and controlled archive-weight movement bound one step objective drift <ref:2610.11803#pg8>.

Jane: That’s crucial because it means they aren't just hoping things stay stable; they are actively controlling the movement of the objectives, preventing a catastrophic shift in what the learner is trying to achieve.

Lu: They also get a proof that archive updates suppress a program relative to any fixed comparator with a persistent cumulative utility advantage under sub-Gaussian noise <ref:2610.11803#pg8>. That suggests we can actually prune less useful symbolic programs over time, even with some random noise in the system.

Meng: So, if we look at the numbers, how does this relate to the learner’s performance? Does it guarantee anything about average performance when the objective is drifting?

Tom: They establish an averagestationarity guarantee whose degradation is governed by cumulative objective drift <ref:2610.11803#pg8>. It means even if the underlying objective changes, the learner's performance stays bounded under standard stochastic gradient descent assumptions.

Title and authors: Jane: That’s a strong result because it connects the drift in the overall objective to how much worse the learner gets on average over time. It’s about managing expectations when you know things are shifting.

Lu: They also have this lemma showing that along-iterate drift is dominated by uniform drift, meaning ∆t is bounded by Edelta t <ref:2610.11803#pg8>. This simplifies the analysis of how the system evolves step by step.

Meng: If we look at the experiments they did, what kind of stability did they actually see when applying this PRAXIS framework across different tasks?

Tom: They saw generator stabilization, decreasing learner loss, and archive concentration across visual robustness, relational graph reasoning, and algorithmic graph reasoning <ref:2610.11803#pg8>. The results align with what their theory predicted about the coupling dynamics.

Jane: It sounds like this framework is robust across different kinds of data problems because it’s modeling the interaction between selection, optimization, and structure simultaneously.

Lu: And we have a bit on component ablation analysis too. They found that full PRAXIS gets the strongest mean and CVaR10 on MIT Indoor, but GraphLog behaved differently where adaptive universe selection without archive regularization was strongest <ref:2610.11803#pg15>.

Meng: That’s an interesting detail for practical application. It suggests that the best setup depends heavily on the specific data and what you value in the archive structure.

Tom: Right, and it supports their idea that you can't just use one component in isolation; these coupled dynamics are what matter, which is why they call it a co-evolutionary framework <ref:2610.11803#pg5>.

Jane: So, to wrap up this summary of PRAXIS: we have a system where the learner adapts data selection and optimization alongside symbolic components, leading to controlled drift bounds and archive suppression based on utility gaps.

Lu: The future work they point toward is interesting because they mention extending PRAXIS to continuous or latent universe spaces, richer compositional archives, and a unified closed-loop theory of long-horizon co-evolution <ref:2610.11803#pg10>. That opens up a lot of possibilities for more complex learning environments.

Meng: From an engineering side, that sounds like the next big hurdle—handling those continuous spaces or making the archives richer than simple deterministic rules. It’s ambitious stuff, but I see the direction for future work.

Tom: Well, if we boil it down for listeners: PRAXIS is a framework that models how a self-improving model makes decisions about what data to use and how to optimize its parameters while using symbolic tools in a way that keeps the objectives under control.

Title and authors: Jane: It shows us how to build systems that don't just learn sequentially, but evolve in a coupled loop where selection, optimization, and structure all influence each other dynamically.

Lu: The paper PRAXIS: Learning Dynamics of Self-Improving Models with Symbolic Archives really pushes the idea that learning is more about managing these interacting dynamics than finding one perfect fixed solution <ref:2610.11803#pg8>.

Meng: I think it gives us a better way to think about designing these complex systems because we can look at the entire coupled process instead of just optimizing the learner parameters in a vacuum.

Lalam: For culture, this means that if we can model and manage these evolutionary dynamics, we might be able to build AI agents that are inherently more robust to shifting data or changing goals without needing constant manual intervention.

Tom: That’s a big implication for the future of AI design, Jane. So PRAXIS gives us a formal way to analyze this coupled evolution under nonstationary conditions.

Jane: It gives us tools to understand the drift and stabilization mechanisms that are happening when these systems try to improve themselves over long periods.

Lu: We’re leaving with the idea that objective drift isn't just something that happens; it’s something we can model and constrain through specific generator updates <ref:2610.11803#pg8>.

Meng: I think the main practical value is in understanding where to put the effort—whether it’s in refining the data selection mechanism or tuning the symbolic constraints.

Lalam: And from a model perspective, this framework suggests that incorporating an archive means giving the learner a persistent memory of useful patterns that helps it navigate those nonstationary environments better.

Tom: So PRAXIS is essentially providing the mathematical language to describe how self-improvement happens when there are three interacting parts—the generator, the learner, and the symbolic archive.

Jane: Exactly. It’s about seeing the whole system as a dynamical process where every part feeds back into every other part constantly.

Lu: We hope this work on PRAXIS inspires more research into these coupled systems in more complex settings than what they tested on for now <ref:2610.11803#pg10>.

Meng: I'm curious to see if we can apply these dynamics modeling ideas to other areas, maybe even resource management, where the objectives are constantly shifting as well.

Lalam: And hopefully, this mathematical structure helps us build AI systems that are more stable and focused over much longer training horizons than we currently can achieve.

Tom: That’s all for this look at PRAXIS. We’ll be back with more papers soon.

The paper's summary: Tom: So, PRAXIS is this framework that models how a self-improving AI system makes decisions about what data to use and how to optimize its parameters while using symbolic tools in a way that keeps the objectives under control.

Jane: It’s like giving an AI a whole feedback loop where it’s not just learning from data, but also changing what data it focuses on and updating its internal rules based on what it learns.

Tom: Exactly. The authors are proposing this co-evolutionary framework where the generator, the learner, and symbolic archives all interact as these coupled dynamical processes.

Jane: They set up a system with three things moving at once: the Learner parameters theta sub t, the Archive scores st, and something that controls what data it selects called phi sub t.

Tom: And this is important because they are trying to solve those nonstationary objectives where the environment keeps changing over time without any external help.

Jane: The core idea is using KL-constrained generator updates to keep the one-step objective drift bounded, which means the AI’s data selection doesn't just wildly jump around every single step.

Tom: That sounds like they’re putting a brake on the AI’s learning process, ensuring that when it tries to improve itself, it doesn't cause a catastrophic shift in its goal.

Jane: Plus, they show how the symbolic archive scores update based on utility—basically how useful a particular piece of knowledge is—and this influences the learner’s weights through these archive weights wj.

Tom: So, if the AI finds something less useful than what’s already in its archive, that program gets suppressed over time because its weight decays. That’s an interesting way to manage memory and focus.

Jane: And they prove that this whole system leads to an averagestationarity guarantee for the learner’s performance, meaning even if the objective is drifting, the average performance stays bounded under standard stochastic gradient descent assumptions.

Tom: That's a strong result because it connects how much things are shifting with how much worse the learner gets on average over time. It’s about managing expectations when you know things are changing.

Jane: The experiments they ran across different tasks—visual robustness, graph reasoning—show generator stabilization and archive concentration that matches what their theory predicted about the coupling dynamics.

Tom: It seems like PRAXIS is providing a formal way to analyze this coupled evolution of data selection, optimization, and symbolic structure under endogenous nonstationarity.

Jane: And they point toward future work where they want to extend this to continuous or latent universe spaces, which is a big step for making these systems more flexible.

Tom: Right. So PRAXIS isn't just about one neat trick; it’s about modeling the whole system as a coupled loop where every part influences every other part constantly.

Jane: It shows how to build AI systems that don't just learn sequentially, but evolve in a coupled way where selection, optimization, and structure all influence each other dynamically.

The paper's improvements: Tom: So, we’re looking at how they suggest improving PRAXIS itself, moving beyond just describing the dynamics to actually making them better systems.

Jane: The authors point out that this framework is quite controlled right now, and they want to see it applied in more complex settings.

Tom: They explicitly mention extending PRAXIS to continuous or latent universe spaces, which means moving away from the finite sets of universes they used for their tests.

Jane: That opens up possibilities for modeling things where the data selection isn't just choosing between a fixed list but something much richer and more continuous.

Tom: They also discuss making the symbolic archives richer, suggesting moving beyond simple deterministic rules to more complex structures that can hold more nuanced knowledge.

Jane: And they want a unified closed-loop theory for long-horizon co-evolution, which means seeing how all these parts evolve together over much longer training periods.

Tom: That’s a big jump from the controlled setup they used for their main results, and it moves PRAXIS toward modeling really complex, real-world AI improvement processes.

Jane: They are essentially looking at how to make this self-improving loop robust enough to handle situations that aren't perfectly contained in the current framework.

Tom: It’s about taking these ideas—the coupled dynamics and the drift control—and figuring out how to scale them up for more sophisticated tasks.

Jane: This suggests we need a deeper understanding of how selection, optimization, and structure interact when there are no hard boundaries on what the data or rules can look like.

Tom: So they aren't just stopping at proving stability in a fixed setup; they’re looking at the next level of complexity for self-improvement.

Jane: It’s about building a learning process that can adapt its selection and its tools in ways that are more flexible and less constrained by simple finite sets.

Tom: This has implications for how we design AI agents that need to operate in environments where the data or the rules themselves are constantly evolving, not just static.

Conclusion: Tom: So, to wrap up, PRAXIS is this framework that models how a self-improving AI system makes decisions about what data to use and how to optimize its parameters while using symbolic tools in a way that keeps the objectives under control.

Jane: It shows us how those three parts—data selection, optimization, and structure—are coupled in a closed loop where they constantly influence each other dynamically.

Tom: The main implication is that we can formally analyze objective drift and stability even when the environment isn't stationary at all.

Jane: They’ve shown that by controlling the generator updates with KL constraints, you can bound how much the objective shifts in one step, which helps keep things from going completely haywire.

Tom: Plus, they give us tools to manage what we know through symbolic archives, showing how those archives can suppress less useful programs based on persistent utility gaps.

Jane: It means that over a long training run, the AI won't just wander around aimlessly; it will naturally focus its effort on what the archive deems most valuable.

Tom: The numbers they show across visual robustness and graph reasoning confirm that this coupled evolution leads to better stability and lower loss for the learner.

Jane: For someone listening, this means we can design systems that don't just learn sequentially, but evolve in a coordinated way where selection, optimization, and structure all feed into each other.

Tom: Lu, what’s your take on the bigger picture possibilities here?

Lu: I think the leap to continuous spaces or latent universe modeling is where the real creative potential lies; it could unlock models that handle much more nuanced, complex environments than we’re currently testing.

Meng: From an engineering standpoint, I see this as a way to build systems that are inherently more resilient because they aren't brittle; they have these built-in mechanisms for managing drift and focusing attention.

Lalam: For culture and how we build things, PRAXIS suggests we can create AI agents that are much better at long-term planning and adaptation because their internal structure is designed to evolve intelligently.

Tom: It’s a lot to take in, Jane; it’s a framework for managing complexity rather than just finding one single perfect algorithm.

Jane: Exactly, Tom; it’s about understanding the dynamics of self-improvement rather than just chasing the final answer.

Tom: So, we have this full picture now of PRAXIS: Learning Dynamics of Self-Improving Models with Symbolic Archives.

Jane: It’s a solid piece for anyone interested in how AI systems manage their own evolution under pressure.

Tom: Next up, we’re looking at something totally different—a paper on how we can predict model degradation based on the source-side training dynamics of the model itself.

Venkat Margapuri, Mustafa Teber

Department of Computing Sciences · Villanova University

cs.LG

Submitted: 2026-10-08

Updated: 2026-10-08

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 92/100

The gist: The gist: PRAXIS introduces a co-evolutionary framework that models self-improving learning as an interacting dynamical process between generators, learners, and symbolic archives to formally analyze

Key concepts

Generator: The Generator is responsible for updating the distribution of possible universes or data selections. It minimizes a KL-regularized objective function that includes constraints to prevent large, destabilizing shifts in its distribution, ensuring the learning process remains controlled and stable.
Learner: The Learner adapts its parameters based on gradient information derived from minibatches drawn from the current data distribution. Its updates are designed to remain stable even when the underlying objective function is nonstationary, tracking performance effectively under changing conditions.
Symbolic Archive: This component stores and manages symbolic programs, treating them as fixed mappings rather than continuously optimized parameters. The archive's scores are updated based on utility, allowing it to selectively suppress less useful programs through weight decay, leading to concentration on elite symbolic solutions.

Summary

The gist: PRAXIS introduces a co-evolutionary framework that models self-improving learning as an interacting dynamical process between generators, learners, and symbolic archives to formally analyze objective drift and stability in nonstationary objectives.

How it works

PRAXIS models the learning system as a closed-loop dynamical process coupling adaptive universe selection, gradient-based optimization, and symbolic archive regularization <ref:2610.11803#pg5>. The framework explicitly represents the interaction between the Generator, Learner, and symbolic Archive <ref:2610.11803#pg5>. This enables formal analysis of KL-controlled objective drift, learner tracking under nonstationarity, and selective suppression of non-competitive symbolic programs <ref:2610.11803#pg5>.

The system maintains a coupled state (θt, st, ϕt), where θt are Learner parameters, st are Archive scores, and ϕt parameterizes a Generator distribution qt <ref:2610.11803#pg6>. Updates are performed online using stochastic first-order information computed under the current state of the others <ref:2610.11803#pg6>.

The Learner update is implemented via a stochastic descent step θt+1 = θt − αt gt, where gt is an unbiased estimator of ∇Ft(θt) computed from minibatches drawn from Dt <ref:2610.11803#pg8>. Symbolic programs are treated as fixed, non-differentiable mappings, and the learner receives gradient only from the clipped discrepancy in Eq. (2) <ref:2610.11803#pg5>.

The Archive update involves updating archive scores according to sj,t+1 = sj,t + ηp Ubj,t, where Ubj,t = Uj,t + ξj,t captures stochastic estimation noise due to finite sampling <ref:2610.11803#pg8>. These scores are mapped to archive weights via a Gibbs distribution wj,t = P exp(βsj,t) / P k=1 exp(βsk,t) <ref:2610.11803#pg5>.

The Generator update minimizes the KL-regularized objective Jt(ϕ), which includes a trust-region constraint limiting step-to-step distributional movement and a term preventing collapse onto degenerate subsets of U <ref:2610.11803#pg8>. These updates implement a stochastic primal–dual tracking procedure for the regularized min–max objective in Eq. (5) <ref:2610.11803#pg8>.

Theoretical Analysis

The analysis focuses on structural guarantees rather than convergence to a stationary optimum <ref:2610.11803#pg6>. Theorem 5.4 proves that KL-regularized Generator updates control nonstationarity of the Learner’s population objective, bounding the one-step pathwise uniform objective drift δt ≤ M∥qt+1 − qt∥1 + at <ref:2610.11803#pg8>.

Lemma 5.6 establishes that along-iterate drift is dominated by uniform drift, stating that ∆t ≤ E[δt] <ref:2610.11803#pg8>. Corollary 5.8 shows that exponential Generator stabilization and controlled archive loss drift imply exponential bounds on the objective drift δt <ref:2610.11803#pg8>.

Theorem 5.12 proves high-probability fixed-comparator archive suppression, showing that a symbolic program persistently less useful than one fixed comparator experiences decay in relative weight, up to stochastic fluctuations induced by finite sampling <ref:2610.11803#pg8>. Corollary 5.13 strengthens this by identifying a sufficient condition under which archive suppression becomes exponential when each non-elite program has a persistent signed utility gap relative to a fixed comparator <ref:2610.11803#pg8>.

Experimental Findings

Experiments across visual robustness, relational graph reasoning, and algorithmic graph reasoning exhibit generator stabilization, decreasing learner loss, and archive concentration consistent with these theoretical mechanisms <ref:2610.11803#pg8>.

RQ1 shows that the Generator adapts while remaining stable: The normalized curves show the same qualitative pattern: entropy decreases, maximum universe probability increases, and the KL drift proxy remains bounded rather than diverging <ref:2610.11803#pg10>.

RQ2 demonstrates that learner optimization remains stable under induced nonstationarity: In all three domains, the rolling training cross-entropy decreases rather than destabilizing under adaptive sampling <ref:2610.11803#pg10>.

RQ3 confirms that the symbolic archive concentrates and suppresses non-elite programs: Archive entropy decreases in all three domains, indicating that the archive does not remain a diffuse pool of interchangeable programs <ref:2610.11803#pg10>. Empirical concentration is demonstrated by the decay of ratios for non-elite programs relative to the final elite set <ref:2610.11803#pg10>.

Component Ablation Analysis

The component ablation analysis reveals different roles across domains, indicating that full PRAXIS obtains the strongest mean and CVaR10 on MIT Indoor, while GraphLog behaves differently where adaptive universe selection without archive regularization is strongest <ref:2610.11803#pg15>. This supports the modular interpretation of PRAXIS: the framework exposes the coupled dynamics, but the empirical value of each component depends on the domain and archive quality <ref:2610.11803#pg15>.

The results support that adaptive universe selection without archive regularization is strongest in GraphLog, whereas frozen archive and frozen generator variants both underperform full PRAXIS in that domain <ref:2610.11803#pg8>. The paper concludes by noting that "PRAXIS is complementary to this line of work: rather than proposing a new agent architecture or self-improvement mechanism, it provides a learning-theoretic framework for modeling and analyzing the coupled evolution of data selection, optimization, and symbolic structure" <ref:2610.11803#pg13>.

The present formulation is intentionally controlled: universes are finite and archive programs are simple deterministic rules <ref:2610.11803#pg8>. Extending PRAXIS to continuous or latent universe spaces, richer compositional archives, and a unified closed-loop theory of long-horizon co-evolution are natural directions for future work <ref:2610.11803#pg10>.

The paper concludes by stating that The present formulation is intentionally controlled <ref:2610.11803#pg8>. The framework successfully models the coupled evolution of data selection, optimization, and symbolic structure under endogenous non-stationarity <ref:2610.11803#pg8>. The analysis separates Generator- and archive-induced objective drift, establishes average SGD stationarity under cumulative drift, and proves fixed-comparator suppression under persistent signed utility advantages <ref:2610.11803#pg8>. Experiments across visual robustness, relational graph reasoning, and algorithmic graph reasoning exhibit stable optimization and archive concentration <ref:2610.11803#pg8>. The present formulation is intentionally controlled <ref:2610.11803#pg8>. The framework successfully models the coupled evolution of data selection, optimization, and symbolic structure under endogenous non-stationarity <ref:2610.11803#pg8>. The analysis separates Generator- and archive-induced objective drift, establishes average SGD stationarity under cumulative drift, and proves fixed-comparator suppression under persistent signed utility advantages <ref:2610.11803#pg8>. Experiments across visual robustness, relational graph reasoning, and algorithmic graph reasoning exhibit stable optimization and archive concentration <ref:2610.11803#pg8>. The present formulation is intentionally controlled <ref:2610.11803#pg8>. The framework successfully models the coupled evolution of data selection, optimization, and symbolic structure under endogenous non-stationarity <ref:2610.11803#pg8>. The analysis separates Generator- and archive-induced objective drift, establishes average SGD stationarity under cumulative drift, and proves fixed-comparator suppression under persistent signed utility advantages <ref:2610.11803#pg8>. Experiments across visual robustness, relational graph reasoning, and algorithmic graph reasoning exhibit stable optimization and archive concentration <ref:2610.11803#pg8>. The present formulation is intentionally controlled <ref:2610.11803#pg8>. The framework successfully models the coupled evolution of data selection, optimization, and symbolic structure under endogenous non-stationarity <ref:2610.11803#pg8>. The analysis separates Generator- and archive-induced objective drift, establishes average SGD stationarity under cumulative drift, and proves fixed-comparator suppression under persistent signed utility advantages <ref:2610.11803#pg8>.

Improvements for AI systems

Bold Header: Endogenous Nonstationarity Modeling

The system can adapt to nonstationary objectives outside standard learning assumptions by explicitly modeling interactions between a Generator, Learner, and symbolic Archive as interacting dynamical processes. This allows the AI to handle environments where training data distributions or reward models change over time without requiring external specification.

Bold Header: KL-Constrained Objective Drift Control

The framework utilizes KL-constrained generator updates to bound one-step objective drift, ensuring that the adaptation of data selection remains controlled. This prevents catastrophic objective shifts during self-improvement by enforcing constraints on the distribution change between iterations.

Bold Header: Archive Suppression Under Utility Gaps

The symbolic archive can suppress non-elite programs based on persistent cumulative utility advantage, as shown in Theorem 5.12, which states that a symbolic program that is persistently less useful than one fixed comparator experiences decay in relative weight. This leads to a more focused and stable set of inductive biases over long training horizons.

Bold Header: Average Learner Stationarity Guarantee

The system provides an averagestationarity guarantee whose degradation is governed by cumulative objective drift, as formalized in Theorem 5.14. This ensures that even when the underlying objective changes, the performance of the learner remains bounded under standard stochastic gradient descent assumptions.

Bold Header: Coupled Closed-Loop Dynamics

The architecture implements a coupled closed-loop learning process in which data selection, inductive structure, and model parameters evolve jointly. This unified dynamics model allows for a holistic approach to self-improvement rather than treating components sequentially or independently.

Abstract

Self-improving learning systems adapt data selection, optimization, and auxiliary symbolic components, inducing nonstationary objectives outside standard learning assumptions. We introduce PRAXIS, a co-evolutionary framework that models generators, learners, and symbolic archives as interacting dynamical processes. We prove that KL-constrained generator updates and controlled archive-weight movement bound one-step objective drift, that archive updates suppress a program relative to any fixed comparator with a persistent cumulative utility advantage under sub-Gaussian noise, and that stochastic gradient descent achieves an average-stationarity guarantee whose degradation is governed by cumulative objective drift. Experiments across visual robustness, relational graph reasoning, and algorithmic graph reasoning exhibit generator stabilization, decreasing learner loss, and archive concentration consistent with these theoretical mechanisms.