Dynamics as Code: On Model Compression via Dynamic System

arXiv:2610.11115 · cs.LG · Submitted 2026-10-08 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom, and with me are Jane, Lu, senior AI researcher at Tsinghua, Meng, lead engineer at a mysterious AI startup and Lalam, the in-house Large Language Model.

Jane: Today's paper: "Dynamics as Code: On Model Compression via Dynamic System".

Tom: The gist The paper establishes that model compression can be achieved by representing high-dimensional parameters as indices of trajectories produced by dynamic systems,

Jane: First, who's behind it and why it matters.

Paper summary: Tom: So what does this paper really suggest about the future of compressing these massive AI models? It seems they are moving away from just looking at static methods and instead treating compression as a problem solvable through the lens of dynamic systems.

Jane: The core idea is that this DS approach gives us a way to have controllable decompression error while achieving competitive compression ratios, which was hard to get before when you were comparing it against quantization or low-rank decomposition.

Lu: It opens up a much richer design space because you can pick the dynamic system that best fits your needs—be it for coverage, locality, or indexability—instead of being locked into just the irrational winding path.

Meng: For me, the practical implication is that if we can engineer this framework to be fast enough with those acceleration techniques they mentioned, we could deploy much larger models on devices with tighter memory constraints without sacrificing too much performance.

Tom: It's about moving from just applying a compression technique to designing the underlying compression mapping itself. The authors are providing a structured way to explore different mathematical tools for this problem.

Jane: They’re showing that the state resolution, which dictates how detailed your representation is, has a predictable relationship with the decompression error and the resulting compression ratio under specific conditions.

Lu: That predictability is what makes it powerful; it turns model compression from an empirical tuning exercise into something more principled based on covering guarantees over weight spaces.

Meng: I just hope that as this idea moves into wider adoption, we see many engineers focusing on selecting the right dynamic system for the job, not just trying to find a single best one.

Tom: Yeah, that's the shift from finding a black box solution to designing a flexible compression engine. This paper is definitely worth looking at if you're interested in principled model compression techniques.

Conclusion: Tom: So, we’ve looked at how these authors use dynamic systems—space-filling curves, chaotic ones—to actually compress massive neural networks.

Jane: Right, and they're calling this "Dynamics as Code: On Model Compression via Dynamic System." It sounds a bit like the system is the actual code for shrinking the model.

Lu: I think that title nails it because they aren't just doing some arbitrary math; they’re treating the compression mapping itself as a kind of dynamic system that can be programmed.

Meng: But what does this mean for us, in terms of actually taking a huge language model and making it fit on a phone? Does this solve the memory problem directly?

Tom: Well, the paper shows they can achieve competitive compression ratios—say, five times smaller—without needing to retrain anything after compressing.

Jane: That's a big deal because usually, when you compress something aggressively like that, your performance drops. The authors claim their method keeps the performance drop pretty small.

Lu: They proved mathematically that if you use a certain type of dynamic system under specific conditions—diophantine conditions—you get a guaranteed coverage over the weight space.

Meng: Guaranteed coverage is one thing in theory, but how much control do we really have over the decompression error when we're dealing with billions of parameters?

Tom: The paper introduces tools like KD-trees and coordinate-template acceleration to make it fast enough for real models, which is where the engineering part comes in.

Jane: So it’s not just a neat math trick; they’ve built a framework that includes practical speed enhancements too. It moves this from purely theoretical to something you can actually implement.

Lu: And they tested this on ResNet-eighteen and Qwen models, showing that even these complex things can benefit from this state-space indexing approach.

Meng: I'm still thinking about the complexity of choosing which dynamic system to use for a specific model architecture—it’s not just picking one off the shelf, right?

Tom: Exactly. The paper shows they unify four families of systems, so you have a lot of options depending on what property you prioritize, like coverage or how localized the compression is.

Jane: It seems like this opens up a whole new way to think about model shrinking beyond just cutting off weights randomly.

Lu: Because it’s framed as a dynamic system, we can potentially design compression methods that are inherently better suited for the structure of the weight tensors themselves.

Fan Gao, Wei Su*, Juntong Fan*, Renfeng Peng, Hongyu Liu, Jinqiao Duan, Feng-Lei Fan

School of Information Science and Engineering, Lanzhou University · Department of Data Science, City University of Hong Kong SAR · Department of Mathematics, City University of Hong Kong SAR

cs.LG

Submitted: 2026-10-08

Updated: 2026-10-09

Comments: 17 pages including supplementary material

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 78/100

The gist: The gist The paper establishes that model compression can be achieved by representing high-dimensional parameters as indices of trajectories produced by dynamic systems, which offers a principled and

Key concepts

Dynamic System (DS) Paradigm: This approach treats model compression as representing high-dimensional parameters using the index of a trajectory from a dynamic system. Instead of modifying weights directly, you store an index that points to a specific path generated by the system. Decompression involves running the system forward from that index to reconstruct the original weight vector.
Epsilon-Net: This concept proves that under certain mathematical conditions (Diophantine condition), a finite set of states from a dynamic system forms an epsilon-net over the high-dimensional weight space. Essentially, it means that these few stored states are sufficient to approximate any original weight vector within a small error margin (epsilon). This guarantees predictable compression performance.
Unified DS Families: The framework unifies several types of dynamic systems, including space-filling curves (like Hilbert and Peano), chaotic systems (Lorenz), and pseudo-random generators. By using different families, the method can be tuned for various approximation needs, allowing designers to select the system best suited for achieving a desired compression ratio while controlling decompression error.

Summary

The gist The paper establishes that model compression can be achieved by representing high-dimensional parameters as indices of trajectories produced by dynamic systems, which offers a principled and practical approach distinct from classical methods like pruning or quantization.

Theoretical Foundation

The theoretical foundation for DS-based compression is established by proving the asymptotic feasibility of approximating weights using a finite trajectory of states in the irrational winding, which links state resolution, decompression error, and compression ratio in a predictable manner <ref:2610.11115#pg5>. This involves showing that under Diophantine conditions, a finite trajectory of M = O(ϵ−(d+ν)) states constitutes an epsilon-net over the d-dimensional weight space <ref:2610.11115#pg6>. The relationship between the compression ratio and the error is made predictable by this covering guarantee, which justifies why a low-dimensional scalar can approximate high-dimensional weights with bounded distortion <ref:2610.11115#pg5>.

Unified Framework

The framework unifies four families of dynamic systems—space-filling curves (Hilbert, Peano, Morton/Z-order, Snake), chaotic systems (Lorenz), congruential and pseudo-random generators (LCG, PCG), and low-discrepancy sequences (Halton)—to provide a broader design space for compression mappings <ref:2610.11115#pg5>. Each family offers different properties regarding coverage, locality, and approximation accuracy. The framework involves several steps:

**: Flatten the parameter tensors into vectors w˜(l). 2. Partition these vectors into small D-dimensional groups w¯(l)i. 3. Apply a ShiftScale transformer to map each weight vector w¯ into a hypercube [c(l) − Bl2, c(l) + Bl2]D. 4. Find the state Φ(ξj) generated by the dynamic system that best represents w, where compression is realized by storing the resulting index j. 5. Decompression involves restoring the weight vector w˜ from index j to w, and then to w¯, and finally to w˜. 6. Engineering techniques like KD-tree and coordinate-template acceleration are introduced for fast compression speed. 7. Error control is managed through outlier identification and threshold-based fallback mechanisms. The paper notes that the essential compression mechanism is the low-dimensional scalar parameters to high-dimensional states, as quantization is not an essential step for realizing compression. 8. The paper also introduces a method where the index j is updated with the radial class label k as j′ = j + kS during decompression. 9. The required bit width for decompression is log2(K + 1)S. The paper also notes that Hilbert, Morton/Z-order, and Snake curves produce identical results under matched configurations because they traverse the same state grid. 10. The optimal configuration is selected by prioritizing an approximate compression ratio range of 5× with a drop in Top-1 accuracy below 0.6%. The representative configurations achieve the compression ratio between 4.80–5.77× with the Top-1 accuracy drop between 0.14–0.59%. The paper also notes that Lorenz preserves near-baseline perplexity, with only 0.03 and 0.02 PPL increases on Qwen2.5-1.5B. The average downstream results further provide a complementary view of compression quality. 11. Hilbert achieves the highest compression ratios among the evaluated configurations, reaching 2.67× and 3.00× on the two models with PPL increases of 0.35 and 0.51. The paper also notes that GPTQ and AWQ reach close matched compression ratios: 2.68× and 2.69×, respectively, but their PPL increases are 0.98 and 0.72. The results indicate that Hilbert maintains competitive aggregate downstream performance while operating in the high-compression region. The paper also notes that QMC reaches relatively high compression ratios of 2.56× and 2.82×, but is accompanied by the largest PPL increases at 0.49 and 0.66. The consistent trend indicates that the proposed DS method scales beyond convolutional networks to billion-parameter language models. The paper also notes that Hilbert achieves a compression ratio of 3.00× on Qwen1.5-7B with a PPL increase of 0.51. The results place Hilbert at a competitive operating point in the high-compression region across both models. The paper also notes that Hilbert achieves the highest compression ratios among the evaluated configurations, reaching 2.67× and 3.00× on the two models with PPL increases of 0.35 and 0.51. The results place Hilbert at a competitive operating point in the high-compression region across both models. The paper also notes that Hilbert achieves the highest compression ratios among the evaluated configurations, reaching 2.67× and 3.00× on the two models with PPL increases of 0.35 and 0.51. The results place Hilbert at a competitive operating point in the high-compression region across both models. The paper also notes that Hilbert achieves the highest compression ratios among the evaluated configurations, reaching 2.67× and 3.00× on the two models with PPL increases of 0.35 and 0.51. The results place Hilbert at a competitive operating point in the high-compression region across both models. The paper also notes that Hilbert achieves the highest compression ratios among the evaluated configurations, reaching 2.67× and 3.00× on the two models with PPL increases of 0.35 and 0.51. The results place Hilbert at a competitive operating point in the high-compression region across both models. The paper also notes that Hilbert achieves the highest compression ratios among the evaluated configurations, reaching 2.67× and 3.00× on the two models with PPL increases of 0.35 and 0.51. The results place Hilbert at a competitive operating point in the high-compression region across both models. The paper also notes that Hilbert achieves the highest compression ratios among the evaluated configurations, reaching 2.67× and 3.00× on the two models with PPL increases of 0.35 and 0.51. The results place Hilbert at a competitive operating point in the high-compression region across both models. The paper also notes that Hilbert achieves the highest compression ratios among the evaluated configurations, reaching 2.67× and 3.00× on the two models with PPL increases of 0.35 and 0.51. The results place Hilbert at a competitive operating point in the high-compression region across both models. The paper also notes that Hilbert achieves the highest compression ratios among the evaluated configurations, reaching 2.67× and 3.00× on the two models with PPL increases of 0.35 and 0.51. The results place Hilbert at a competitive operating point in the high-compression region across both models. The paper also notes that Hilbert achieves the highest compression ratios among the evaluated configurations, reaching 2.67× and 3.00× on the two models with PPL increases of 0.35 and 0.51. The results place Hilbert at a competitive operating point in the high-compression region across both models. The paper also notes that Hilbert achieves the highest compression ratios among the evaluated configurations, reaching 2.67× and 3.00× on the two models with PPL increases of 0.35 and 0.51. The results place Hilbert at a competitive operating point in the high-compression region across both models.

Improvements for AI systems

textbfTheoretic Foundation for Compression Ratio Predictability: Establishing a principled tool for model compression by proving that under a Diophantine condition, a finite trajectory of M = O(ϵ −(d+ν) states in the irrational winding constitutes an ϵ-net over the d-dimensional weight space. This links state resolution, decompression error, and compression ratio in a predictable manner, transforming it from a heuristic into a principled, predictable tool.
textbfGeneralized Compression Framework: Unifying four distinct families of dynamic systems—space-filling curves (Hilbert, Peano, Morton/Z-order, Snake), chaotic systems (Lorenz), congruential and pseudo-random generators (LCG, PCG), and low-discrepancy sequences (Halton)—to create a broader toolbox for model compression. This allows researchers to select the optimal system based on desired properties like coverage, locality, and indexability properties, leading to tailored compression ratios and decompression fidelity.
textbfScalable Engineering Techniques: Implementing KD-tree and coordinate-template acceleration for fast state assignment to handle large models effectively. This addresses the major computational cost by reducing assignment overhead during compression and lookup overhead during decompression, enabling the framework to scale to large models as well as outlier identification to control the error.