TensorDex: A Compact, Tensor-Centric Storage System for Modern AI Models

arXiv:2604.17104 · cs.DC, cs.AI, cs.LG · Submitted 2026-04-18 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "TensorDex: A Compact, Tensor-Centric Storage System for Modern AI Models".

Jane: The gist: TStore presents a tensor-centric storage system that rethinks model storage compression by treating tensors—not models—as the first-class citizen for delta compression,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So what does the actual mechanism behind TensorDex look like? The core idea is decomposing models into tensors and then using bit-level fingerprints to predict how compressible those tensor pairs are.

Jane: They use something called TensorSketch, which is a bit-level CountSketch construction that captures both the bit distribution and structural layout of each tensor without having to read the entire tensor.

Lu: That fingerprint allows them to estimate similarity using what they call the normalized Hamming distance, which measures how similar those bit patterns are on a scale from zero to one.

Meng: So, instead of doing expensive comparisons on full model weights, they use this sketch to quickly guess if two tensors are similar enough to compress the difference between them.

Lalam: And then they have this lightweight regressor called TensorPred that takes those sketches and predicts the reduction ratio without actually performing any real compression or materializing any deltas.

Tom: The prediction model is modeled by an equation: R (pˆ) = α pˆ + β τ + γ (pˆ · τ) + ε, which captures both the raw bit divergence and the entropic cost of that divergence #pg2.

Jane: That formula sounds pretty sophisticated. It’s trying to model not just how different the tensors are in terms of bits, but also how much information is actually contained in those differences #pg2.

Meng: That sounds like it's trying to solve the NP-hard problem of finding the best base for every new model as you upload them, and they do it adaptively.

The paper's summary: Tom: So what are the concrete improvements they are showing? They claim this system achieves a seventy point five percent reduction in storage footprint on a real-world trace of two thousand eight hundred ninety randomly sampled Hugging Face models #pg1.

Lu: They also claim they get high throughput numbers: twenty-two point nine gigabytes per second for compression and twenty-eight point four gigabytes per second for decompression <ref:2604.17104#pg3>.

Meng: That speed is important because if the system is slow, it doesn't matter how much storage you save; we need to deploy this on actual traffic #pg1.

The paper's improvements: Tom: So we just saw how TensorDex tackles model storage by treating tensors as the primary unit for delta compression instead of whole models, which is a big shift in how we think about this stuff.

Jane: It’s really about finding these hidden cross-tensor relationships across different models without needing all that perfect metadata you usually need to start a compression pipeline #pg2.

Lu: The success hinges on exploiting that tensor redundancy, and it proves that fine-grained tensor-level decisions are essential for maximizing delta compressibility #pg2.

Meng: From my side, the implication is this isn't just a theoretical win; it’s a practical storage solution that’s compute efficient enough to be deployed in production environments #pg1.

Lalam: I'm just sure. The overall performance shows that TensorDex is both compute-efficient and high quality, proving that this method works for large-scale model repositories #pg1.

Tom: So the full story of "TensorDex: A Compact, Tensor-Centric Storage System for Modern AI Models" is about moving away from model-level thinking to a data-driven approach that targets fine-grained tensor redundancy #pg1.

Tom: That’s all we have time for today on this paper about TensorDex. Next up, we’re looking at how different AI systems are handling continuous evaluation and evolution through human feedback, which is another huge area of research right now.

Conclusion: Tom: So we’ve seen how TensorDex treats tensors as the primary unit for delta compression instead of whole models, which is a big shift in how we think about this stuff.

Jane: It’s really about finding these hidden cross-tensor relationships across different models without needing all that perfect metadata you usually need to start a compression pipeline.

Lu: The success hinges on exploiting that tensor redundancy, and it proves that fine-grained tensor-level decisions are essential for maximizing delta compressibility.

Meng: From my side, the implication is this isn't just a theoretical win; it’s a practical storage solution that’s compute efficient enough to be deployed in production environments.

Lalam: I'm just sure. The overall performance shows that TensorDex is both compute-efficient and high quality, proving that this method works for large-scale model repositories.

Jane: It's a lot of work to build this system, but it shows how crucial it is to look at the data structure itself when we try to compress large language models.

Lu: This work sets a new direction for how we think about model storage compression and suggests that the future of efficient LLM storage lies in exploiting these fine-grained relationships.

Meng: We need to see if this actually keeps up with the demands of continuously evolving, massive model families, but for now, it's a very promising piece of infrastructure.

Lalam: TensorDex is a system that shows that focusing on the data structure itself can yield substantial storage reductions in AI models.

University of Virginia · Harvard University

cs.DC, cs.AI, cs.LG

Submitted: 2026-04-18

Updated: 2026-10-15

Comments: 12 pages, 6 figures. Systems paper on AI model storage

Code: https://github.com/Cyan4973/xxHash

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 70/100

The gist: The gist: TStore presents a tensor-centric storage system that rethinks model storage compression by treating tensors—not models—as the first-class citizen for delta compression, achieving a

Key concepts

TensorSketch: This is a bit-level fingerprint that captures the exact layout and distribution of bits within a tensor. It's designed to be a better measure of compressibility than just looking at the numbers in the tensor, allowing the system to quickly estimate how much redundancy exists between two tensors without reading their full contents.
TensorPred: This is a lightweight model that takes the TensorSketch fingerprints and predicts exactly how much compression will be achieved for a pair of tensors. It uses mathematical relationships to estimate the reduction ratio, helping the system decide which tensor bases are most effective for future models.
FlexSplit: This is an adaptive clustering algorithm that organizes tensors into groups called 'bases.' It works incrementally as new models arrive, deciding which existing base is best for a new tensor or splitting old groups if they become too large, ensuring the system always selects the most effective reference points.
Delta Compression at Tensor Granularity: Instead of compressing an entire model at once, this technique compresses only the difference (the delta) between two specific tensors. This fine-grained approach allows the system to find and exploit cross-model redundancies where different models share similar tensor structures.

Terminology

Redundancy — 2× in this paper · explained in 2 episodes
Clustering — 1× in this paper · explained in 2 episodes
Fingerprinting — 1× in this paper · explained in 2 episodes
World Model — 1× in this paper · explained in 7 episodes

Summary

The gist: TStore presents a tensor-centric storage system that rethinks model storage compression by treating tensors—not models—as the first-class citizen for delta compression, achieving a 70.5% reduction in storage footprint on a real-world trace of 2,890 randomly sampled Hugging Face models.

Problem Formulation and Challenges

The paper identifies three fundamental limitations in existing delta compression approaches: Missing Lineage Metadata, Model Pair Sensitivity, and Compression Granularity The first limitation is that model lineage information is often missing or unreliable in real-world model repositories, as only 25.8% of over 2.7M Hugging Face models contain usable base-model metadata When this metadata is unavailable, systems fall back to bruteforce pairwise bit distance, which incurs I/O cost that grows quadratically with the size of the model hub The second challenge is Model Pair Sensitivity, where compression effectiveness becomes highly sensitive to the choice of base because fine-tuning induces continuous parameter drift The third challenge is Compression Granularity, as redundancy emerges at the fine-grained tensor level rather than the model level, meaning coarse-grained model-level delta compression obscures substantial cross-model redundancy.

TStore Architecture and Components

TStore is a first-of-its-kind, lossless, tensor-centric storage system that departs from modellevel designs and instead uncovers and exploits hidden crosstensor relationships across models It comprises four components:

TensorSketch is a compressionaware fingerprint that captures bit-level structure and enables efficient similarity estimation without accessing full tensor contents.
A lightweight regressor predicts the compressibility of tensor pairs using TensorSketch without reading full tensor data or performing actual compression.
FlexSplit is a scalable clustering algorithm that identifies the best tensor bases incrementally and adaptively as new models are uploaded.
An optimized codec performs high-performance compression and decompression.

TensorSketch Fingerprinting

TensorSketch is a bit-level CountSketch construction that hashes each element’s individual bit positions into sketch buckets, capturing both bit distribution and layout in a compact, fixed-size fingerprint This fingerprint is designed to be more faithful proxy for delta compressibility than one based on scalar magnitudes. The normalized Hamming distance, pˆ(i, j) = Hˆ (i, j) / (p n), lies in [0, 1], where near zero indicates near-identical bit patterns and ≈ 0.5 indicates random divergence.

Prediction and Planning

TensorPred is a lightweight regression model that converts TensorSketch distances into predicted reduction ratios without materializing any actual delta The predicted reduction ratio R (pˆ) is modeled by the equation R (pˆ) = α pˆ + β τ + γ (pˆ · τ) + ε, where pˆ captures raw bit divergence, τ captures its information-theoretic cost, and pˆ · τ models the nonlinear coupling between divergence magnitude and entropic uncertainty. FlexSplit is a twophase online algorithm that operates entirely on TensorSketch fingerprints and TensorPred estimates to select bases incrementally. Phase I greedily assigns each arriving tensor to the cheapest existing base, while Phase II periodically splits overgrown clusters by promoting tensors whose predicted reduction ratio falls significantly below the cluster average.

Performance and Evaluation

Evaluations on a real-world trace of 2,890 randomly sampled models show that TStore reduces storage footprint by 70.5%, 37% lower than state-of-the-art design It achieves 22.9 GB/s compression and 28.4 GB/s decompression throughput, which are 3.86× and 1.49× faster than the next best system, respectively. TStore-FM++ consistently achieves the highest median reduction across all ten families, followed by TStore-TX. The comparison between ZipLLM and the standalone codecs highlights that TStore’s gains stem from both better planning and better compression, unlocking an additional 12–17 percentage points on top of ZipLLM. Furthermore, TensorSketch maintains a perfect 1.00 Recall@1 across all families, while its end-to-end QPS sustains over 25,000 QPS—500–800× faster than Direct HNSW The prediction model accuracy shows a mean absolute error of only 1.11% and a median absolute error of 0.79% across 5.7M tensor pairs

Scalability

FlexSplit maintains near-constant solving time across all scales, approaching zero even at 800 tensors, unlike ILP which grows super-linearly This scalability enables TStore to continuously ingest large model families while retaining ILP-level compression effectiveness. The ingestion pipeline is implemented as SIMD/AVX2-accelerated kernels over zero-copy mmap buffers, allowing it to scale linearly with thread count.

Conclusion

TStore addresses the granularity mismatch in existing delta compression approaches by exploiting data-driven cross-tensor relationships It synthesizes three novel techniques—a compression-aware fingerprint, a lightweight prediction model, and an adaptive clustering planner—to enable an efficient, metadataoblivious compression pipeline that substantially reduces storage while maintaining high performance in large-scale real-world model repositories >. The paper concludes by demonstrating that the primary benefit comes from exposing previously hidden cross-tensor redundancy, with the compression codec further amplifying these gains. The system is robust because it does not rely on user-supplied model-card annotations. The overall performance indicates that TStore is both compute-efficient and high quality, demonstrating that the algorithm is both compute-efficient and high quality. The system's success proves that fine-grained tensor-level decisions are essential for maximizing delta compressibility >.

Improvements for AI systems

textbf Tensor-centric storage architecture for LLMs (TStore): Develop a system that treats tensors—not models—as the first-class citizen for delta compression. This allows the system to uncover and exploit hidden crosstensor relationships across models, fundamentally improving model storage efficiency by treating redundancy at the fine-grained tensor level.
textbf Content-aware fingerprinting via TensorSketch: Implement a bit-level CountSketch construction that captures both the bit-level distribution and structural layout of each tensor, enabling efficient similarity estimation without accessing full tensor contents. This enables the system to estimate similarity using the normalized Hamming distance (Equation 1).
textbf Predictive compression modeling via TensorPred: Design a lightweight regression model that converts TensorSketch distances into predicted reduction ratios without materializing any actual delta. This model is crucial for solving the challenge of predicting compressibility, as it captures both raw bit divergence and its entropic cost through the formula: R (pˆ) = α pˆ + beta xi + gamma (pˆ · xi) + epsilon.
textbf Adaptive clustering via FlexSplit: Create a twophase online algorithm that uses the predictions to select bases and assign tensors, solving the NP-hard problem. This system performs Greedy Incremental Assignment followed by Multi-Center Splitting, ensuring that it discovers multi-center tensor clusters and selecting high-quality base–delta pairs.
textbf High-performance, parallel compression engine: Implement SIMD/AVX2-accelerated kernels for ingestion and decompression. Specifically, use the TensorX codec to achieve high throughput of 9,821 MB/s during ingestion while maintaining a higher reduction ratio over TensorX.