Lifting 3D Gaussians: A Wavelet Multiresolution Representation for Scalable Gaussian Splatting Streaming

summary

The gist

Streaming 3D Gaussian Splatting requires highly scalable, progressive representations The paper proposes EvoGS, a continuous-layering representation organized as an Evolution Tree…

In short

The paper introduces Lifting 3D Gaussians, a wavelet multiresolution representation for scalable Gaussian Splatting streaming. It replaces discrete layering with a continuous structure organized as an evolution tree. This method allows higher layers to structurally correct errors in lower layers using wavelet-inspired refinement signals, leading to better quality-to-storage efficiency and smooth quality transitions during progressive streaming.

Key concepts

Continuous Layering Representation
Instead of fixed, separate quality levels (discrete), this approach uses an Evolution Tree where each splat refines into two children. This structure creates a continuous progression of detail, allowing higher quality layers to actively correct and refine the geometry inherited from lower layers without accumulating errors.
Evolution Tree
This is the organizational structure for the continuous layering representation. It functions like a family tree where an internal splat node evolves into two children through learned refinement equations. This hierarchy explicitly defines parent-child relationships, enabling structural error correction across quality tiers.
Wavelet Multiresolution Representation
The refinement mechanism uses principles from wavelet decomposition to generate the signals that guide layer evolution. This wavelet-inspired approach ensures that the inter-layer refinement residuals are inherently sparse, which is crucial for making the overall representation highly compressible and efficient for streaming.
Structural Error Correction
A core feature where child nodes are explicitly designed to correct errors from their ancestors. Using learned refinement directions ($ heta$) and asymmetry factors ($\alpha$), higher layers can actively reposition or adjust ancestral splats, resolving the error accumulation issues found in traditional discrete layering methods.

Terminology used across episodes

This episode discusses

The paper

Lifting 3D Gaussians: A Wavelet Multiresolution Representation for Scalable Gaussian Splatting Streaming · Read on arXiv

Yuang Shi, Simone Gasparini, Géraldine Morin, Wei Tsang Ooi

National University of Singapore · IRIT - University of Toulouse, France · IPAL, IRL2955, Singapore

3D Gaussian Splatting (3DGS) represents a scene as an unstructured set of primitives optimized at a single scale, offering no native support for progressive delivery to heterogeneous devices. Existing scalable approaches organize these primitives into independent quality layers or spatial hierarchies. However, maintaining separate splat representations across scales introduces redundancy, while independent layers have limited ability to correct coarse geometry as finer detail arrives. We present LiftGS, a multiresolution representation that organizes a scene into a coarse Gaussian set and a tree of refinements. Each refinement replaces one coarse Gaussian with two finer children whose attributes are learned relative to their parent. Inspired by wavelet lifting, we encode these changes using a shared detail vector and per-attribute factors that allow the two children to differ asymmetrically from the parent. A client first receives the coarse set, then reconstructs and substitutes children as refinement data arrives. This construction reuses previously transmitted attributes to encode finer detail and removes replaced parents from the active rendering set, reducing both refinement payload and rendering memory. Extensive experiments on four benchmark datasets show LiftGS cuts splat redundancy from over 65% to under 25%, reduces transmission payload and GPU memory by up to 2.4x and 5.5x over state-of-the-art scalable-3DGS baselines, achieves smooth quality transitions in simulated progressive delivery, and compresses a further 8.7x with standard coding at negligible quality loss. Project website: https://yuang-ian.github.io/liftgs/

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Lifting 3D Gaussians".

Title and authors: Tom: So we're talking about this new paper today called "Lifting three dee Gaussians: A Wavelet Multiresolution Representation for Scalable Gaussian Splatting Streaming" <ref:2606.07179#pg1>. It looks like they've been tackling the problem of making these huge three dee scenes stream smoothly and efficiently <ref:2606.07179#pg1>.

Jane: That’s right, Tom. It seems like they are moving away from the older ways of doing this, which have some pretty big issues with quality jumps and wasted data when you try to stream things progressively.

Lu: The core idea here is that they replace the discrete layering approach with something continuous called an Evolution Tree, where each splat actually refines into two children. This structure is designed to handle those error accumulations we see in the old methods, so child nodes can structurally correct errors from their parents.

Meng: Structurally correcting errors sounds ambitious; what does that actually mean for the practical side? Does it just make the file bigger, or does it actually help with how fast we can download it?

Tom: Well, on the mechanics side of things, they are using these specific equations to define that refinement. They have C one equals P plus theta one and C two equals P minus alpha times theta. This means they use a shared direction, called theta, and an asymmetry factor called alpha to decide how the splats split when they evolve.

Jane: That's a very technical way of saying they are mathematically defining how the refinement happens between layers. It sounds like this asymmetry factor is key to getting those specific gains in quality and storage efficiency that they mention later.

Lalam: I can confirm that the paper states that using an asymmetric collinear residual, which is option D, gives them five parameters per split to capture complex details while keeping wavelet sparsity. This tells me they are being very deliberate about how much detail each step adds.

Tom: Exactly, they are not just adding random noise or detail at every level; they are using a specific structure that is inspired by wavelets to ensure the refinement signals themselves stay sparse. That sparsity is what makes it good for progressive coding.

Jane: So if I understand correctly, the main thing here is that this wavelet-inspired refinement mechanism creates these inherent sparse signals, which directly translates into better compression when you try to save the model down for streaming purposes.

Lu: It does; and they show some pretty solid numbers on that compression. They report that applying standard compression to those refinement residuals shrinks the storage footprint by a factor of eight point seven times compared to their previous work, which is pretty significant for transmission <ref:2606.07179#pg3>.

Meng: A factor of eight point seven times sounds promising when you’re shipping assets over a network with limited bandwidth <ref:2606.07179#pg3>. But what about the trade-off? Does this better compression come at the cost of visual quality?

Title and authors: Tom: That’s a fair question, Meng. They show that this compression is achieved with minimal visual degradation, and they even report a minimal point error of zero point one dB when they apply standard compression to those residuals.

Jane: So it seems like you get the benefit of much smaller files and better quality than before, all while maintaining a very smooth transition between qualities. That smooth quality progression is another big feature they highlight for streaming applications.

Lu: The continuous layering structure itself is what enables that smoothness; transitioning from one layer to the next isn't just replacing splats randomly; it’s replacing parents with correlated children offset by those sparse refinement signals, which keeps the visual updates minute and localized.

Meng: I worry about how this scales on real hardware. If we have a million splats, does the tree structure keep rendering costs manageable when we are doing per-region adaptive rendering?

Tom: They address that by saying the model is traversed as a tree, which lets clients render different spatial regions at mixed resolutions simultaneously. This fine-grained spatial adaptation is something discrete layering simply can't do structurally.

Jane: That’s a huge deal for real-time applications because it means the user can see high quality where they are looking and lower quality in the periphery without having to load massive files everywhere.

Lalam: And I've looked at the results on certain datasets, like Playroom, and it seems like this continuous layering approach achieves a better visual quality and a smaller model size compared to methods like LapisGS, which is what they’re comparing against.

Lu: Yes, looking at the sample renderings for Drjohnson or Playroom, you can see that instead of the errors just piling up in one layer, the continuous layering structure ensures that higher layers actively correct those ancestral splats through those learned refinements.

Meng: So if I were to deploy this right now, what’s the biggest immediate concern? Is it training time or just deployment complexity?

Tom: The training part is intensive because you are learning all these refinement directions and asymmetry factors, but they show that the resulting model is highly compressible, which helps with storage. The main operational win seems to be the reduction in transmission payload and GPU memory footprint by factors of up to two point five times and five point five times.

Jane: It sounds like for someone who just wants to see cool three dee scenes streamed on a mobile device, this paper suggests a much more robust way than what we've seen before <ref:2606.07179#pg1>. But they also mention that the symmetric variant of this approach has a noticeably higher redundancy rate at level three compared to the asymmetric model because forcing equal magnitude constraints can sometimes put child splats in less optimal geometric positions.

Lalam: That’s an important caveat; it means you have to choose between a slightly more redundant but potentially simpler asymmetric structure and the symmetric one, depending on what you prioritize.

Title and authors: Lu: Ultimately, the paper demonstrates that this continuous layering paradigm provides a way to scale visual fidelity smoothly and proportionally with the data received, unlike discrete layering which has those stagnant improvements punctuated by abrupt jumps at LOD boundaries.

Tom: It seems like they've found a solid way to solve that inherent structural inefficiency in progressive representations by making the refinement explicit between layers.

Jane: So we’re talking about a representation called "Lifting three dee Gaussians: A Wavelet Multiresolution Representation for Scalable Gaussian Splatting Streaming" that uses an evolution tree to make streaming much smoother and smaller <ref:2606.07179#pg1>.

Meng: It sounds like the efficiency gains in storage and rendering footprint are substantial, which is what we need for deployment on constrained hardware.

Lalam: I'm confident that the findings regarding the reduction in splat redundancy from over sixty five percent down to under twenty five percent are a very strong indicator of its success.

Tom: Yeah, it’s a concrete number showing that this isn't just theoretical; there is actual data on how much redundant geometry they are eliminating.

Jane: It really shows how important it is to have those explicit parent-child relationships when you are building a system designed for streaming rather than just static rendering.

Lu: And the way they use wavelet decomposition to generate those refinement residuals gives them that inherent sparsity, which is what makes the signals highly compressible.

Meng: So we’ve covered how it works and why it’s good for size and quality, but where does this leave things for future work? Are there limitations they flag?

Tom: They do mention that the method relies on training the base layer L zero on data D zero for thirty thousand iterations, and subsequent levels Li on Di for another thirty thousand iterations. That’s a lot of training data and time commitment upfront.

Jane: And they also note that the work is built on existing three deeGS codebases, which is good because it means it integrates well with what we already know about three dee Gaussian Splatting <ref:2606.07179#pg1>.

Lalam: The main limitation they point out seems to be that the symmetric variant has a higher redundancy rate at level three compared to the asymmetric model, suggesting a design choice needs to be made based on your specific application needs.

Lu: That’s the trade-off they are presenting; it’s not a perfect solution for every scenario, but it clearly shows how continuous layering resolves the core issues of error accumulation in discrete systems.

Tom: So, in summary, this paper proposes a continuous layering paradigm with an evolution tree to achieve smooth quality transitions and better quality-to-size efficiency by using wavelet-inspired refinement signals.

Jane: It’s a really interesting direction for scalable three deeGS streaming because it tackles the structural limitations of discrete methods head-on with this evolutionary approach <ref:2606.07179#pg1>.

Lalam: I think the findings on dramatic reductions in transmission payload and rendering memory footprint make this paper a very important piece of literature for anyone working on efficient three dee content delivery <ref:2606.07179#pg1>.

The paper's summary: Tom: So, we're diving into this paper now called "Lifting three dee Gaussians: A Wavelet Multiresolution Representation for Scalable Gaussian Splatting Streaming". Basically, they’re ditching the old way of layering three dee Gaussian Splatting because it causes errors to pile up and wastes a ton of data.

Jane: That's right. Think of it like building a house layer by layer where each new floor messes up the foundation below it, and this paper is proposing a way to build that house continuously, like one big tree structure instead of separate boxes.

Lu: The core idea is an Evolution Tree. Each splat node actually refines into two children through some learned refinement process. This structure lets the child nodes fix mistakes made by their parent layers directly.

Meng: So, structurally correcting errors means that when you look at a lower quality layer, the higher quality layers are actively moving and fixing the geometry underneath them? That sounds like a lot of computation.

Tom: It is computationally heavy to train because they're learning all these refinement directions and asymmetry factors, but they show it works by showing how much redundancy they eliminate. They cut that redundancy from over sixty-five percent down to under twenty-five percent.

Jane: That reduction in redundancy is huge because it means the file size shrinks dramatically when you stream it. But what about how smooth the quality jump is when someone is watching?

Tom: The transition between levels isn't a sudden jump anymore. It’s just replacing one set of parent splats with correlated children that are offset by these sparse refinement signals. It results in visual updates that are tiny and localized, which makes the quality progression look continuous instead of abrupt.

Lu: And that sparsity comes from the wavelet-inspired refinement mechanism they use, which means the signals between layers are inherently very compressible. They show applying standard compression shrinks the storage footprint by a factor of eight point seven times at level three compared to older methods like LapisGS.

Meng: Halving or quartering the size of a three dee model is exactly what I need for mobile deployment, and they’re saying this happens with negligible visual loss, only about zero point one decibel in PSNR penalty. That's solid evidence if it holds up in real-world traffic.

Jane: So, for someone just listening to the show, this means that instead of waiting for a huge file to download before you can see anything good, you get a continuous stream where the quality improves smoothly as the data arrives. It’s adaptive rendering across different spatial regions too.

Tom: Right. They can render high quality where you're looking and lower quality in the edges at the same time because they are treating it like a tree structure rather than just one fixed layer for every level of detail.

Lu: The main implication is that this continuous layering paradigm offers a way to scale visual fidelity proportionally with the data received, which is fundamentally different from discrete layering where you get stuck at certain quality levels until you download the next big chunk.

Jane: It really shows how they’ve solved that structural problem of error accumulation inherent in those older progressive methods by making the correction process explicit between every layer. This could be a real deal for immersive three dee streaming applications.

Tom: The training setup is pretty demanding, though, requiring thirty thousand iterations for the base layer and another thirty thousand for each subsequent level, but they show that the resulting model stays compact in terms of memory footprint compared to baselines like LapisGS.

Meng: I see the training time as a hurdle for getting this into production quickly, but if you can get that refinement learning done once, the operational savings from smaller files and faster streaming will be worth it.

Jane: So while there are some upfront costs in terms of training time and complexity, the payoff seems to be a representation that is dramatically more efficient for transmission and better at maintaining a smooth viewing experience. We’ll take a closer look at how this actually scales with bigger scenes next.

The paper's improvements: Tom: So, we’re looking at what they actually improved in terms of performance here in "Lifting three dee Gaussians". They aren't just adding a new layer; they are changing the fundamental way those layers talk to each other.

Jane: That means if you’re trying to stream something, the quality doesn't just jump when you get more data; it actually flows smoothly as the data arrives, which is a big deal for real-time applications.

Lu: The paper focuses on how the wavelet-inspired refinement residuals create this inherent sparsity. It means the signals between layers are structured in a way that is naturally very easy to compress without losing much detail.

Meng: So, if we look at deployment costs, this suggests we can cut down on the memory footprint of these three dee models significantly because they aren't storing all that redundant geometry anymore. How much smaller are we talking?

Tom: They show a dramatic reduction in storage. Applying standard compression to those residuals shrinks the file size by about eight point seven times at level three, which is what makes it viable for transmission over networks with limited bandwidth.

Jane: That compression benefit is massive because it means less data has to travel from the server to your device, and that translates directly into faster downloads and less battery drain on mobile phones.

Lu: And they also show better quality retention when you compress it down. They report only a zero point one decibel penalty in PSNR when applying standard compression, which is very low for this kind of reduction.

Tom: That’s the clean ablation we needed; they didn't sacrifice visual fidelity for the sake of smaller file sizes. It shows that this isn't just a trick to make things small; it’s a structural change that makes the data itself more efficient.

Meng: I gotta ask about those training requirements again because I can’t ignore them, Tom. They said thirty thousand iterations for the base layer and another thirty thousand for each subsequent level. How much does that slow down my pipeline when I'm trying to iterate on different scenes?

Jane: That training cost is definitely a hurdle; it means you have a significant upfront investment in learning those refinement parameters, but once trained, the inference process benefits from that structural efficiency we just talked about.

Lu: The asymmetry factor alpha plays a role here too. They compare the symmetric variant to the asymmetric one and note that the symmetric version actually has a higher redundancy rate at level three because forcing equal magnitude constraints can sometimes lead to less optimal geometric positions for the child splats.

Tom: That’s a real caveat; so you have to choose between a slightly more redundant but simpler structure and something asymmetric that might be geometrically better, depending on what you prioritize in your specific streaming use case.

Jane: It means there's no single perfect answer; the choice depends on whether you want the simplest math or the most accurate geometry correction at every step.

Lu: Overall, this continuous layering structure solves the core problem of error accumulation by allowing higher layers to actively fix lower layers, which is a major structural improvement over discrete methods.

Tom: So it’s not just about getting a better quality model; it’s about building a representation that scales visually and proportionally with the data you actually receive, which is what makes this work for streaming.

Jane: Exactly. It moves us away from those old systems where you either get poor quality or a huge file size, and towards something adaptive where the visual fidelity matches exactly what the network can deliver at that moment. We’ll keep digging into how they handle those per-region adaptive renderings next.

Conclusion: Tom: So, to wrap up, the paper "Lifting three dee Gaussians: A Wavelet Multiresolution Representation for Scalable Gaussian Splatting Streaming" introduces a continuous layering approach using an evolution tree to fix errors structurally and make streaming much smaller and smoother.

Jane: It’s essentially taking those messy discrete layers and turning them into a flowing evolution where the layers actively correct each other, leading to better quality while keeping the file size way down.

Lu: The core of it is that wavelet-inspired refinement signals are inherently sparse, which means we can use compression much more aggressively without hurting the visual quality at all.

Meng: For deployment purposes, it seems like the main trade-off is that you have to commit to a lot of training time upfront for those refinement directions and asymmetry factors before you can even start making models.

Lalam: I’m pretty sure this approach is sound; the reduction in splat redundancy from over sixty-five percent down to under twenty-five percent is a concrete number showing its success in cutting wasted geometry.

Tom: Yeah, that’s what tells me it’s not just theoretical stuff; there's real data on how much redundant geometry they are eliminating across different datasets.

Jane: It really shows how important those explicit parent-child relationships are for building systems designed for streaming, especially when you have limited bandwidth or memory on the client side.

Lu: The way this structure allows per-region adaptive rendering is a huge practical benefit; it means clients can load different quality levels depending on what they’re currently viewing in the scene.

Meng: That spatial adaptation is definitely something we need to think about for our next generation of resource-constrained applications. It changes how much memory we have to budget for rendering at any given moment.

Tom: So, this paper moves us toward a representation that scales visual fidelity smoothly and proportionally with the data you receive, rather than having those sudden quality drops at level boundaries anymore.

Jane: That smooth progression is what makes it so appealing for immersive environments; it’s not just about getting a high-quality image, it’s about the experience of seeing that quality improve gracefully as you wait for more data to arrive.

Lu: This work sets a new baseline by showing how continuous layering can resolve the error accumulation problems that plague discrete representations in scalable three dee Gaussian Splatting.

Tom: It’s a robust and efficient foundation for moving three dee models out of the research lab and into actual streaming applications, provided you can handle the initial training complexity.

Jane: We’ll definitely be watching how this continuous layering paradigm evolves as researchers try to push those refinement mechanisms even further in future work.

More episodes

← Home