PQDT: Pseudo-Query Dual Transformer for Robust Point Cloud Restoration

arXiv:2605.25127 · cs.CV, cs.LG · Submitted 2026-05-24 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "PQDT: Pseudo-Query Dual Transformer for Robust Point Cloud Restoration".

Jane: The gist The proposed Pseudo-Query Dual Transformer (PQDT) framework introduces a unified,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: Hey everyone, welcome back. So we're talking about this paper today, "PQDT: Pseudo-Query Dual Transformer for Robust Point Cloud Restoration". It’s from Haoqing Wu, Alexa Nawotki, Jochen Garcke at Mercedes-Benz AG and the University of Bonn and Fraunhofer SCAI. Basically, they’re tackling the problem of taking real-world point clouds that are messy—you know those with noise or holes—and trying to pull out clean shapes.

Jane: Yeah, it's a fundamental representation in computer vision, but when you get real sensor data, it gets incomplete and noisy because of things like occlusions or sensor limits. So recovering detailed shapes from that bad data is super important for any downstream task.

Tom: Exactly. And what they claim with the PQDT framework is that it’s a unified network that takes point clouds as input directly, and it adapts to different types of degradation scenarios to reconstruct high-quality geometry. They're saying existing methods often rely on global features, which lose fine detail and get messed up easily by varying input quality.

Lu: What I find interesting about this paper is how they introduce this Pseudo-Query module inside a Transformer backbone, which they use to reformulate geometric translation into two cooperative stages to boost structural clarity and local detail preservation. It’s essentially a way to do it adaptively rather than relying on one fixed approach for everything.

Meng: That sounds ambitious, but I'm always curious how it handles the practical stuff. When you say "adaptively reconstruct high-quality geometry under diverse degradation scenarios," what kind of real-world degradation are we talking about here? Are we looking at just noise, or are they handling things like severe incompleteness and outliers?

Jane: That’s a good question, Meng. The paper mentions that the input point clouds can suffer from incompleteness, noise, outliers, and irregular density because of sensor limitations or occlusions. And the authors show it handles complex combinations of completion, deformation, and denoising degradations. They aren't just dealing with one type of problem; they’re tackling a mix.

Tom: Right. And to get to that high-quality reconstruction, the core idea is this Pseudo-Query module within the Transformer backbone. It seems they decompose the transformer into two complementary stages: an observation-guided stage and a prior-guided stage that refines those queries.

Lalam: I see them generating intermediate entities called pseudoqueries, which are used for observation guidance in Stage I, stabilizing coarse geometry from distorted observations, while Stage II refines those into consolidated queries under the influence of learned shape priors. It’s like they have a two-step process to get there.

Lu: The mechanism involves extracting local features first via a transition-down module, and then translating those through these dual transformers. They use spherical points as static query initialization, which is then built up using geometric embedding decoder blocks that act like a DETR-like decoder to build the final proxies.

Paper summary: Meng: Okay, so they have Stage I generating initial pseudoqueries Qps through an encoding-decoding process, and then Stage II further refines those into consolidated queries Q using another encoding stage and a dynamic query selection module. That sounds like a lot of processing at once. What’s the specific mechanism for that refinement?

Tom: The refinement involves this DQS, or Dynamic Query Selection module, which adaptively filters out redundant or noisy candidate points. Instead of just using some fixed sampling method, they use a Perturb-and-Top-k strategy to select the most representative features. They also use a two-step encoding scheme where those auxiliary pseudoqueries act as noiseresilient anchors while the DQS ensures only the best features are kept throughout the query generation process.

Jane: That selection process sounds crucial for stability, especially when dealing with noisy data. And they employ this Geometric-Embedding Self-Attention strategy to capture global geometric structures by limiting relations to k-nearest neighbors, which results in a sparse representation Rs where k is much smaller than M.

Lu: The sparse representation is important because it reduces the size of the embedding from RM times M times Ce down to RM times k times Ce when using that strategy, which keeps the complexity manageable while still retaining essential geometric relationships. They also use a multi-scale feature extraction strategy, using EdgeConv from DGCNN hierarchically with light-weighted transformer encoder blocks appended at each level of the transition down process.

Meng: From a deployment standpoint, how does this structure affect the computational cost? If we’re running this on a large point cloud for production, what’s the overhead of these dual encoders and the multiple DQS steps? It sounds heavy to run compared to simpler methods.

Tom: The paper trains the model using a multi-level reconstruction loss Lrec, which is the sum of Chamfer Distance losses at each level, defined as l CDell1(P↑i, FPS(G, P↑i)). They also have an intermediate loss Lpq to constrain those pseudo-query points Ppq against the ground truth G. The overall loss is L equals Lrec plus Lpq.

Jane: That dual loss structure is interesting because it forces the network to do two things simultaneously: get the final high-quality reconstruction and keep those intermediate query points accurate relative to the true geometry. It’s a way of enforcing consistency throughout the entire process.

Lu: The training setup includes using PyTorch with the Adam optimizer and a cosine annealing learning rate scheduler. They trained this on ShapeNet-fifty-five/thirty-four and ShapeNet-Deform datasets for three hundred epochs with a batch size of thirty-two. And they report final coarse prediction outputs Fcpred as H in R512 times three hundred eighty-four and Pcpred as PQ in R512 times three.

Paper summary: Meng: So the final output sizes are pretty large, with one of those features being over three hundred thousand points at this coarse level. That’s a lot of data to handle before you get to the fine-level refinement stages. How do you manage that scale for inference?

Tom: The paper shows that on curated benchmarks, PQDT surpasses state-of-the-art performance in general three dee restoration across multiple tasks and degradation scenarios <ref:2605.25127#pg1>. For instance, on ShapeNetCar-Occ, it achieves the best average CD2 of zero point eight two and an F1-Score of zero point two six one.

Jane: And for another set, the PFS dataset, PQDT reduces the CD2 by seventy-six point eight percent compared to the second-best method while improving F1 and F0 point 5 scores by twenty-seven point one percent and forty point one percent respectively. That shows a significant improvement in robustness across different tests without needing to do specific task tuning for every single scenario.

Lu: The authors also confirm that on ShapeNet-Deform, PQDT achieves consistently superior performance across nearly all difficulty levels, surpassing strong baselines like SeedFormer, AdaPoinTr and AnchorFormer. They even analyze their design and confirm the effectiveness of each component they proposed.

Meng: So what’s the limitation here? I need to know where this model stops working or what it can't do reliably in a real deployment setting. The paper has to admit some weaknesses, right?

Tom: Yes, it does. The paper discusses the limitations of their approach when dealing with certain input conditions, and they flag that the performance is sensitive to the quality of the input point clouds themselves. They also mention that while it handles complex combinations of degradation, if you move outside those specific scenarios defined in their training data, its robustness might drop off.

Jane: So what this means for people who are just listening to the show is that this approach gives us a powerful tool for three dee restoration—it’s unified and adaptive—but we have to be careful about feeding it data that falls outside the specific degradation patterns they were trained on <ref:2605.25127#pg1>. It’s not a perfect fix for every possible sensor issue out there.

Lu: Ultimately, the paper proposes this novel unified, point-only backbone for robust three dee restoration, which is what enables more versatile three dee perception in general <ref:2605.25127#pg1>. They are aiming to create a foundation that can handle diverse inputs better than existing global bottleneck features.

Meng: So to wrap up, the PQDT framework is a dual transformer approach that uses pseudoqueries generated in two stages—observation-guided and prior-guided—to adaptively reconstruct clean geometry from noisy point clouds. It's powerful on benchmarks like ShapeNetCar-Occ and ShapeNet-Deform, but it’s sensitive to input quality outside its trained distribution.

Tom: That’s a solid summary of what the PQDT paper is about. It’s a novel unified backbone that uses pseudoqueries to bridge the gap between raw, messy data and high-quality three dee reconstruction <ref:2605.25127#pg1>. We'll talk more on this in our next segment about how these ideas are applied in real applications.

Conclusion: Tom: So we’ve been deep in the weeds on this PQDT paper from Wu, Nawotki, Garcke at Mercedes-Benz and Bonn University. What’s actually happening here is they've created a unified network that uses two stages of pseudoqueries to rebuild messy three dee shapes from bad point clouds.

Jane: That's right, Tom. They call it the Pseudo-Query Dual Transformer framework, and they are showing how this approach handles noise, missing data, and irregular density all at once by adapting its internal structure.

Lu: What’s really interesting is that they move away from just relying on global features and instead use those intermediate pseudoqueries to guide the reconstruction. It’s like they have a way to stabilize the shape even when the raw input data is pretty distorted.

Meng: I need to know what this means for deployment, Jane. If it's using these complex dual stages, how much computational overhead are we talking about when we try to run this on a production server?

Jane: Well, Tom says they trained it on ShapeNet-fifty-five and ShapeNet-Deform and it actually performs better than some strong baselines across almost every difficulty level. They're not just hitting one benchmark; they're showing consistency.

Tom: Exactly, and the results are solid numbers. On ShapeNetCar-Occ, they got a CD2 of zero point eight two and an F1 score of zero point two six one which is quite high for this kind of restoration task.

Lalam: From my perspective in the literature, this unified backbone concept is what matters most. It suggests a new way to build representations where you don't have to build separate networks just for completion or just for denoising.

Jane: So the big picture here is that they’ve given us a more versatile tool than we had before for tackling real-world three dee perception problems. But we still need to look closely at where it might struggle if the input data gets totally out of its training distribution.

Tom: Right, and that’s the caveat. It works great on the curated sets they used, but if you feed it something completely new, the performance might drop off significantly because of that reliance on learned priors.

Lu: So the title itself points to this idea—pseudoqueries are guiding two distinct stages to create robustness. It’s less about one perfect network and more about a flexible system that balances raw data evidence with what it has learned.

Jane: And if you only listen for one thing from this paper, I’d say it's how they manage that input quality by using these dual query refinement stages instead of just one fixed way to process the geometry.

Tom: Yeah, so PQDT is a point-only backbone that aims to make three dee restoration more robust and adaptable to various types of damage. We’ll talk more later about what these ideas mean for building the next generation of three dee perception models.

Haoqing Wu, Alexa Nawotki, Jochen Garcke

Mercedes-Benz AG · University of Bonn · Fraunhofer SCAI

cs.CV, cs.LG

Submitted: 2026-05-24

Updated: 2026-05-24

Comments: To be published in The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2026

Journal ref: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026, pp. 24195-24205

Code: https://github.com/ins-uni-bonn/PQDT

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 80/100

The gist: The gist The proposed Pseudo-Query Dual Transformer (PQDT) framework introduces a unified, point-only backbone that adaptively reconstructs high-quality geometry under diverse degradation scenarios

Key concepts

Pseudo-Query Module
This module generates intermediate entities called pseudoqueries within a Transformer backbone. It is implemented in two stages: an observation-guided stage that stabilizes coarse geometry, and a prior-guided stage that refines these queries using learned shape priors to ensure structural plausibility.
Geometric-Embedding Self-Attention (SGE)
This strategy enhances global geometric structure by formulating sparse attention. It limits point relations to k-nearest neighbors, resulting in a sparse representation where the embedding size is significantly reduced, making the model more efficient at capturing essential geometric relationships.
Dynamic Query Selection (DQS)
The DQS modules (S1 and S2) adaptively filter candidate points. Instead of fixed sampling methods, they use a Perturb-and-Top-k strategy with Gumbel noise to select the most representative features, effectively filtering out noisy or redundant input points for better reconstruction quality.

Terminology

Summary

The gist The proposed Pseudo-Query Dual Transformer (PQDT) framework introduces a unified, point-only backbone that adaptively reconstructs high-quality geometry under diverse degradation scenarios by reformulating geometric translation into two cooperative stages

How it works

The core of the PQDT approach is a Transformer backbone that employs auxiliary intermediate entities termed pseudoqueries to recover high-quality shapes under diverse input degradations. This network decomposes the transformer into two complementary and functionally distinct stages: (i) an observation-guided stage, which generates pseudoqueries by broadly attending to encoded inputs, stabilizing coarse geometry from distorted observations; and (ii) a prior-guided stage, which refines these queries under stronger influence of learned shape priors, selectively balancing prior knowledge with input evidence to enforce geometric consistency and structural plausibility.

The process involves several key steps:

  1. Extraction of local features: The framework first extracts the local feature Fcsrc with a light-weighted transition-down module.

  2. First Stage Encoding and Pseudoquery Generation: This stage generates pseudoqueries Qps through an encoding–decoding process, providing observation-guided query initialization. The auxiliary pseudoqueries Qps are first decoded from the outputs of ME1 and the static spherical point queries Qsph in a DETR-like manner.

  3. Second Stage Refinement: These entities will be further refined through a second encoding stage after another DQS step, producing consolidated queries Q. The final queries Q are then aggregated using the max-pooled features fg1 and fg2 from the outputs F1 and F2 of the two encoders, together with the selected query points PQ.

  4. Coarse-to-Fine Expansion: The output of the second transformer decoder are highdimensional point proxies H = HiM i=1 with Hi ∈ RCH containing contextual shape information. To unwrap detailed geometry from these proxies, they are fed into a hierarchical upsample transformer layers to progressively refine the point cloud and recover fine-grained geometric details.

Key Architectural Components

The PQDT framework is built upon several sophisticated modules designed to enhance robustness and geometric fidelity. The network utilizes a two-step encoding scheme to enhance robustness in sequence translation, where the auxiliary pseudoqueries act as noiseresilient anchors, while the DQS mechanism ensures that only the most representative features are retained throughout the query generation.

The Geometric-Embedding Self-Attention (SGE) strategy is crucial for capturing global geometric structures. This strategy involves a sparse formulation by limiting relations to the k-nearest neighbors for each point, resulting in a sparse representation Rs where k << M. The attention score of the attention head with SGE is formulated using distance embedding (DE) and angular embedding (AE) from input point coordinates. This sparse representation not only retains essential geometric relationships but also significantly reduces the size of the embedding from RM×M×Ce to Rs in RM×k×Ce, where k << M.

Adaptive Query Generation and Selection

The quality of the query sets is critical to reconstruction performance, which is managed by two Dynamic Query Selection (DQS) modules, S1 and S2. These modules adaptively filter out redundant or noisy candidate points. Instead of learnable Gumbel-Top-k subset sampling, the model adopts a simple, non-learnable Perturb-and-Top-k strategy which yields better stability and performance. This strategy involves computing standardized scores Zcand = (MLP(Xcand) − µ)/σ, perturbing them with Gumbel noise gi = − log(− log(Uniform(0, 1))), and selecting the indices of the top-k scores.

The model also employs a multi-scale feature extraction strategy where EdgeConv head from DGCNN is used hierarchically and light-weighted transformer encoder blocks are appended to refine features at each level of the transition down process. The positional encoding uses a sparse geometric embedding (SGE) strategy that computes distance and angular embeddings from input point coordinates.

Training and Optimization

The model is trained using a multi-level reconstruction loss Lrec defined as the sum of Chamfer Distance losses at each level, Lrec = Σ l CDl1(P↑i, FPS(G, P↑i)). Additionally, intermediate pseudo-query points Ppq are constrained by the ground truth G with a separate loss Lpq. The overall loss function is the sum of these two losses: L = Lrec + Lpq.

The model is trained using PyTorch and the Adam optimizer with a cosine annealing learning rate scheduler. For ShapeNet-55/34 and ShapeNet-Deform datasets, the network is trained for 300 epochs with a batch size of 32. The final coarse prediction outputs are Fcpred = H ∈ R512×384, Pcpred = PQ ∈ R512×3.

Experimental Results and Robustness

Extensive experiments on curated benchmarks demonstrate that the approach surpasses state-of-the-art performance in general 3D restoration. The method achieves state-of-the-art performance across multiple tasks and degradation scenarios, significantly surpassing prior approaches in robustness and reconstruction quality. For ShapeNetCar-Occ, PQDT achieves the best average CDl2 of 0.82 and F1-Score of 0.261. On the PFS dataset, PQDT reduces CDl2 by 76.8% compared to the second-best method, while improving F1- and F0.5-Scores by 27.1% and 40.1%, respectively. The results on ShapeNet-Deform show that PQDT achieves consistently superior performance across nearly all difficulty levels, surpassing strong baselines such as SeedFormer, AdaPoinTr and AnchorFormer.

The model design analysis confirms the effectiveness of each proposed component; Model B introduces advanced query generation, and Model C further replaces the standard linear projection from global features with a query-based DETR-like decoder for seed proxy prediction. The geometric-embedding (GE) in place of raw coordinate-based positional encoding achieves the best overall performance.

The final conclusion is that the proposed model provides a novel unified, point-only backbone for robust 3D restoration, enabling more versatile 3D perception.

--- Page 5 ---

PQDT: Pseudo-Query Dual Transformer for Robust Point Cloud Restoration

Haoqing Wu1,2 Alexa Nawotki1 Jochen Garcke2,3 Mercedes-Benz AG, Germany University of Bonn, Germany Fraunhofer SCAI, Germany

--- Page 6 ---

Related Work Much of the research into 3D shape restoration focuses on completion tasks. Deep learning based methods have shown great potential in 3D geometric processing. Recent works focus on point cloud-based shape completion, which is more efficient and scalable due to the sparsity of point clouds as 3D representations.

--- Page 7 ---

Multi-task Point Cloud Restoration In real-world scenarios, the acquired point clouds are usually incomplete and noisy simultaneously. The common strategy is to handle the single tasks separately with dedicated networks.

--- Page 8 ---

Method To effectively capture the geometric structure of defective input point clouds, we propose a transformer-based architecture that employs auxiliary pseudoqueries as a noiseresistant backbone. Overall, the proposed framework adaptively balances input fidelity and learned priors, enabling the network to handle incomplete, noisy, or unevenly distributed point clouds.

--- Page 9 ---

PQDT: Pseudo-Query Dual Transformer for Robust Point Cloud Restoration

Given an incomplete and noisy input point cloud Pfsrc, we first extract the local feature Fcsrc with a light-weighted transition-down module, and let the coarse level point-wise feature be translated via several novel geometric-embedding encoder (GEE) blocks. With spherical points serving as the static query initialization, we employ our geometricembedding decoder (GED) blocks to build up a DETR-like decoder. Intermediate entities, serving as an observation guidance, are generated after the first decoding stage and further filtered by a dynamic query selection (DQS) module to obtain Ppq and Fpq, which we call pseudo-queries. Instead of being used directly for decoding (hence the term pseudo-queries), these entities will be further refined through a second encoding stage. After another DQS step, the consolidated queries are passed to the decoder.

--- Page 10 ---

Transformer with Auxiliary Pseudo-Query As overall structure in Fig. 2 shows, with a two-step encoding scheme we aim to enhance robustness in sequence translation: the auxiliary pseudoqueries act as noiseresilient anchors, while the DQS mechanism ensures that only the most representative features are retained throughout the query generation.

Improvements for AI systems

  1. Bold header: Dual-Stage Pseudo-Query Backbone

This improvement replaces global bottleneck features with a novel transformer backbone that adaptively balances input fidelity and learned priors. This allows the system to handle incomplete, noisy, or unevenly distributed point clouds more robustly than methods relying on fixed global feature pooling.

  1. Bold header: Geometry-Aware Positional Encoding

The implementation of sparse geometric embedding (SGE) strategy using pair-wise distance and triplet-wise angular embeddings allows the transformer to capture transformation-invariant spatial relationships. This ensures the model learns geometric context refinement in each level during upsampling, leading to a more fine-grained and stable manner to recover the detailed geometry.

  1. Bold header: Dynamic Query Selection (DQS) with Gumbel Perturbation

The introduction of two DQS modules, S1 and S2, uses a simple, non-learnable Perturb-and-Top-k strategy with Gumbel noise to select representative features. This mechanism allows the model to adapt its query set based on input reliability: "when input information is sparse or highly corrupted, the network relies more heavily on shape priors from the pseudo-queries; conversely, when the input is relatively clean or locally consistent, more input points are preserved as queries."

  1. Bold header: Unified Restoration Capability

By proposing a framework that unifies completion, denoising, and deformation within a single model, the system can perform general 3D restoration where multiple fundamental 3D tasks, including completion, deformation, and denoising, are combined. This makes the resulting AI capable of handling complex combinations of degradations simultaneously.

  1. Bold header: High-Fidelity Industrial Reconstruction

The inclusion of the ShapeNetDeform and Patchified Freeform Surface (PFS) datasets allows the model to achieve superior capability in producing consistent, deformation-aware, and high-fidelity point cloud reconstructions for real-world industrial applications like reverse engineering and repair.

Sources

Related papers