LCPNet: Latent Consistent Proximal Unfolding Network for Infrared Small Target Detection

The gist: Latent Consistent Proximal Unfolding Network (LCPNet) proposes a novel deep unfolding method for infrared small target detection that preserves physical constraints by operating in latent space and utilizes a shared optimization memory to guide all decomposition variables across stages.

How it works

1. The method verifies that the low-rank prior remains valid in latent representations and performs unfolding in this space, preserving physical constraint while avoiding repeated compression of intermediate states.

2. It derives a Latent Consistent Proximal (LCP) solver that evolves each latent variable from its previous state rather than reconstructing through an indirect residual, and stabilizes small target updates through task-adaptive normalization and gain control.

3. A Shared Optimization Memory (SOM) is introduced, a common historical state shared by all decomposition variables to provide coordinated guidance across unfolding stages.

Latent-Space Decomposition Formulation

1. The formulation starts with the image-domain separation problem: X = B + T + N, s.t. X, B, T, N ∈ R H×W. (1).

2. To address limitations of image-domain unfolding where variables are transmitted between adjacent stages and projected back to low-dimensional images, the decomposition is formulated in the latent space: X = B + T + N, s.t. X, B, T, N ∈ R H×W×C. (3).

3. The general decomposition objective in this latent-space physical model is written as L = RB(B) + RT (T) + RN (N; σ).

4. Following the Alternating Direction Method of Multipliers (ADMM), the problem is decomposed into three main subproblems for B, T, and N.

Latent Consistent Proximal (LCP) Solver

1. The LCP solver is derived from the B-subproblem, which involves minimizing a surrogate objective JˆB(B) that includes both the unknown latent regularizer RB(·) and a quadratic term.

2. The closed-form solution for the background variable B is given by B k = LB + µB B k−1+µB LB + µB V k−1 B − ∇RB(B k−1)LB + µB.

3. This update can be written as a gradient-like descent step: B k = B k−1 − ηBG k−1, where ηB = 1/(LB +µB).

4. The ideal update direction is approximated by a learnable latent proximal surrogate Gk−1B ≈ Gek−1B = ΨB(Bk−1 − Pk−1, hk).

Shared Optimization Memory (SOM)

1. SOM constructs a joint observation at stage k by concatenating the current latent variables: ok = [B k, T k, N k, Y k, X], (38).

2. An optimization state cell updates the shared state hk through a gated recurrent mechanism: zk = σ(Wz ∗ [ok, hk−1] + bz), rk = σ(Wr ∗ [ok, hk−1] + br), h˜k = tanh(Wq ∗ [ok, rk ⊙ hk−1] + bq), hk = (1 − zk) ⊙ hk−1 + zk ⊙ h˜k.

3. The shared state hk is fed into the proximal surrogate updaters for all primal variables, making it a global control variable for the unfolding solver.

4. The SOM forms an unbounded-order effective memory of the whole latent decomposition system rather than isolated branch-level feature cache.

Network Architecture and Implementation

1. LCPNet lifts the input into the latent domain and performs K stages of latent decomposition unfolding, with each stage updating variables under the latent decomposition constraint.

2. The update module in each stage estimates the learned proximal direction from the variable residual and shared optimization state.

3. To stabilize updates, Group Normalization (GN) is used instead of standard batch normalization because it reduces dependence on batch-level statistics while preserving scene-specific feature variations.

4. Spectral Normalization (SN) constrains the gain of the convolutional updater, preventing background fluctuations from being repeatedly amplified and improving discrimination of weak target responses.

Experimental Results

1. Quantitative comparison shows that LCPNet-6 obtains the best performance on larger benchmarks like NUDT-SIRST [12] and IRSTD-1K [10], with IoU/F1 of 95.77%/97.84% and 65.95%/79.48%, respectively.

2. LCPNet variants achieve the lowest False Alarm Rate (Fa) on all four benchmarks, indicating that the proposed decomposition is more conservative in suppressing background responses.

3. Visual comparisons show that LCPNet-6 completely detects both targets in cases where other methods miss one target or mix the target response with nearby background.

4. Ablation studies confirm that the benefit of latent-domain execution is clearer when the updater is equipped with stronger regularization, showing gains in IoU and F1 while reducing Fa.

5. The SOM provides more consistent gains than simple concatenation or ConvLSTM, increasing IoU/F1 by 0.27/0.14 on NUDTSIRST and 0.43/0.31 on IRSTD-1K.

6. Increasing the unfolding depth from K = 2 to K = 4 generally improves detection quality, suggesting that a moderate-to-deep unfolding depth provides better representation refinement.

Discussion and Analysis

1. LCPNet differs from RPCANet-style networks by lifting the observation and decomposition variables into a latent domain, which gives each stage a higher-dimensional interface where structures can evolve more flexibly.

2. The S-style proximal surrogate update keeps the proximal form while removing SOM-related state input for fair comparison, resulting in gains in IoU and F1 while lowering Fa.

3. The SOM separates LCPNet from RPCANet++ because it encodes the coupled state of B, T, N, Y, and X and feeds the same historical state to all primal-variable updates.

Conclusion

LCPNet provides an effective and interpretable framework for IRSTD by confirming that low-rank decomposition prior remains valid in latent representations. The method extends unfolding from image domain to latent domain, preserving the physical constraint while avoiding repeated compression of intermediate states. Through modeling and detailed derivations on latents, the proposed solver turns indirect residual reconstruction into direct proximal-state evolution and introduces regularized update dynamics for stable small target separation. The SOM further upgrades branch-specific memory into a shared optimization state, so historical information can guide all coupled decomposition variables across stages. Extensive experiments on four public benchmarks demonstrate the effectiveness of LCPNet. The quantitative and qualitative results show that LCPNet achieves accurate and robust detection across cluttered scenes and multiple-target cases. The ablation studies progressively demonstrate that these benefits stem from the proposed improvements<ref:2607.

1. Bold latent-space unfolding for physical constraint preservation: The system can lift the execution domain of unfolding from image to latent, preserving the decomposition constraint while avoiding repeated projection of intermediate states, which allows background, target, and noise representations to evolve on a richer latent manifold.

2. Implement direct proximal-state evolution: The system will adopt the LCP solver derived in Eq. 17 to ensure each variable is evolved from its previous state rather than reconstructed from an indirect residual, thereby guaranteeing the continuity of background trajectory and may cause target leakage is avoided.

3. Utilize shared optimization memory for coordinated guidance: By replacing branch-specific memory with Shared Optimization Memory (SOM), the system can provide a common historical state shared by all decomposition variables to provide coordinated guidance across unfolding stages.

4. Employ group-wise normalization and spectral normalization: The updater will use group-wise normalization (GN) to normalize features within each sample, reducing the dependence on batch-level statistics and Spectral Normalization (SN) to constrain the gain of the convolutional updater, preventing background fluctuations from being repeatedly amplified.

5. Adapt model depth based on scene complexity: The system can dynamically adjust the unfolding stage number, leveraging Table V's findings that a moderate-to-deep unfolding depth provides better representation refinement rather than arbitrarily increasing stages which bring smaller and less stable gains.

Infrared small target detection (IRSTD) aims to identify long distance small targets from complex infrared backgrounds, and is a fundamental task in remote sensing. Deep learning methods have improved IRSTD by learning discriminative image-to-mask mappings, but such feed-forward designs often underuse physical decomposition structure between targets and backgrounds. Deep unfolding methods partially address this issue by embedding model-driven iterations into neural networks, yet existing designs still operate mainly in image domain and use updates and memory mechanisms that are not fully coupled with underlying optimization process. To address these limitations, we propose Latent Consistent Proximal unfolding network (LCPNet). First, we verify that low-rank prior remains valid in latent representations and perform unfolding in this space, preserving physical constraint while avoiding repeated compression of intermediate states. Second, we derive a Latent Consistent Proximal (LCP) solver that evolves each latent variable from its previous state rather than reconstructing through an indirect residual, and stabilizes small target updates through task-adaptive normalization and gain control. Third, we introduce Shared Optimization Memory (SOM), a common historical state shared by all decomposition variables to provide coordinated guidance across unfolding stages. Extensive experiments on four public benchmarks demonstrate that LCPNet achieves accurate and robust detection with low false-alarm rates, together with competitive efficiency among high-accuracy deep unfolding methods. Model and code are available at Model and code are available at https://github.com/Tianfang-Zhang/LCPNet.