Hyperspectral Image Data Reduction for Endmember Extraction

arXiv:2512.10506 · eess.IV, cs.LG, eess.SP · Submitted 2025-12-11 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.

Rosa: I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: We're looking at the paper: "Hyperspectral Image Data Reduction for Endmember Extraction".

Dev: Here's the gist of the paper. Endmember extraction from hyperspectral images aims to identify the spectral signatures of materials present in a scene by developing a data reduction technique integrated with a self-dictionary method that substantially reduces computational time without sacrificing accuracy Problem Formulation and Motivation The paper addresses the high computational cost of self-dictionary methods, particularly the LP method, which involves solving an LP relaxation whose size grows quadratically with the number of pixels in a large HSI The research question is whether removing pixels corresponding to mixtures of multiple endmember signatures preserves pixels that lie close to the endmembers Assuming a hyperspectral image follows the linear mixing model with the pure-pixel assumption, this data reduction step is analyzed to show it preserves pixels that lie close to the endmembers The motivation stems from observing that removing columns in the conical hull of an HSI matrix can substantially reduce computational time without discarding columns close to the endmembers Theoretical Analysis of Data Reduction The theoretical analysis under the assumption that the HSI matrix follows the linear mixing model with the pure-pixel assumption shows that even after removing columns located inside the conical hull of the HSI matrix, there still exist columns that lie close to the endmembers, provided that noise level is below a certain threshold This result is summarized as Theorem 3.2, which demonstrates this preservation of relevant information The analysis introduces concepts like r-separable matrices and conditioning parameter ρ(W) to quantify the linear independence of endmember columns Theorem 3.2 proves that if noise level epsilon is bounded by O(ρξ/r), the LP method returns an estimated endmember matrix Wout such that the error with respect to the true endmember matrix W, namely ∥W − Wout∥1, is bounded by O(rϵ/ρ) Development of REDIC Method The paper proposes a data-reduced self-dictionary method abbreviated as REDIC, which integrates data reduction with a variant of the LP method proposed in

34: The REDIC framework involves several steps: 1.

Rosa: Applying SVD-based dimensionality reduction to construct a reduced matrix A′ 2.

Dev: Applying DRS to A′ with parameter p to obtain K ∈ Γ(A′) 3.

Rosa: Constructing the model H using the reduced matrix and solving it with the RCE algorithm 4.

Paper summary: Rosa: So, to recap what we just heard about "Hyperspectral Image Data Reduction for Endmember Extraction," the core idea is tackling the high computational cost of self-dictionary methods when dealing with large hyperspectral images.

Dev: They propose a data reduction technique based on assuming a linear mixing model with the pure-pixel assumption, aiming to remove pixels that represent mixtures of multiple endmember signatures.

Rosa: The central claim is that this data reduction step preserves the pixels that lie close to the actual endmembers, which is important because it keeps relevant information while cutting down on computation.

Dev: This leads them to propose REDIC, a method where they integrate this data reduction with a self-dictionary approach based on linear programming formulation.

Taro: So it's basically proposing that you can pre-process the image by intelligently discarding interior points in the cone without losing the important boundary points near the endmembers.

Rosa: That’s right, and they analyze how this reduction step behaves theoretically, showing that under certain noise conditions, this process successfully keeps pixels close to those endmembers.

Dev: The motivation is clear: self-dictionary methods are accurate but too slow for large images, so they developed a way to reduce the size of the problem before applying the dictionary method.

Taro: It's about making sure that when you shrink the data space, you don't accidentally throw away the very things you need for your final extraction.

Rosa: And in terms of impact, they show that this approach can substantially reduce computational time without compromising extraction performance on datasets like Urban.

Conclusion: Dev: Looking at the title, "Hyperspectral Image Data Reduction for Endmember Extraction," it tells us exactly what the paper is about: they are focused on using data reduction specifically to find material signatures in hyperspectral images.

Rosa: And the authors, Tomohiko Mizutani and their team, have put forward a practical solution to a big problem in remote sensing where processing time is a major bottleneck.

Dev: The implication for users is that if you're working with very large hyperspectral data, you don't have to wait hours for endmember extraction anymore because of methods like REDIC.

Rosa: Basically, they show that by intelligently reducing the data space first, you can get accurate results in a fraction of the time it used to take using traditional self-dictionary approaches.

Dev: It moves the industry away from just brute-force computation toward more efficient pre-processing steps before running expensive algorithms.

Taro: I'm curious about what this means for future work, because they mentioned that a combinatorial method might be better for accuracy, so are they planning to pursue that path?

Rosa: They suggest exploring a combinatorial method to replace the linear programming formulation within REDIC because it could potentially lead to even higher extraction accuracy than what their current version achieves.

Dev: And looking ahead, this technique has potential for other areas too, like topic modeling and community detection in other types of large-scale data problems.

Taro: So the big picture is that combining intelligent pre-processing with dictionary methods lets us handle these massive datasets efficiently enough to actually do meaningful analysis on them.

Tomohiko Mizutani

Shizuoka University

eess.IV, cs.LG, eess.SP

Submitted: 2025-12-11

Updated: 2026-05-25

Comments: 37 pages, code is available at https://github.com/tomohiko-mizutani/REDIC

Journal ref: SIAM Journal on Imaging Sciences, 19(4), pp. 2410-2445, 2026

DOI: 10.1137/25M1829565

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 79/100

The gist: Endmember extraction from hyperspectral images aims to identify the spectral signatures of materials present in a scene by developing a data reduction technique integrated with a self-dictionary…

Key concepts

Self-dictionary Method
This is an approach used in hyperspectral image analysis where the system learns a dictionary of spectral signatures (endmembers) directly from the data. It helps identify and separate different materials present in an image by finding the best match for each pixel's spectrum against these learned signatures.
Data Reduction (DRS)
This is a crucial step that filters out pixels in a large hyperspectral image that represent mixtures of multiple endmember signatures. The paper shows this removal process is safe because it keeps pixels near the true endmembers, which are the most important information for extraction.
Conical Hull
In this context, the conical hull describes the geometric boundary formed by a set of spectral vectors (pixels) in a high-dimensional space. Removing columns inside this hull speeds up calculations without losing data points that are close to the actual endmember signatures.

Terminology

Summary

Endmember extraction from hyperspectral images aims to identify the spectral signatures of materials present in a scene by developing a data reduction technique integrated with a self-dictionary method that substantially reduces computational time without sacrificing accuracy

Problem Formulation and Motivation

The paper addresses the high computational cost of self-dictionary methods, particularly the LP method, which involves solving an LP relaxation whose size grows quadratically with the number of pixels in a large HSI The research question is whether removing pixels corresponding to mixtures of multiple endmember signatures preserves pixels that lie close to the endmembers Assuming a hyperspectral image follows the linear mixing model with the pure-pixel assumption, this data reduction step is analyzed to show it preserves pixels that lie close to the endmembers The motivation stems from observing that removing columns in the conical hull of an HSI matrix can substantially reduce computational time without discarding columns close to the endmembers

Theoretical Analysis of Data Reduction

The theoretical analysis under the assumption that the HSI matrix follows the linear mixing model with the pure-pixel assumption shows that even after removing columns located inside the conical hull of the HSI matrix, there still exist columns that lie close to the endmembers, provided that noise level is below a certain threshold This result is summarized as Theorem 3.2, which demonstrates this preservation of relevant information The analysis introduces concepts like r-separable matrices and conditioning parameter ρ(W) to quantify the linear independence of endmember columns Theorem 3. p. 7

Where the paper says it

Our analysis employs the conditioning parameter ρ(W) for W ∈ R d×r + , defined by ρ(W) = min ∥x∥1=1 ∥Wx∥1. This parameter measures the degree of linear independence among the columns of W.

2 proves that if noise level epsilon is bounded by O(ρξ/r), the LP method returns an estimated endmember matrix Wout such that the error with respect to the true endmember matrix W, namely ∥W − Wout∥1, is bounded by O(rϵ/ρ)

Development of REDIC Method

The paper proposes a data-reduced self-dictionary method abbreviated as REDIC, which integrates data reduction with a variant of the LP method proposed in [34] The REDIC framework involves several steps:

  1. Applying SVD-based dimensionality reduction to construct a reduced matrix A′

  2. Applying DRS to A′ with parameter p to obtain K ∈ Γ(A′)

  3. Constructing the model H using the reduced matrix and solving it with the RCE algorithm

  4. Performing a postprocessing step using method-C to obtain estimated endmember signatures

To improve performance, REDIC incorporates a data augmentation technique and an averaging technique to mitigate variability introduced by random selection

Experimental Validation and Performance

Numerical experiments on synthetic and real HSI datasets (Jasper Ridge, Samson, Urban) demonstrate the effectiveness of REDIC in terms of both computational time and extraction accuracy The results show that DRS retains only about 0.2%–0.6% of the columns for each dataset Furthermore, when compared to the LP method, REDIC can achieve extraction accuracy comparable to that of the LP method while being substantially faster; for instance, on the Urban dataset, REDIC requires only 16 minutes whereas the LP method takes 8 hours The ablation studies confirm that incorporating DRS is necessary for high performance in cases where only a small number of similar columns are present for some endmembers

Conclusion and Future Directions

The paper concludes by suggesting that a combinatorial method in place of the LP method within REDIC could be used to achieve higher accuracy, potentially outperforming the current version Future research directions include applying this data reduction technique to problems like topic modeling and community detection The work suggests that combining data reduction with self-dictionary methods enables the efficient processing of large-scale datasets within a reasonable time frame

Appendix A. Analysis of Reduced Hyperspectral Image Data

Lemma A.1 states that ρ(Z) = 0 if and only if the n columns of Z are linearly dependent Lemma A. p. 7

Where the paper says it

However, when ρ(W) is too small relative to ϵ, this result does not apply and hence does not guarantee the existence of such columns. For instance, this situation can occur when the columns of W are nearly linearly dependent.

2 establishes that ρ(Z) ≤ 1 when ∥Z∥1 = 1 Corollary A.3 shows that if ρ(Z) > 0 for Z ∈ R m×n, then m ≥ n

Appendix B. Analysis of the Outputs Produced by DR and DRS

Theorem B.1 states that for the output K produced by either DR or DRS for the input matrix A, K ∈ Γ(A) and no set K - k belongs to Γ(A) for any k ∈ K The proof for DRS shows that cone(A) = cone(A(K)) and cone(A)! p. 8

Where the paper says it

, Kt that satisfy cone(A(K1)) = · · · = cone(A(Kt)). In particular, since K1 = [n] and Kt = K, the final output K belongs to Γ(A).

= cone(A(K - k)) for any k ∈ K

Appendix C. Supplementary Experimental Results

Figure 5 summarizes the experimental results for Datasets 1–3 using the L1 distance, showing that the distances decrease as λ increases for all datasets Table 6 shows that for Samson, if we randomly select a few hundred columns from the HSI matrix, a few dozen of them have MRSA values below 4 with respect to each reference endmember signature The results indicate that the lower MRSA scores of the modified REDIC for Samson can be attributed to the presence of relatively many columns similar to each reference endmember signature in the HSI matrix The final conclusion is that incorporating DRS into self-dictionary methods enables the efficient processing of large-scale datasets within a reasonable time frame

Acknowledgments

The author would like to thank the anonymous reviewers for their helpful and insightful comments, which significantly improved the quality of this paper

References

[1] U. M. C. Araujo, B. T. C. Saldanha, R. K. H. Galv ´ ao, T. Yoneyama, H. C., and ˜ V., The successive projections algorithm for variable selection in spectroscopic multicomponent analysis

[2] S. Arora, R., Ge, Y., Halpern, D., Mimno, A., Moitra, D. Sontag, Y. Wu, and M. Zhu, A practical algorithm for topic modeling with provable guarantees

[3] S. Arora, R., Ge, R., Kannan and A. Moitra, Computing a nonnegative matrix factorization – Provably

[4] S. Arora, R., Ge, and A. Moitra, Learning topic models – Going beyond SVD

[5] P. Awasthi, B., Kalantari and Y. p. 9

Where the paper says it

Such a vector γ always exists when u > 0 and A > 0; hence, this assumption is likely to hold in Step 2 of DR when the algorithm is applied to HSI matrices. Kalantari [26] developed a geometric algorithm called the Triangle Algorithm for testing whether u ∈ conv(A) and analyzed its computational complexity.

Zhang, Robust vertex enumeration for convex hulls in high dimensions

[6] J. M. Bioucas-Dias, A., Plaza, N., Dobigeon, M., Parente, Q., Du, P., Gader and J. Chanussot, Hyperspectral unmixing overview: Geometrical, statistical, and sparse regression-based approaches

[7] V. Bittorf, B. p. 0

Where the paper says it

Endmembers are identified using the optimal solution of the relaxed problem. Although various self-dictionary methods have been developed in the literature [7, 11, 12, 20, 36], this paper mainly focuses on the method proposed by Bittorf et al.

Recht, C. Re, and J. A. Tropp, Factoring nonnegative matrices with linear programs

[8] J. W., Boardman, F. A., Kruse and R. O., Mapping target signatures via partial unmixing of aviris data

[9] K. L., Clarkson, More output-sensitive geometric algorithms

[10] J. H. Dula, R. V. Helgason, and N. Venugopal ´, An algorithm for identifying the frame of a pointed finite conical hull

[11] E. Elhamifar, G. p. 6

Where the paper says it

The work by Esser et al. [12] can be viewed as a self-dictionary approach to the endmember extraction problem, whereas the work by Elhamifar et al.

, Sapiro, and R., See all by looking at a few: Sparse modeling for finding representative objects

[12] E. Esser, M., Moller, S., Osher, G., Sapiro, and J. Xin ¨, A convex model for nonnegative matrix factorization and dimensionality reduction on physical space

[13] X. Fu, K., Huang, N. D. Sidiropoulos and W.-K. Ma, Nonnegative matrix factorization for signal and data analytics: Identifiability, algorithms, and applications

[14] K. Fukuda, B., Gartner and M. Szedl ¨ ak´, Combinatorial redundancy detection

[15] N. Gillis, Robustness analysis of Hottopixx, a linear programming model for factoring nonnegative matrices

[16] N. Gillis, Successive nonnegative projection algorithm for robust nonnegative blind source separation

[17] N. Gillis, Separable simplex-structured matrix factorization: Robustness of combinatorial approaches

[18] N. Gillis, Nonnegative Matrix Factorization

[19] N. Gillis and R. p. 2

Where the paper says it

In the works of Esser et al. [12] and Gillis and Luce [20], data reduction techniques based on clustering were employed within self-dictionary methods.

p. 2
Where the paper says it

In the works of Esser et al. [12] and Gillis and Luce [20], data reduction techniques based on clustering were employed within self-dictionary methods.

Luce, Robust near-separable nonnegative matrix factorization using linear optimization

[20] N. Gillis and R. Luce, A fast gradient method for nonnegative sparse regression with self-dictionary

[21] N. Gillis and S. p. 2

Where the paper says it

In the works of Esser et al. [12] and Gillis and Luce [20], data reduction techniques based on clustering were employed within self-dictionary methods.

p. 2
Where the paper says it

In the works of Esser et al. [12] and Gillis and Luce [20], data reduction techniques based on clustering were employed within self-dictionary methods.

A. Vavasis, Fast and robust recursive algorithms for separable nonnegative matrix factorization

[22] N. Gillis and S. A. Vavasis, Semidefinite programming based preconditioning for more robust nearseparable nonnegative matrix factorization

[23] K. Huang and X. Fu, Detecting overlapping and correlated communities without pure nodes: Identifiability and algorithm

[24] S. Ibrahim, X. Fu, N. Kargas, and K. Huang, Crowdsourcing via pairwise co-occurrences: Identifiability and algorithms

[25] A. Ifarraguerri and C.-I. Chang, Multispectral and hyperspectral image analysis with convex cones

[26] B. Kalantari, A characterization theorem and an algorithm for a convex hull problem

[27] A. Kumar, V., Sindhwani and P. Kambadur, Fast conical hull algorithms for near-separable nonnegative matrix factorization

[28] V. Leplat, N., Gillis, and A. p. 2

Where the paper says it

In the works of Esser et al. [12] and Gillis and Luce [20], data reduction techniques based on clustering were employed within self-dictionary methods.

M. Ang, Blind audio source separation with minimum-volume betadivergence nmf

[29] W.-K. Ma, J. M., Bioucas-Dias, T.-H., Chan, N., Gillis, P. p. 2

Where the paper says it

In the works of Esser et al. [12] and Gillis and Luce [20], data reduction techniques based on clustering were employed within self-dictionary methods.

, Gader, A. J., Plaza, A. Ambikapathi and C.-Y. Chi, A signal processing perspective on hyperspectral unmixing

[30] T. Mizutani, Ellipsoidal rounding for nonnegative matrix factorization under noisy separability

[31] T. Mizutani, Robustness analysis of preconditioned successive projection algorithm for general form of separable nmf problem

[32] T. Mizutani, Convex programming based spectral clustering

[33] T. Mizutani, Refinement of Hottopixx method for nonnegative matrix factorization under noisy separability

[34] T. Mizutani, Endmember extraction from hyperspectral images using self-dictionary approach with linear programming

[35] J. M. P. Nascimento and J. M., Bioucas-Dias, Vertex component analysis: A fast algorithm to unmix hyperspectral data

[36] T. Nguyen, X., Fu, and R. Wu, Memory-efficient convex optimization for self-dictionary separable nonnegative matrix factorization: A Frank-Wolfe approach

[37] C. Soussen, S., Miron, F., Caland, D., Brie, P., Billard and C. Mustin, Complexity reduction by convex cone detection for unmixing hyperspectral images of bacterial biosensors

[38] M. E. Winter, N-FINDR: an algorithm for fast autonomous spectral end-member determination in hyperspectral data

[39] F. Zhu, Hyperspectral unmixing: Ground truth labeling, datasets, benchmark performances and survey

Table 1

Table 1 summarizes the specifications of the HSI datasets

Table 2

Table 2 summarizes the specifications of the DRS output K for each dataset

Figure 2

Figure 2 shows the RGB images of the Jasper Ridge (left), Samson (center), and Urban (right) datasets

Table 3

Table 3 summarizes the experimental results for Datasets 1–3 using the MRSA distance

Figure 4

Figure 4 summarizes the experimental results for the Urban dataset Panels (a) and (b) show Datasets 1 and 2, while Panels (c) and (d) show Dataset 3 with λ from 0 to 2500 (step 500) (d) Dataset 3 with λ from 0 to 12500 Each panel shows the MRSA distance of the DRS output K ±add, averaged over 50 realizations of Kadd

Table 4

Table 4 summarizes the MRSA scores of these methods for the three datasets

Table 5

Table 5 summarizes the results for Jasper Ridge, Samson, and Urban datasets for different values of λ and τ Table 6 shows that for Samson, if we randomly select a few hundred columns from the HSI matrix, a few dozen of them have MRSA values below 4 with respect to each reference endmember signature

Table 6

Table 6 summarizes the proportion of columns whose MRSA values are at most θ for each reference endmember signature in three datasets The results show that, for Samson, if we randomly select a few hundred columns from the HSI matrix, a few dozen of them have MRSA values below 4 with respect to each reference endmember signature

Figure 4 shows the elapsed time (Panel (a)), the mean number of columns in the matrices processed in Step 2 of DR during iterations (Panel (b)), and the number of feasibility tests (Panel (c)) for DRS at each value of p and for DR Panel (a) shows that DRS is faster than DR Specifically, DRS takes less than 6 minutes for all values of p, and in particular, it takes 2 minutes when p = 30, whereas DR takes 22 minutes The elapsed time of DRS gradually decreases as p increases Panels (b) and (c) show that DRS substantially reduces the size of problem (3.2) in the feasibility tests, while the number of feasibility tests performed by DRS is nearly the same as that of DR These results indicate that, with an appropriate choice of p, DRS reduces the problem sizes arising in the feasibility tests of DR while maintaining a similar number of feasibility tests to DR, thereby reducing the elapsed time

Figure 4 shows the MRSA distance of the DRS output K ±add, averaged over 50 realizations of Kadd The figures show that the distances decrease as λ increases for all datasets In particular, even when λ is set to 1% of n, the distances are substantially reduced compared with those at λ = 0

Figure 3

Figure 3 shows the experimental results for Datasets 1–3 using the MRSA distance The baseline corresponds to the LP method, and REDIC with λ > 0 achieves a lower MRSA score than the LP method for all datasets For Urban, REDIC with (λ, τ) = (10000, 5) achieves nearly the same MRSA scores as the LP method, with a speed-up of 29 times The results suggest that incorporating DRS into self-dictionary methods enables the efficient processing of large-scale datasets within a reasonable time frame

Figure 5

Figure 5 shows that, for Samson, if we randomly select a few hundred columns from the HSI matrix, a few dozen of them have MRSA values below 4 with respect to each reference endmember signature The lower MRSA scores of the modified REDIC for Samson can be attributed to the presence of relatively many columns similar to each reference endmember signature in the HSI matrix

The paper concludes by suggesting that a combinatorial method in place of the LP method within REDIC could be used to achieve higher accuracy, potentially outperforming the current version Future research directions include applying this data reduction technique to problems like topic modeling and community detection The work suggests that combining data reduction with self-dictionary methods enables the efficient processing of large-scale datasets within a reasonable time frame

**Appendix C.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Data reduction for hyperspectral images can substantially reduce computational time for endmember extraction by removing pixels corresponding to mixtures of multiple endmember signatures, as demonstrated by Algorithm 3.2 DRS which is faster than DR on real HSI datasets.

  2. The proposed REDIC method integrates data reduction with an LP formulation and incorporates a data augmentation process and averaging technique to improve the accuracy of extracted endmembers, allowing it to achieve extraction accuracy comparable to the LP method while being substantially faster, such as 29 times faster on the Urban dataset.

  3. The REDIC system can be configured to identify a set K with the smallest number of elements in Gamma(A) by using a splitting technique (DRS), which ensures that the output K is a set with the smallest number of elements in Gamma(A).

  4. The system can perform endmember extraction on large-scale hyperspectral images efficiently by first applying top-r truncated singular value decomposition to construct a reduced matrix A', which simplifies the subsequent model solving step.

  5. The system can be optimized for high accuracy by incorporating a postprocessing step in REDIC that chooses representative elements from clusters based on their centroids, referred to as method-C, which is shown to improve endmember extraction performance.

Abstract

Endmember extraction from hyperspectral images aims to identify the spectral signatures of materials present in a scene. Recent studies have shown that self-dictionary methods can achieve high extraction accuracy; however, their high computational cost limits their applicability to large-scale hyperspectral images. Although several approaches have been proposed to mitigate this issue, it remains a major challenge. Motivated by this situation, this paper pursues a data reduction approach. Assuming that a hyperspectral image follows the linear mixing model with the pure-pixel assumption, we develop a data reduction technique to remove pixels corresponding to mixtures of multiple endmember signatures. We analyze the theoretical properties of this reduction step and show that it preserves pixels that lie close to the endmembers. Building on this result, we propose a data-reduced self-dictionary method that integrates the data reduction with a self-dictionary method based on a linear programming formulation. Numerical experiments demonstrate that the proposed method can substantially reduce the computational time of the original self-dictionary method without sacrificing endmember extraction accuracy.

Sources

Related papers