Quantum Incremental Learning with Mixed State Prototypes

Quantum Incremental Learning with Mixed State Prototypes

Abstract

Incremental learning models are required to learn new classes sequentially without catastrophic forgetting, while operating under parameter and memory constraints. In the Noisy Intermediate-Scale Quantum (NISQ) era, although quantum neural networks offer advantages in feature mapping, hardware limitations restrict circuit width. Furthermore, traditional quantum classifiers are constrained by the number of orthogonal basis states, limiting their capacity to accommodate a continually growing number of categories. Thus, we introduce a novel quantum incremental learning framework based on trainable mixed-state prototypes. Its original design incorporates new classes by adding class prototypes rather than increasing the circuit width of the shared quantum backbone. The use of mixed-state prototypes is another key contribution, since they have representation capabilities to represent information than a single pure-state prototype. And the decomposable mixed-state calculation provides lower production costs and a convenient Hilbert–Schmidt (HS) distance metric for classification. Simulation results show that our model achieves high-dimensional feature concentration using a minimal number of qubits, while demonstrating lower computational complexity and robust representation in incremental learning tasks compared with classical baselines.

Introduction

Deep learning has achieved remarkable success under the assumption that all training data is available at once. However, real-world data typically arrives in a continuous, ever-changing stream. Iteratively retraining a model from scratch to integrate new classes is computationally exorbitant and inefficient for resource-constrained applications. Consequently, incremental learning has emerged to continuously update models with new knowledge while minimizing storage and computing overhead.

In incremental learning, the fundamental challenge is catastrophic forgetting, where neural networks erase previously learned representations upon adapting to new tasks. Early works introduced memory replay to explicitly rehearse a small buffer of historical exemplars, alongside parameter regularization and knowledge distillation to constrain critical weight updates. As storing raw data is often restricted by memory limits or privacy concerns, research subsequently explored exemplar-free statistical prototype matching, extracting and freezing abstract representations to avoid iterative retraining. More recently, dynamic networks have been proposed to mitigate forgetting by expanding architectural branches. Despite these diverse efforts, classical incremental learning still confronts the bottleneck of computational overhead and residual representational drift, which limits their real-world scalability.

Harnessing quantum mechanics, quantum computing introduces a fundamentally distinct paradigm for information processing, catalyzing the rapid exploration of quantum machine learning. In the NISQ era, hybrid architectures based on variational quantum circuits (VQC) have shown immense promise. By projecting classical data into an exponentially expansive Hilbert space, VQCs facilitate the efficient manipulation of highly non-linear features. Thus, quantum models achieve competitive accuracy with significantly reduced parameters compared to classical networks. Moreover, quantum entanglement captures non-local correlations, providing a physical foundation for enhancing model expressiveness.

Despite these theoretical merits, classical incremental learning strategies cannot be directly quantized, as the structural differences between quantum and classical networks. In classical deep learning, accommodating new classes is often as straightforward as expanding the final linear layer. However, a large class of commonly used quantum classifiers constructs class scores from a fixed set of basis-state probabilities or expectation values. As the label space grows, these classifiers may require an expanded measurement or readout rule, additional observables, or more class-dependent parameters. In basis-state encoding schemes, accommodating additional classes may also require additional qubits. Crucially, unlike modular classical expansion, appending a qubit fundamentally alters the global quantum state and the underlying Hilbert space. This structural shift disrupts previously learned entanglement correlations, exacerbating catastrophic forgetting. Furthermore, blindly increasing the circuit width or depth inevitably triggers the barren plateau phenomenon, rendering the quantum network untrainable.

To circumvent this structural expansion dilemma, quantum metric learning emerges as a promising paradigm. By classifying instances based on state distances rather than relying on explicit parameterized classification heads, it naturally bypasses the need for circuit widening. Its effectiveness heavily depends on exploring advanced quantum information representations. In quantum feature mapping, pure states encode individual inputs into single unit state vectors in the Hilbert space. Consequently, they lack the statistical capacity to represent a diverse distribution compared to mixed states. By formulating information as a convex combination of orthogonal states, mixed states encapsulate data randomness and complex intra-class variance. Compared to classical information representations, this quantum method exponentially broadens the representation dimensionality. Furthermore, it provides physically grounded distance metrics, such as the HS distance, trace distance, and Bures distance. Ultimately, these comprehensive advantages help to solve the problems in incremental learning tasks.

Motivated by the representational power of mixed states and the physical constraints of quantum circuits, this paper proposes a novel quantum incremental learning framework based on mixed-state prototypes. To transcend the capacity bottleneck of traditional quantum classifiers, our design shifts from explicit softmax regression classification to an implicit, distance-based prototype matching mechanism. Moreover, recognizing the difficulty of dynamically altering quantum network, we design a hybrid architecture: we append a lightweight, expandable classical module (i.e., an MLP) to manage the incremental class adjustments without disrupting the underlying quantum states.

The main contributions of this work are summarized as follows:

(1) A quantum prototype framework with an expandable module that avoids the capacity constraint associated with orthogonal basis states.

(2) Mixed-state prototypes via dynamic weighting of pure states avoid redundant qubits and provide PCA-like noise-filtering properties.

(3) Experiments on static and incremental classification tasks demonstrate high-dimensional semantic concentration, exhibiting parameter advantages and noise robustness.

Preliminaries

Incremental learning strategies: Incremental learning requires a model to sequentially learn new classes from a continuous data stream without catastrophically forgetting previously acquired knowledge. Current classical incremental learning frameworks generally rely on three fundamental strategies:

1. Exemplar replay strategy: Replay mechanisms mitigate forgetting by preserving a bounded memory of old data. A prominent example is iCaRL, which utilizes a herding algorithm to select and store a small set of highly representative samples, for each observed class.

2. Regularization techniques: Instead of directly reusing raw data, regularization methods impose mathematical constraints on the network’s optimization trajectory. Parameter-based regularization approaches like EWC evaluate the importance of individual network weights for past tasks and penalize drastic deviations in these critical parameters. Furthermore, knowledge distillation techniques such as LwF mitigate the representational drift of the network.

3. Prototype-based classification: Due to the data imbalance between old and new classes, traditional classifiers suffer from task-recency bias in incremental learning. To address this, prototype-based inference relies on distance metrics within the embedding space. Methods like the nearest-mean-of-exemplars classifier in iCaRL or centroid-based classifiers in FeTrIL and FeCAM represent each class by a global prototype. An input instance is classified by assigning it to the nearest class prototype. This decoupling of feature extraction from explicit classification effectively alleviates the inherent prediction bias and serves as a theoretical cornerstone for our proposed quantum prototype design.

Quantum mixed state compiling: Quantum state compilation aims to learn a parameterized quantum circuit that can prepare a reliable approximation of target state. While early works primarily focused on pure states, compiling mixed states provides a more generalized framework for handling classical probabilities, noise, and complex data distributions in the NISQ era.

1. Density matrix and mixed state representation: In quantum mechanics, a closed quantum system with complete information is described by a pure state, denoted by a state vector ψ⟩ in a Hilbert space. However, in practical open quantum systems, particularly within the NISQ era, a principal system inevitably interacts with its environment and undergoes decoherence. Tracing out the environment produces a reduced mixed state. A mixed state is described with a density matrix ρ. It is a positive-semidefinite Hermitian operator with a unit trace Tr(ρ) = 1. According to the spectral decomposition theorem, any mixed state ρ can be expressed as a probabilistic ensemble of orthogonal pure states.

2. Convex Combination of Pure States ansatz: State-purification ansatzes prepare a larger pure state and trace out ancillary registers, which increases the required circuit width. The convex combination of pure states (CCPS) ansatz avoids this overhead by representing a rank-R mixed state as a probabilistic mixture of transformed basis states. Under HS-distance optimization, a rank-R CCPS state provides a variational low-rank approximation of the target mixed state, yielding a PCA-like procedure that retains its leading components. It controls the prototype rank without introducing ancillary qubits and can suppress low-contribution components. These properties make CCPS suitable for constructing the class prototypes in our framework.

3. Distance measurement via HS distance and SWAP test: The HS distance is used to measure the distance between two quantum states, and also to describe the quantum distance in quantum geometry. Using the Frobenius norm, the squared HS distance is L = ∥ρ − σ∥2F, which can be decomposed into state purities and their quantum overlap. The HS distance can be decomposed into classical probability evaluations and a series of quantum SWAP tests between mixed and pure states. The SWAP test requires one 0⟩ initialized ancillary qubit, utilizing a controlled-swap gate sandwiched between two Hadamard gates. By measuring the ancilla, the probability of observing the state 0⟩, denoted as P(0), is represented as P(0) = 1/2 + 1/2 Tr[ρψi(θ)⟩⟨ψi(θ)]. Thus, the required quantum state overlap can be directly expressed from the measurement statistic as Fi(θ) = 2P(0) − 1. This decomposition is the main reason for using the HS distance in our framework. In dense-matrix simulation, evaluating the HS distance between two d × d states requires O(d2) operations. The trace distance, fidelity, Bures distance, and quantum Jensen–Shannon divergence generally require O(d3) operations because they involve spectral decompositions or matrix square roots. Therefore, the HS distance is more suitable for repeated prototype fitting and comparison.

Mixed-State Based Quantum Prototype Classifier

Overview of the quantum prototype classifier: The core architecture of our proposed framework is a mixed-state quantum prototype classifier. A large class of commonly used quantum classifiers constructs class scores from a fixed set of measured basis-state probabilities or expectation values. Their measurement or readout structures may need to be expanded as the label space grows. Our design instead performs distance-based prototype matching within a shared density-matrix space. Formally, given an input instance x ∈ X, the quantum feature extraction backbone acts as a parameterized mapping function Fq(·; Θbb). It encodes and processes x to output an intermediate quantum mixed state, represented by a density matrix ρ(x) ∈ Cd×d, where d = 24 = 16 for our 4-qubit subsystem. In parallel we construct an independent mixed-state quantum prototype circuit for each category c ∈ Y. This circuit generates a class prototype ρc residing in the identical d × d Hilbert space. The classification prediction ŷ is then determined by identifying the class prototype that yields the minimum HS distance to the query instance. This design decouples representation learning from prototype classification. The backbone learns characteristic information, while the prototype circuit learns the characteristic distribution of specific categories within the same density matrix space.

Quantum feature extraction backbone: The quantum feature extraction backbone serves as a hybrid classical-quantum bridge, mapping high-dimensional classical inputs into compact quantum density matrices. This backbone mainly consists of a classical pre-processing module and a subsequent quantum evolution network. Initially, a classical neural network is employed to extract high-level semantic features from the raw input data. To bridge the dimensional gap between classical feature spaces and the quantum Hilbert space, the extracted features undergo a dimension reduction and normalization process. The resulting compact vector is then embedded into the initial pure state of the quantum system using amplitude encoding, translating classical data into probability amplitudes. The embedded pure quantum state is processed by the quantum backbone, which typically employs a sequence of parameterized unitary operations interleaved with dimensionality reduction mechanisms, such as quantum pooling. The primary objective of these reduction operations is to achieve semantic concentration, extract robust features, and filter out uncorrelated noise. However, operations that discard redundant degrees of freedom, such as partial tracing, inevitably induce information loss regarding the global pure state. That means the retained subsystem is taken into a quantum mixed state. Therefore, the emergence of a mixed-state representation is a physical consequence of performing feature abstraction and denoising within a quantum architecture. The resulting density matrix ρ(x) intrinsically encapsulates the structural uncertainty and hierarchical semantic information of the input, serving as a highly robust, noise-resilient foundational input for the subsequent prototype-based classification.

Mixed quantum prototype preparation circuit: For each class, the model maintains an independent parameterized prototype circuit. The purpose of this circuit is to generate a mixed quantum state in the identical density-matrix space as the backbone’s quantum feature output. Following the CCPS architecture, the prototype for class c is constructed as a probabilistic mixture of transformed basis states. By selecting a subset of k computational basis states i⟩, the prototype density matrix ρc is formulated as a convex mixture of transformed basis states. This convex mixture allows a single class prototype to encapsulate complex intra-class variations, thereby circumventing the representational bottleneck of forcing an entire diverse class into a rigid, singular pure state. To ensure high expressivity, the circuit Uc is constructed using a hierarchical multi-layer structure. Within each layer, generic two-qubit unitary blocks are applied in an alternating entanglement topology. This configuration enables the prototype circuit to span a highly expressive transformation space, maintaining both representation capacity and parameter efficiency. A challenge in optimizing the CCPS ansatz lies in the constraints of the mixture weights wc,i, which must satisfy th…

Based on this paper, here are the specific improvements you can make to AI systems:

• Improvement: Replace single-point (pure-state) class prototypes with mixed-state density matrix prototypes that represent each class as a probabilistic convex combination of multiple transformed basis states.

• What the improved system can do: Capture complex intra-class variations and multi-modal distributions that single prototypes miss. The system can model uncertainty and statistical spread within each class, leading to more robust classification, especially when class distributions overlap or have high variance.

• Improvement: Introduce a tunable rank parameter (K) in the prototype representation that controls how many basis components are retained, analogous to principal component analysis.

• What the improved system can do: Automatically filter out low-contribution, noisy components in the feature space while preserving dominant structural patterns. This provides built-in denoising and regularization, improving generalization on noisy or high-dimensional data without explicit noise modeling.

• Improvement: Replace explicit softmax classification heads with a distance-based matching mechanism using the Hilbert-Schmidt distance between density matrices.

• What the improved system can do: Avoid the capacity bottleneck of fixed-size output layers. The system can handle an arbitrarily growing number of classes without modifying the network architecture, making it naturally suited for open-set or incremental scenarios where new categories appear over time.

• Improvement: Train the feature extraction backbone first (with an auxiliary head), then freeze it and independently fit class prototypes without altering the shared representation.

• What the improved system can do: Prevent catastrophic forgetting during incremental learning by ensuring the feature space remains stable. The system can add new classes by only fitting new prototypes, avoiding the need to retrain the entire network, which reduces computational cost and memory overhead.

• Improvement: Select replay exemplars based on their proximity (Hilbert-Schmidt distance) to the learned class prototypes, rather than random or herding-based selection.

• What the improved system can do: Store the most representative samples that best align with the class distribution, maximizing the information retained per memory unit. This improves old-class preservation under tight memory budgets and reduces the risk of biased sampling.

• Improvement: Use a compact quantum backbone for feature extraction and a lightweight, expandable classical module (MLP) for handling class count changes.

• What the improved system can do: Scale to larger label spaces without increasing quantum circuit width, which is critical for NISQ-era hardware constraints. The system achieves high-dimensional feature concentration with minimal qubits, making it practical for resource-limited quantum devices.

• Improvement: Parameterize the convex combination weights of the mixed-state prototype using a softmax function over unconstrained logits.

• What the improved system can do: Ensure valid probability simplex constraints are automatically satisfied during training, leading to smoother and more stable optimization. The system avoids manual constraint handling and converges more reliably.

• Improvement: Decompose the Hilbert-Schmidt distance into pre-computable purities and a series of SWAP tests, avoiding expensive spectral decompositions.

• What the improved system can do: Achieve O(d2) complexity for distance evaluation instead of O(d3), enabling faster training and inference. The system can scale to larger state dimensions and more classes while maintaining computational efficiency.

• Improvement: Accommodate new classes by adding class prototypes rather than expanding the quantum circuit width or output layer.

• What the improved system can do: Avoid the catastrophic forgetting caused by structural changes to the network. The system maintains previously learned entanglement correlations and representations, preserving old knowledge while seamlessly integrating new categories.

• Improvement: Use quantum pooling layers that partially trace out qubits, naturally inducing mixed-state representations that capture structural uncertainty.

• What the improved system can do: Filter out uncorrelated noise and redundant information during feature extraction, producing more robust representations that are less sensitive to input perturbations. The system inherently handles noisy or incomplete data better than pure-state approaches.