Optimizing Entanglement Distillation Policies via Markov Decision Process Formulation
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Quantum Radio. Generated commentary on the latest quantum physics and condensed matter papers.
Kai: We're looking at the paper: "Optimizing Entanglement Distillation Policies via Markov Decision Process Formulation".
Mira: Here's the gist of the paper. Optimizing Entanglement Distillation Policies via Markov Decision Process Formulation is a study that formulates entanglement distillation as a Markov decision process to obtain optimal deterministic policies that minimize the expected time required to achieve a target fidelity The optimal policy we seek is a map from the space of system configurations to the space of possible actions that minimizes the expected time to obtain a single state with fidelity at least equal to target threshold fT.
Kai: Problem Formulation The problem is modeled as an MDP defined by the 4-tuple (S, A, P, R) where S is the set of all possible states s ∈ S that the agent can observe, A is the set of all possible actions a ∈ A that the agent can take, P(s's, a) is the transition probability of system (referred to as the “environment” in MDP literature) moving from state s ∈ S to s'∈S under action a, and R is the values for the reward function r(s, a) ∈ R.
Mira: The agent’s objective is to learn a policy π(as), i.e.
Kai: mapping from state to action, that maximizes the expected cumulative reward.
Title and authors: Kai: So let's talk about who wrote this. The paper is "Optimizing Entanglement Distillation Policies via Markov Decision Process Formulation," and it’s by Jigyen Bhavsar, Rajni Bala, and Siddhartha Santra from the Indian Institute of Technology Bombay.
Mira: I think it’s interesting because they aren't just describing a process; they are framing it as a decision problem for an AI to solve.
Kai: Exactly. They take entanglement distillation and turn it into this Markov decision process structure, which is really powerful because it lets you treat the entire sequence of operations as a series of choices under uncertainty.
Lev: From an engineering standpoint, that sounds like they are trying to find a robust control strategy for a system where you're dealing with probabilistic outcomes and time delays.
Kai: And that’s right, because entanglement generation and distillation are both probabilistic processes, and they take different amounts of time based on what you choose to do next.
The paper's summary: Mira: The summary boils down to them defining the state space as all possible fidelity levels for each quantum memory pair—basically a vector of fidelities—and then identifying an absorbing state where every link has a fidelity of at least fT.
Kai: And the actions they define are two main things: entanglement generation, which is probabilistic with success probability p, and entanglement distillation, which involves two-to-one operations across multiple pairs <ref:2606.14908#pg3>.
Lev: So when you look at the dynamics described on page two of this paper, it’s clear that both these operations are probabilistic because their success depends on the input fidelities <ref:2606.14908#pg2>.
Mira: And they show how different protocols for distillation, like a two-to-one operation between states of fidelity f1 and f2, result in a new fidelity fd given by the formula involving those inputs.
Kai: So they’re not just saying you can distill; they are giving you the math on what happens to the fidelity when you perform that specific type of distillation, which is crucial for building a policy.
The paper's improvements: Lev: The improvement they highlight here is using this MDP framework with value iteration to obtain these optimal deterministic policies that minimize the expected waiting time T pi.
Kai: So what does that mean for us? It means instead of guessing or following a simple heuristic, you get a policy that minimizes the average time it takes to hit your target fidelity fT across all possible system configurations.
Mira: They also point out how the optimal policy depends heavily on system parameters like the generation probability p and the initial fidelity f0, as well as how many memory pairs m you have.
Lev: I see that dependence on p is important; they found that T pi increases when the entanglement generation probability p is small because you need more attempts to establish those initial links.
Kai: And if you increase m, the number of memory pairs, it actually reduces that waiting time because it gives you more options for distillation configurations to work with.
Conclusion: Mira: So to wrap up, the core finding of "Optimizing Entanglement Distillation Policies via Markov Decision Process Formulation" is that the optimal deterministic policy is a function of the current system configuration and it minimizes the expected waiting time to reach fidelity fT.
Lev: And what's really impactful is that they show this optimal policy can substantially reduce the expected waiting time compared to those baseline strategies you see in other literature, like greedy or pumping policies.
Kai: It shows that selecting distillation operations based on the full memory configuration is better than just using simple heuristics, and the magnitude of improvement depends on all those system parameters like m and p.
Mira: They also flag a limitation: they are analyzing this under assumptions where entanglement generation has a fixed probability p, which might not perfectly match real-world hardware imperfections.
Lev: And another thing they mention is that the relative magnitudes of the time required for entanglement generation versus distillation can really determine the policy found optimal under their value iteration algorithm.
Kai: So it’s a solid mathematical framework to design smarter, state-dependent control over these noisy quantum systems, and we'll be looking at how this plays out in our actual experimental setups next.
Jigyen Bhavsar, Rajni Bala, Siddhartha Santra
Indian Institute of Technology Bombay
quant-ph
Submitted: 2026-06-12
Updated: 2026-07-03
Journal ref: Quantum Sci. Technol. 11 (2026) 0450b2
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 75/100
The gist: Optimizing Entanglement Distillation Policies via Markov Decision Process Formulation is a study that formulates entanglement distillation as a Markov decision process to obtain optimal deterministic…
Key concepts
- Markov Decision Process (MDP)
- An MDP is a mathematical framework used to model decision-making in situations where outcomes depend on the current state and actions. Here, it helps determine the best sequence of operations (actions) to minimize waiting time while navigating the stochastic process of entanglement generation and distillation.
- State Space (S)
- The state space represents all possible configurations of quantum memories, defined by the fidelity levels of each stored entangled link. This space includes an absorbing 'target' state where the required fidelity threshold is met, allowing the system to track its progress toward success.
- Action Space (A)
- Actions define what the agent can do at any given time step. These include two main types: entanglement generation attempts and 2-to-1 entanglement distillation operations. The optimal policy chooses the best combination of these actions simultaneously to advance toward the target fidelity efficiently.
- Expected Waiting Time (Tπ)
- This is the primary metric minimized by the optimal policy. It represents the average number of time steps required for a system, following a specific decision rule (policy π), to achieve an entanglement fidelity of at least the target threshold fT.
Terminology
Summary
Optimizing Entanglement Distillation Policies via Markov Decision Process Formulation is a study that formulates entanglement distillation as a Markov decision process to obtain optimal deterministic policies that minimize the expected time required to achieve a target fidelity The optimal policy we seek is a map from the space of system configurations to the space of possible actions that minimizes the expected time to obtain a single state with fidelity at least equal to target threshold fT. p. 1 , different distillation policies) yield different
probabilities—and hence different expected times—for
reaching a desired target fidelity fT . The optimal policy
we seek is a map from the space of system configurations
to the space of possible actions that minimizes the expected time to obtain a single state with fidelity at least
equal to fT .Where the paper says it
Problem Formulation
The problem is modeled as an MDP defined by the 4-tuple (S, A, P, R) where S is the set of all possible states s ∈ S that the agent can observe, A is the set of all possible actions a ∈ A that the agent can take, P(s's, a) is the transition probability of system (referred to as the “environment” in MDP literature) moving from state s ∈ S to s'∈S under action a, and R is the values for the reward function r(s, a) ∈ R. The agent’s objective is to learn a policy π(as), i. p. 2 The system
then transitions to a new state s
′
, according to the transition probability P(s
′
|s, a) and returns a reward r(s, a)
to the agent, as shown in Fig. The agent’s objective
is to learn a policy π(a|s), i. The agent’s objective
is to learn a policy π(a|s), i. mapping from state to
action, that maximizes the expected cumulative reward.Where the paper says it
Where the paper says it
System Dynamics and State Space
The state of the system is represented as ⃗st = ⃗f st = f1, f2, …, fm ∈ S (6) where m is the total number of quantum memories at each node and element fj of the state ⃗st is the random variable which gives the fidelity of entangled link stored in j th memory from the allowed fidelity level. The state space S is given by S ∈ (0, f0, …, fT) m, where each component specifies the fidelity of a corresponding link, taking values from this set of allowed fidelity levels. p. 3 Formally, the state space is given by
S ∈ {0, f0, . , FT }
m, where each component specifies the
fidelity of a corresponding link, taking values from this
set of allowed fidelity levels. An
entanglement distillation policy, denoted by π, is thus a
rule that specifies which action to take based on the current configuration of the system. The problem can thus
be stated as follows: given the current set of stored entangled pairs and their fidelities, which action should be
performed next, in order to minimize the expected time
required to obtain at least one entangled pair with fidelity
greater than or equal to target threshold fT ? Any fidelity greater than or
equal to the target fidelity, i. , f ≥ fT , is represented by
a single absorbing state FT in the state space of MDP.Where the paper says it
Where the paper says it
Where the paper says it
Action Space and Reward Structure
The available actions depend on the state, can be broadly classified into two categories: (i) Entanglement generation (Ag): This action specifies, for each elementary link, whether an entanglement generation attempt is made, and (ii) Entanglement distillation (Ad): This action allows simultaneous 2-to-1 distillation across multiple pairs of parallel links within single time step. The full action at each time step is represented by a tuple a = (M, ag) specifying distillation matching and generation vector on the remaining free links. p. 4 The
set of all valid matchings, including the empty matching
M = ∅ (no distillation), is denoted M. The full action at each time step is represented by a
tuple a = (M, ag) specifying distillation matching and
generation vector on the remaining free links. In the units of one-way classical communication delay(l/c), the reward function evaluates to
r = −1 for entanglement generation and r = −2 for
entanglement distillation. When the terminal state is
reached, the reward function evaluates to r = 0.Where the paper says it
Where the paper says it
Performance Analysis and Policy Comparison
The optimal policy minimizes the expected waiting time, Tπ = E[tπ] (4) where the expectation is taken over the stochastic outcomes of entanglement generation and distillation conditioned on the decisions prescribed by policy π. Numerical results show that Topt increases when the entanglement generation probability p is small, since a larger number of attempts are required to successfully establish entangled links. p. 6 In this section, we numerically analyze the dependence
of the expected waiting time under an optimal policy,
denoted by Topt, on the system parameters (m, p, f0, fT )
and compare it with baseline policies, as defined in
Sec. Our results show that Topt increases when the
entanglement generation probability p is small, since a
larger number of attempts are required to successfully
establish entangled links. However, increasing
the number of quantum memories m reduces the Topt,
as it provides more distillation configurations to achieve
the target fidelity efficiently. We further find that the
advantage of the optimal policy over baseline strategies
depends nontrivially on the system parameters, with
its expected waiting time being comparable to that of
pumping at intermediate f0 and nested policy at low
and high f0 for a fixed fidelity gap ∆f, as discussed in
detail throughout this section. However, increasing
the number of quantum memories m reduces the Topt,
as it provides more distillation configurations to achieve
the target fidelity efficiently. We further find that the
advantage of the optimal policy over baseline strategies
depends nontrivially on the system parameters, with
its expected waiting time being comparable to that of
pumping at intermediate f0 and nested policy at low
and high f0 for a fixed fidelity gap ∆f, as discussed in
detail throughout this section.Where the paper says it
Where the paper says it
Where the paper says it
Baseline Policy Benchmarking
Several heuristic entanglement distillation policies are used as benchmarks to evaluate and highlight the performance gains achievable with the optimal policy. These baseline policies include: (a) Pumping policy, which prioritizes the use of elementary pairs of initial fidelity f0 as a resource to “pump” higher-fidelity links, (b) Nested policy, which performs distillation only between pairs of identical fidelity in state configuration, and (c) Greedy policy, which prioritizes immediate 2-to-1 ED action between the most compatible available pairs. p. 5 , fm}, the policy prioritizes the use of
elementary pairs of initial fidelity f0 as a resource to
“pump” higher-fidelity links. The action selection proceeds according to the following rules:
Entanglement generation: For all inactive memories
(fj = 0), the policy sets a
j
g = 1 attempt generation,
while leaving all occupied memories unchanged (a
j
g = 0
for fj ̸= 0)
Entanglement distillation: Let L0 = {j : fj = f0} be
the set of links at initial fidelity.Where the paper says it
Conclusion
The optimal deterministic policy minimizes the expected waiting time required to reach the target fidelity fT based on the current system configuration. The numerical results demonstrate that Topt increases when p is small, since a larger number of attempts are required to successfully establish entangled links. p. 6 In this section, we numerically analyze the dependence
of the expected waiting time under an optimal policy,
denoted by Topt, on the system parameters (m, p, f0, fT )
and compare it with baseline policies, as defined in
Sec. Our results show that Topt increases when the
entanglement generation probability p is small, since a
larger number of attempts are required to successfully
establish entangled links. Our numerical results demonstrate that the expected
waiting time under the optimal policy decreases with increasing entanglement generation probability p and the
number of quantum memories m, as both enhance the
availability of entangled resources and distillation opportunities. In contrast, the expected waiting time exhibits
a non-monotonic dependence on the initial fidelity f0 for
fixed ∆f. In contrast, the expected waiting time exhibits
a non-monotonic dependence on the initial fidelity f0 for
fixed ∆f. We further show that the optimal policy can
substantially reduce the expected waiting time compared
with heuristic baseline strategies, including entanglement
pumping, nested and greedy policy, with the magnitude
of the improvement depending on the system parameters (m, p, f0, fT ). In contrast, the expected waiting time exhibits
a non-monotonic dependence on the initial fidelity f0 for
fixed ∆f. We further show that the optimal policy can
substantially reduce the expected waiting time compared
with heuristic baseline strategies, including entanglement
pumping, nested and greedy policy, with the magnitude
of the improvement depending on the system parameters (m, p, f0, fT ). However, the variation of the
optimal policy’s advantage in expected waiting time with
the fidelity gap ∆f depends nontrivially on both the chosen baseline policy and the system parameters (p, m, f0). These findings highlight the importance of carefully balancing the trade-off between immediate distillation and
delayed operations, a feature that is not captured by simple heuristic policies.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Future Work
An interesting extension of this work would be to incorporate memory decoherence into the framework, thereby providing a more realistic description of quantum hardware. Furthermore, an important direction is to account for operational imperfections such as measurement errors. p. 9 In addition, while the present model considers
only two end nodes, it is natural to generalize this setting to multi-node quantum repeaters, where intermediate nodes assist in entanglement distribution between
Alice and Bob. Furthermore, an important direction is
to account for operational imperfections such as measurement errors. Furthermore, an important direction is
to account for operational imperfections such as measurement errors. In this case, the problem of optimizing
entanglement distillation policy can be formulated as a
partially observable Markov decision process (POMDP),
which naturally captures uncertainty regarding the system state and provides a more realistic decision-making
framework.Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Optimal policy selection in entanglement distillation can be achieved by formulating it as a Markov decision process (MDP) using value iteration to obtain optimal deterministic policies that minimize expected waiting time. This allows AI systems to systematically design policies that achieve target fidelity thresholds in realistic resource-constrained settings.
-
The improved AI system can determine the optimal sequence of entanglement generation and distillation operations based on the current configuration of quantum memory pairs, minimizing the expected time required to reach a target fidelity fT.
-
The system can utilize an action space defined by both entanglement generation and 2-to-1 entanglement distillation matchings, allowing for fully parallel, state-dependent distillation across multiple link pairs simultaneously.
-
The AI can learn policies that outperform baseline strategies like greedy, nested, and pumping policies by adapting the action based on the system parameters m, p, f0, and fT.
-
The system can achieve substantial reductions in expected waiting time compared to baseline strategies by selecting distillation operations based on the full memory configuration rather than simple heuristics.
Abstract
Entanglement distillation is a fundamental operation in quantum information processing used to obtain higher-fidelity entangled pairs from a supply of less entangled quantum states using local operations aided by classical communication (LOCC). In a physically relevant setting, where states with an initial fidelity of f 0, probabilistically generated over multiple, m, memory pairs distributed between two parties, Alice and Bob, are pairwise distilled, the optimal policy identifies the system-configuration dependent sequence of entanglement generation and distillation operations that need to be performed in order to minimize the expected time to reach some target fidelity f T>f 0. Here, we formulate and systematically analyze this task as a Markov decision problem and using a value iteration algorithm, obtain optimal deterministic policies that minimize the expected waiting time required to reach a target fidelity. Our results show that the expected waiting time under the optimal policy decreases with increasing generation probability p and number of quantum memories m - as expected. In contrast, it exhibits non-monotonic behavior with respect to f 0 for a fixed fidelity gap, (Δf = f T-f 0). While the optimal policy consistently outperforms baseline policies such as the greedy, nested and entanglement pumping policies, its relative advantage is regime-dependent, being determined by the system parameters (p,f 0,f T,m), and exhibits a nontrivial dependence on the fidelity gap Δf. Our results highlight the value of formulating entanglement distillation as a Markov decision problem, enabling the systematic design of policies that achieve target fidelity thresholds for quantum information tasks in realistic resource-constrained settings.
Sources
Related papers
- Computational regimes in matrix-product-state-based quantum trajectory simulations
- Toward chemically accurate quantum simulations on quantum hardware: A quantum-HPC hybrid approach
- Transversal surface-code game with reconfigurable qubits
- Fulqrum: a generalized framework for quantum subspace diagonalization
- Flux-Activated Resonant Control of a Bosonic Quantum Memory
- Machine learning the effects of many quantum measurements