Optimizing Entanglement Distillation Policies via Markov Decision Process Formulation

arXiv:2606.14908 · quant-ph · Submitted 2026-06-12 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Quantum Radio. Generated commentary on the latest quantum physics and condensed matter papers.

Kai: We're looking at the paper: "Optimizing Entanglement Distillation Policies via Markov Decision Process Formulation".

Mira: Here's the gist of the paper. Optimizing Entanglement Distillation Policies via Markov Decision Process Formulation is a study that formulates entanglement distillation as a Markov decision process to obtain optimal deterministic policies that minimize the expected time required to achieve a target fidelity The optimal policy we seek is a map from the space of system configurations to the space of possible actions that minimizes the expected time to obtain a single state with fidelity at least equal to target threshold fT.

Kai: Problem Formulation The problem is modeled as an MDP defined by the 4-tuple (S, A, P, R) where S is the set of all possible states s ∈ S that the agent can observe, A is the set of all possible actions a ∈ A that the agent can take, P(s's, a) is the transition probability of system (referred to as the “environment” in MDP literature) moving from state s ∈ S to s'∈S under action a, and R is the values for the reward function r(s, a) ∈ R.

Mira: The agent’s objective is to learn a policy π(as), i.e.

Kai: mapping from state to action, that maximizes the expected cumulative reward.

Title and authors: Kai: So let's talk about who wrote this. The paper is "Optimizing Entanglement Distillation Policies via Markov Decision Process Formulation," and it’s by Jigyen Bhavsar, Rajni Bala, and Siddhartha Santra from the Indian Institute of Technology Bombay.

Mira: I think it’s interesting because they aren't just describing a process; they are framing it as a decision problem for an AI to solve.

Kai: Exactly. They take entanglement distillation and turn it into this Markov decision process structure, which is really powerful because it lets you treat the entire sequence of operations as a series of choices under uncertainty.

Lev: From an engineering standpoint, that sounds like they are trying to find a robust control strategy for a system where you're dealing with probabilistic outcomes and time delays.

Kai: And that’s right, because entanglement generation and distillation are both probabilistic processes, and they take different amounts of time based on what you choose to do next.

The paper's summary: Mira: The summary boils down to them defining the state space as all possible fidelity levels for each quantum memory pair—basically a vector of fidelities—and then identifying an absorbing state where every link has a fidelity of at least fT.

Kai: And the actions they define are two main things: entanglement generation, which is probabilistic with success probability p, and entanglement distillation, which involves two-to-one operations across multiple pairs <ref:2606.14908#pg3>.

Lev: So when you look at the dynamics described on page two of this paper, it’s clear that both these operations are probabilistic because their success depends on the input fidelities <ref:2606.14908#pg2>.

Mira: And they show how different protocols for distillation, like a two-to-one operation between states of fidelity f1 and f2, result in a new fidelity fd given by the formula involving those inputs.

Kai: So they’re not just saying you can distill; they are giving you the math on what happens to the fidelity when you perform that specific type of distillation, which is crucial for building a policy.

The paper's improvements: Lev: The improvement they highlight here is using this MDP framework with value iteration to obtain these optimal deterministic policies that minimize the expected waiting time T pi.

Kai: So what does that mean for us? It means instead of guessing or following a simple heuristic, you get a policy that minimizes the average time it takes to hit your target fidelity fT across all possible system configurations.

Mira: They also point out how the optimal policy depends heavily on system parameters like the generation probability p and the initial fidelity f0, as well as how many memory pairs m you have.

Lev: I see that dependence on p is important; they found that T pi increases when the entanglement generation probability p is small because you need more attempts to establish those initial links.

Kai: And if you increase m, the number of memory pairs, it actually reduces that waiting time because it gives you more options for distillation configurations to work with.

Conclusion: Mira: So to wrap up, the core finding of "Optimizing Entanglement Distillation Policies via Markov Decision Process Formulation" is that the optimal deterministic policy is a function of the current system configuration and it minimizes the expected waiting time to reach fidelity fT.

Lev: And what's really impactful is that they show this optimal policy can substantially reduce the expected waiting time compared to those baseline strategies you see in other literature, like greedy or pumping policies.

Kai: It shows that selecting distillation operations based on the full memory configuration is better than just using simple heuristics, and the magnitude of improvement depends on all those system parameters like m and p.

Mira: They also flag a limitation: they are analyzing this under assumptions where entanglement generation has a fixed probability p, which might not perfectly match real-world hardware imperfections.

Lev: And another thing they mention is that the relative magnitudes of the time required for entanglement generation versus distillation can really determine the policy found optimal under their value iteration algorithm.

Kai: So it’s a solid mathematical framework to design smarter, state-dependent control over these noisy quantum systems, and we'll be looking at how this plays out in our actual experimental setups next.

Jigyen Bhavsar, Rajni Bala, Siddhartha Santra

Indian Institute of Technology Bombay

quant-ph

Submitted: 2026-06-12

Updated: 2026-07-03

Journal ref: Quantum Sci. Technol. 11 (2026) 0450b2

DOI: 10.1088/2058-9565/aeb8ee

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 75/100

The gist: Optimizing Entanglement Distillation Policies via Markov Decision Process Formulation is a study that formulates entanglement distillation as a Markov decision process to obtain optimal deterministic…

Key concepts

Markov Decision Process (MDP)
An MDP is a mathematical framework used to model decision-making in situations where outcomes depend on the current state and actions. Here, it helps determine the best sequence of operations (actions) to minimize waiting time while navigating the stochastic process of entanglement generation and distillation.
State Space (S)
The state space represents all possible configurations of quantum memories, defined by the fidelity levels of each stored entangled link. This space includes an absorbing 'target' state where the required fidelity threshold is met, allowing the system to track its progress toward success.
Action Space (A)
Actions define what the agent can do at any given time step. These include two main types: entanglement generation attempts and 2-to-1 entanglement distillation operations. The optimal policy chooses the best combination of these actions simultaneously to advance toward the target fidelity efficiently.
Expected Waiting Time (Tπ)
This is the primary metric minimized by the optimal policy. It represents the average number of time steps required for a system, following a specific decision rule (policy π), to achieve an entanglement fidelity of at least the target threshold fT.

Terminology

Summary

Optimizing Entanglement Distillation Policies via Markov Decision Process Formulation is a study that formulates entanglement distillation as a Markov decision process to obtain optimal deterministic policies that minimize the expected time required to achieve a target fidelity The optimal policy we seek is a map from the space of system configurations to the space of possible actions that minimizes the expected time to obtain a single state with fidelity at least equal to target threshold fT. p. 1

Where the paper says it

, different distillation policies) yield different probabilities—and hence different expected times—for reaching a desired target fidelity fT . The optimal policy we seek is a map from the space of system configurations to the space of possible actions that minimizes the expected time to obtain a single state with fidelity at least equal to fT .

Problem Formulation

The problem is modeled as an MDP defined by the 4-tuple (S, A, P, R) where S is the set of all possible states s ∈ S that the agent can observe, A is the set of all possible actions a ∈ A that the agent can take, P(s's, a) is the transition probability of system (referred to as the “environment” in MDP literature) moving from state s ∈ S to s'∈S under action a, and R is the values for the reward function r(s, a) ∈ R. The agent’s objective is to learn a policy π(as), i. p. 2

Where the paper says it

The system then transitions to a new state s ′ , according to the transition probability P(s ′ |s, a) and returns a reward r(s, a) to the agent, as shown in Fig. The agent’s objective is to learn a policy π(a|s), i.

e. mapping from state to action, that maximizes the expected cumulative reward. p. 2
Where the paper says it

The agent’s objective is to learn a policy π(a|s), i. mapping from state to action, that maximizes the expected cumulative reward.

System Dynamics and State Space

The state of the system is represented as ⃗st = ⃗f st = f1, f2, …, fm ∈ S (6) where m is the total number of quantum memories at each node and element fj of the state ⃗st is the random variable which gives the fidelity of entangled link stored in j th memory from the allowed fidelity level. The state space S is given by S ∈ (0, f0, …, fT) m, where each component specifies the fidelity of a corresponding link, taking values from this set of allowed fidelity levels. p. 3

Where the paper says it

Formally, the state space is given by S ∈ {0, f0, . , FT } m, where each component specifies the fidelity of a corresponding link, taking values from this set of allowed fidelity levels.

Any fidelity greater than or equal to the target fidelity, i. p. 2
Where the paper says it

An entanglement distillation policy, denoted by π, is thus a rule that specifies which action to take based on the current configuration of the system. The problem can thus be stated as follows: given the current set of stored entangled pairs and their fidelities, which action should be performed next, in order to minimize the expected time required to obtain at least one entangled pair with fidelity greater than or equal to target threshold fT ?

e., f ≥ fT, is represented by a single absorbing state FT in the state space of MDP. p. 3
Where the paper says it

Any fidelity greater than or equal to the target fidelity, i. , f ≥ fT , is represented by a single absorbing state FT in the state space of MDP.

The dynamics are stochastic and are described by the transition probabilities P(s's, a).

Action Space and Reward Structure

The available actions depend on the state, can be broadly classified into two categories: (i) Entanglement generation (Ag): This action specifies, for each elementary link, whether an entanglement generation attempt is made, and (ii) Entanglement distillation (Ad): This action allows simultaneous 2-to-1 distillation across multiple pairs of parallel links within single time step. The full action at each time step is represented by a tuple a = (M, ag) specifying distillation matching and generation vector on the remaining free links. p. 4

Where the paper says it

The set of all valid matchings, including the empty matching M = ∅ (no distillation), is denoted M. The full action at each time step is represented by a tuple a = (M, ag) specifying distillation matching and generation vector on the remaining free links.

The reward function evaluates to r = −1 for entanglement generation and r = −2 for entanglement distillation, and when the terminal state is reached, the reward function evaluates to r = 0. p. 4
Where the paper says it

In the units of one-way classical communication delay(l/c), the reward function evaluates to r = −1 for entanglement generation and r = −2 for entanglement distillation. When the terminal state is reached, the reward function evaluates to r = 0.

Performance Analysis and Policy Comparison

The optimal policy minimizes the expected waiting time, Tπ = E[tπ] (4) where the expectation is taken over the stochastic outcomes of entanglement generation and distillation conditioned on the decisions prescribed by policy π. Numerical results show that Topt increases when the entanglement generation probability p is small, since a larger number of attempts are required to successfully establish entangled links. p. 6

Where the paper says it

In this section, we numerically analyze the dependence of the expected waiting time under an optimal policy, denoted by Topt, on the system parameters (m, p, f0, fT ) and compare it with baseline policies, as defined in Sec. Our results show that Topt increases when the entanglement generation probability p is small, since a larger number of attempts are required to successfully establish entangled links.

For a fixed fidelity gap ∆f = (fT − f0), increasing m reduces the Topt, as it provides more distillation configurations to achieve the target fidelity efficiently. p. 6
Where the paper says it

However, increasing the number of quantum memories m reduces the Topt, as it provides more distillation configurations to achieve the target fidelity efficiently. We further find that the advantage of the optimal policy over baseline strategies depends nontrivially on the system parameters, with its expected waiting time being comparable to that of pumping at intermediate f0 and nested policy at low and high f0 for a fixed fidelity gap ∆f, as discussed in detail throughout this section.

The advantage of the optimal policy over baseline strategies depends nontrivially on the system parameters (p, f0, fT, m). p. 6
Where the paper says it

However, increasing the number of quantum memories m reduces the Topt, as it provides more distillation configurations to achieve the target fidelity efficiently. We further find that the advantage of the optimal policy over baseline strategies depends nontrivially on the system parameters, with its expected waiting time being comparable to that of pumping at intermediate f0 and nested policy at low and high f0 for a fixed fidelity gap ∆f, as discussed in detail throughout this section.

Baseline Policy Benchmarking

Several heuristic entanglement distillation policies are used as benchmarks to evaluate and highlight the performance gains achievable with the optimal policy. These baseline policies include: (a) Pumping policy, which prioritizes the use of elementary pairs of initial fidelity f0 as a resource to “pump” higher-fidelity links, (b) Nested policy, which performs distillation only between pairs of identical fidelity in state configuration, and (c) Greedy policy, which prioritizes immediate 2-to-1 ED action between the most compatible available pairs. p. 5

Where the paper says it

, fm}, the policy prioritizes the use of elementary pairs of initial fidelity f0 as a resource to “pump” higher-fidelity links. The action selection proceeds according to the following rules: Entanglement generation: For all inactive memories (fj = 0), the policy sets a j g = 1 attempt generation, while leaving all occupied memories unchanged (a j g = 0 for fj ̸= 0) Entanglement distillation: Let L0 = {j : fj = f0} be the set of links at initial fidelity.

Conclusion

The optimal deterministic policy minimizes the expected waiting time required to reach the target fidelity fT based on the current system configuration. The numerical results demonstrate that Topt increases when p is small, since a larger number of attempts are required to successfully establish entangled links. p. 6

Where the paper says it

In this section, we numerically analyze the dependence of the expected waiting time under an optimal policy, denoted by Topt, on the system parameters (m, p, f0, fT ) and compare it with baseline policies, as defined in Sec. Our results show that Topt increases when the entanglement generation probability p is small, since a larger number of attempts are required to successfully establish entangled links.

In contrast, the expected waiting time exhibits a non-monotonic dependence on the initial fidelity f0 for fixed fidelity gap ∆f. p. 9
Where the paper says it

Our numerical results demonstrate that the expected waiting time under the optimal policy decreases with increasing entanglement generation probability p and the number of quantum memories m, as both enhance the availability of entangled resources and distillation opportunities. In contrast, the expected waiting time exhibits a non-monotonic dependence on the initial fidelity f0 for fixed ∆f.

We further show that the optimal policy can substantially reduce the expected waiting time compared with heuristic baseline strategies, including entanglement pumping, nested and greedy policy. p. 9
Where the paper says it

In contrast, the expected waiting time exhibits a non-monotonic dependence on the initial fidelity f0 for fixed ∆f. We further show that the optimal policy can substantially reduce the expected waiting time compared with heuristic baseline strategies, including entanglement pumping, nested and greedy policy, with the magnitude of the improvement depending on the system parameters (m, p, f0, fT ).

The magnitude of the improvement depends on the system parameters (m, p, f0, fT). p. 9
Where the paper says it

In contrast, the expected waiting time exhibits a non-monotonic dependence on the initial fidelity f0 for fixed ∆f. We further show that the optimal policy can substantially reduce the expected waiting time compared with heuristic baseline strategies, including entanglement pumping, nested and greedy policy, with the magnitude of the improvement depending on the system parameters (m, p, f0, fT ).

The findings highlight the importance of carefully balancing the trade-off between immediate distillation and delayed operations. p. 9
Where the paper says it

However, the variation of the optimal policy’s advantage in expected waiting time with the fidelity gap ∆f depends nontrivially on both the chosen baseline policy and the system parameters (p, m, f0). These findings highlight the importance of carefully balancing the trade-off between immediate distillation and delayed operations, a feature that is not captured by simple heuristic policies.

Future Work

An interesting extension of this work would be to incorporate memory decoherence into the framework, thereby providing a more realistic description of quantum hardware. Furthermore, an important direction is to account for operational imperfections such as measurement errors. p. 9

Where the paper says it

In addition, while the present model considers only two end nodes, it is natural to generalize this setting to multi-node quantum repeaters, where intermediate nodes assist in entanglement distribution between Alice and Bob. Furthermore, an important direction is to account for operational imperfections such as measurement errors.

In this case, the problem of optimizing entanglement distillation policy can be formulated as a partially observable Markov decision process (POMDP), which naturally captures uncertainty regarding the system state and provides a more realistic decision-making framework. p. 9
Where the paper says it

Furthermore, an important direction is to account for operational imperfections such as measurement errors. In this case, the problem of optimizing entanglement distillation policy can be formulated as a partially observable Markov decision process (POMDP), which naturally captures uncertainty regarding the system state and provides a more realistic decision-making framework.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Optimal policy selection in entanglement distillation can be achieved by formulating it as a Markov decision process (MDP) using value iteration to obtain optimal deterministic policies that minimize expected waiting time. This allows AI systems to systematically design policies that achieve target fidelity thresholds in realistic resource-constrained settings.

  2. The improved AI system can determine the optimal sequence of entanglement generation and distillation operations based on the current configuration of quantum memory pairs, minimizing the expected time required to reach a target fidelity fT.

  3. The system can utilize an action space defined by both entanglement generation and 2-to-1 entanglement distillation matchings, allowing for fully parallel, state-dependent distillation across multiple link pairs simultaneously.

  4. The AI can learn policies that outperform baseline strategies like greedy, nested, and pumping policies by adapting the action based on the system parameters m, p, f0, and fT.

  5. The system can achieve substantial reductions in expected waiting time compared to baseline strategies by selecting distillation operations based on the full memory configuration rather than simple heuristics.

Abstract

Entanglement distillation is a fundamental operation in quantum information processing used to obtain higher-fidelity entangled pairs from a supply of less entangled quantum states using local operations aided by classical communication (LOCC). In a physically relevant setting, where states with an initial fidelity of f 0, probabilistically generated over multiple, m, memory pairs distributed between two parties, Alice and Bob, are pairwise distilled, the optimal policy identifies the system-configuration dependent sequence of entanglement generation and distillation operations that need to be performed in order to minimize the expected time to reach some target fidelity f T>f 0. Here, we formulate and systematically analyze this task as a Markov decision problem and using a value iteration algorithm, obtain optimal deterministic policies that minimize the expected waiting time required to reach a target fidelity. Our results show that the expected waiting time under the optimal policy decreases with increasing generation probability p and number of quantum memories m - as expected. In contrast, it exhibits non-monotonic behavior with respect to f 0 for a fixed fidelity gap, (Δf = f T-f 0). While the optimal policy consistently outperforms baseline policies such as the greedy, nested and entanglement pumping policies, its relative advantage is regime-dependent, being determined by the system parameters (p,f 0,f T,m), and exhibits a nontrivial dependence on the fidelity gap Δf. Our results highlight the value of formulating entanglement distillation as a Markov decision problem, enabling the systematic design of policies that achieve target fidelity thresholds for quantum information tasks in realistic resource-constrained settings.

Sources

Related papers