Geometric Characterisation and Structured Trajectory Surrogates for Clinical Dataset Condensation

arXiv:2604.21638 · cs.LG · Submitted 2026-04-23 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom, and with me are Jane, Lu, senior AI researcher at Tsinghua, Meng, lead engineer at a mysterious AI startup and Lalam, the in-house Large Language Model.

Jane: Today's paper: "Geometric Characterisation and Structured Trajectory Surrogates for Clinical Dataset Condensation".

Tom: The gist Dataset condensation constructs compact synthetic datasets that retain training utility, and Bézier Trajectory Matching (BTM) improves predictive performance across diverse clinical datasets, especially in low-prevalence regimes.

Jane: First, who's behind it and why it matters.

Title and authors: Tom: So we're talking about this paper called "Geometric Characterisation and Structured Trajectory Surrogates for Clinical Dataset Condensation". It looks like they're looking at how to make those synthetic datasets from real data better.

Jane: Yeah, they are focusing on trajectory matching, which is a way to use changes in model parameters from training on real data to supervise the synthetic data. But the structure of that supervision signal remains pretty unclear for a long time.

Lu: This paper essentially gives us a geometric way to look at that trajectory matching residual. They show that if you have a fixed synthetic dataset, the error in trajectory matching is lower bounded by how well you can approximate parameter changes from real data using just what that synthetic dataset can actually reproduce.

Meng: So, if the supervision signal is too broad compared to what the synthetic set can handle, there's a problem with it. What's that problem exactly?

Tom: It leads to something they call a conditional representability bottleneck when the resulting supervision signal is spectrally broad relative to the reachable span >

Jane: They find that standard trajectory matching often uses parameter changes with broad spectral structure, especially in low-prevalence settings, which explains weaker performance when predicting rare events >

Lu: To fix this mismatch, they propose Bézier Trajectory Matching or BTM. Instead of using those discrete SGD trajectories, they replace them with quadratic Bézier trajectory surrogates between the initial and final model states >

Meng: So what's the big idea behind those surrogates? Are they just smoother paths?

Tom: They are optimized to reduce average loss along the path while replacing that broad SGD-derived supervision with a more structured, lower-rank signal that fits better with the optimization constraints of a fixed synthetic dataset >

Jane: That means instead of noisy, random updates across epochs, you get something much more consistent and aligned with what the synthetic data can actually support >

Lu: The key structural advantage is that all segment displacements from a single quadratic Bézier trajectory lie in a two-dimensional subspace, meaning any displacement matrix constructed from segments of a single surrogate trajectory has rank at most two > <ref:2604.21638#pg1>

Title and authors: Meng: That sounds like it’s simplifying the complexity of the supervision signal significantly. How much does that actually help in practice?

Tom: It substantially reduces storage requirements across all clinical datasets, yielding approximately thirty-three times lower storage on Oxford and eICU, and twenty times lower storage on MIMIC-III > <ref:2604.21638#pg1>

Jane: That's a huge number for data storage when you're working with these large clinical datasets. And it’s not just about size, right?

Lu: No, the surrogate family is not only low-rank but also functionally meaningful under explicit geometric conditions, suggesting that structured supervision improves robustness to architecture shift >

Meng: Robustness to architecture shift is important because models trained on different network types can behave differently when you try to transfer them >

Tom: Exactly. The training loss profiles along those surrogate trajectories show that the Bézier trajectory remains consistently in lower-loss regions due to its task-optimised construction >

Jane: So, these BTM methods consistently match or improve upon standard trajectory matching, especially in low-prevalence and low-synthetic-budget settings >

Lu: The results indicate that effective trajectory matching depends on structuring the supervision signal rather than just reproducing the stochastic optimization paths >

Meng: So it’s less about capturing every random step and more about capturing a general structure >

Tom: Precisely. Specifically, BTM achieves the best AUPRC in low-prevalence settings, such as the Portsmouth dataset, showing gains of three point two percent over the strongest baseline at ipc=five hundred >

Jane: That three point two percent gain is pretty significant when dealing with rare events that are hard to preserve under severe compression > <ref:2604.21638#pg1>

Lu: The paper shows that replacing raw SGD supervision with structured, low-rank surrogate trajectories improves the utility of condensed datasets, and this effect is most evident when transferring across model classes with different inductive biases >

Meng: So if I'm building a new model architecture, BTM seems like it helps keep the task structure intact even when I change things up >

Tom: It does. And they show that the performance of BTM is robust under architecture shift, with absolute AUPRC gains ranging from zero point zero zero eight to zero point zero two four on eICU >

Title and authors: Jane: So for someone just listening to the show, what does this mean for how we think about making these synthetic datasets useful?

Lu: It suggests that effective dataset condensation depends not only on how much supervision you provide, but also on how that supervision is structured >

Meng: I think the practical part here is the storage reduction. Thirty-three times less storage means we can actually deploy these kinds of methods in resource-constrained clinical settings >

Tom: And they even found that you can reduce the inner loop steps N to thirty while sustaining strong AUROC and AUPRC performance, showing Bézier surrogates provide stable supervision, enabling the student to train effectively with few gradient steps > <ref:2604.21638#pg1>

Jane: That makes sense. If you need fewer gradient steps for good results, it's a big win for training efficiency >

Lu: The overall results reinforce that replacing raw SGD supervision with structured, low-rank surrogate trajectories improves the utility of condensed datasets >

Meng: It’s about getting the right signal at the right time. We need that structure to make sure we aren't just overfitting to specific optimization dynamics >

Tom: So, in summary, this paper on "Geometric Characterisation and Structured Trajectory Surrogates for Clinical Dataset Condensation" shows that structuring supervision via Bézier trajectories gives us a more structured signal while preserving functionally meaningful training dynamics >

Jane: It’s particularly effective in the low-data, low-prevalence regimes most relevant to clinical practice >

Lu: The paper also cautions that these condensed or synthetic datasets are not inherently safe to share and BTM doesn't provide a formal privacy guarantee >

Meng: That’s a necessary caveat. We can't just assume this is the end of the story for data sharing security, even with these neat structural improvements >

Tom: So, what’s next? The paper suggests incorporating differential privacy into this approach, which is a natural next step for future work >

Jane: It seems like the main point here is that when you condense data, focusing on the structure of the supervision signal matters as much as just how much of it you have >

The paper's summary: Tom: So, we're looking at how this paper tackles dataset condensation for clinical AI by focusing on structuring the supervision signal instead of just throwing more data at the problem, right?

Jane: Exactly. This paper says that standard methods of trajectory matching often produce a very broad supervision signal when you’re working with synthetic data, and that’s where things get messy in low-prevalence settings.

Tom: They show there's a geometric reason for this—the residual error is actually lower bounded by how well the synthetic dataset can replicate parameter changes from real training. So, if the supervision signal is too wide compared to what the data can really handle, you run into a bottleneck where it just doesn't work well.

Lu: And that’s why they propose Bézier Trajectory Matching, or BTM. They swap out those rough standard SGD paths for quadratic Bézier curves between the starting and ending points of your model states.

Meng: So instead of relying on every noisy step the AI takes during training, you use a smoother path as a guide? What's the real gain there?

Tom: The main thing is that this creates a much more structured, lower-rank supervision signal. It’s better aligned with the rules of your fixed synthetic dataset because it avoids that diffuse high-rank variability we talked about earlier.

Jane: And they found that this structure helps even when the data is scarce, like in low-prevalence clinical prediction. They saw gains of over three percent on specific datasets by using this structured approach instead of just raw trajectory matching.

Lu: The geometry of these Bézier curves is actually really powerful—all the movements along a single curve stay within a two-dimensional subspace, meaning the resulting supervision signal has a rank at most two. That’s incredibly constrained.

Meng: Constrained how? Does that translate to real-world deployment for an engineer?

Tom: It translates directly into storage and robustness. Because the signal is low-rank, they got about thirty-three times less storage on some datasets and twenty times less on others. Plus, this structured supervision makes the system more robust when you try to move a model from one architecture to another.

Jane: That robustness is important because we often have to transfer models between different network types in clinical settings. So, the structure helps keep the core task signal intact regardless of how you change the underlying code.

Lu: They also found that even with these structured curves, you can still get strong performance and reduce training steps significantly, sometimes cutting the inner loop steps down to just thirty while keeping good results.

Tom: So it’s not just about getting a bigger number of data points; it’s about making sure the signal you *do* have is perfectly shaped for what the model needs to learn.

Jane: And they did put a big caution out there, though—the paper makes it clear that these synthetic datasets aren't safe to share blindly because BTM doesn't provide any formal privacy guarantees.

Lu: Yeah, and since they’re looking at this for clinical settings, the authors suggest the next step should be integrating differential privacy into this whole framework.

Meng: So, to summarize for someone just listening—the core idea is that structuring your supervision with Bézier curves makes synthetic data much more useful in low-data scenarios by giving you a clean, low-rank signal that saves massive amounts of storage and keeps models stable across different types.

The paper's improvements: Tom: So, we’re talking about how they actually improve this method beyond just swapping out the trajectory path for something smoother, right?

Jane: Yeah, they show that by optimizing these Bézier surrogates through unrolled student updates—essentially training the surrogate itself—they can get a better fit for the synthetic data constraints.

Tom: Exactly. This means you’re not just drawing a curve; you’re making sure that curve is optimized to induce training dynamics that line up perfectly with what your fixed synthetic dataset expects. It makes the synthetic data generation process smarter, not just faster or cheaper.

Lu: And structurally, this refinement reinforces the idea that we need to optimize for structure rather than just following some general optimization rule. It ties back to that low-rank property we discussed earlier.

Meng: So if I’m building a system, what does this structural improvement actually mean for my deployment pipeline? Is it more stable in production?

Tom: It is. The paper shows that because the supervision signal is so tightly constrained and structured, the performance of the AI model remains consistent even when you switch to different network architectures. That’s crucial for real-world deployment where models aren't always perfectly interchangeable.

Jane: That consistency across architectures is a big deal for reliability in clinical use cases. It suggests that the task-relevant signal is more important than being tied to one specific way of training the model.

Lu: It points toward a general principle, Tom—that we need to capture that underlying task structure without getting stuck in overfitting to the specific optimization quirks of one particular AI setup.

Tom: Right. And they also found a way to make the training itself more efficient, reducing those inner loop steps down to just thirty while keeping high performance metrics like AUROC and AUPRC. That’s a practical win for training time.

Meng: Thirty steps is good; that’s something I can actually measure on my hardware. So, for the engineer out there, what does this mean about how much effort they need to put into creating these synthetic datasets?

Jane: It means that the quality of the *structure* in your supervision signal matters more than just having a huge volume of data points. If you can create a structured signal, you can get better results with less raw input.

Lu: It opens up possibilities for how we generate synthetic data—we aren't just generating noise; we’re generating signals optimized for the student model’s geometry. This is really about learning the shape of the ideal training path for a given task.

Tom: So, it shifts the focus from just maximizing data quantity to maximizing signal quality and structure. It’s a fundamental shift in how we think about dataset condensation itself.

Jane: And this leads us to thinking bigger—if we can get this much better at structuring the supervision signal, what does that mean for the long-term development of helpful AI?

Conclusion: Tom: So to wrap things up, this paper on "Geometric Characterisation and Structured Trajectory Surrogates for Clinical Dataset Condensation" shows that we get much better synthetic data utility by focusing on structure over just volume.

Jane: It’s really about recognizing that the way we supervise our AI models should be organized according to the constraints of the data we have, especially when it comes to rare events in clinical settings.

Lu: The main implication is that for dataset condensation to work well, it has to be smarter about how it structures that supervision signal, moving away from those broad random updates we used before.

Meng: Structuring the supervision means better storage and better stability when we move models around, which is what matters for putting these things into actual clinical workflows.

Tom: Exactly. The numbers are striking—we’re talking about thirty-three times less storage on Oxford data, and that low-rank property keeps the model robust under architecture shift.

Jane: And they showed that this structured approach consistently beats the standard trajectory matching, especially when you have very little data to start with or you're dealing with rare outcomes.

Lu: It suggests a future where synthetic data isn't just a filler; it becomes a carefully engineered training partner tailored to the inherent geometry of the task.

Tom: So, we’re moving from simply throwing more stuff at an AI problem to designing the *shape* of that supervision signal itself.

Jane: It’s a shift in how we approach synthetic data generation—making it intentional rather than just expansive.

Lu: If we can master this geometric characterization, it opens up new avenues for creating highly efficient and reliable synthetic training environments across all domains.

Meng: From an engineering standpoint, having a lower-rank structure that’s guaranteed to be stable is exactly what we need for robust deployment pipelines.

Tom: It’s a solid step forward in making trajectory-based condensation practical and useful for high-stakes fields like medicine.

Jane: Ultimately, this work on "Geometric Characterisation and Structured Trajectory Surrogates for Clinical Dataset Condensation" tells us that structure is the secret ingredient to making compact datasets actually do good things.

Pafue Christy Nganjimi pafue.nganjimi@eng.ox.ac.uk, Andrew Soltan andrew.soltan@oncology.ox.ac.uk, Danielle Belgrave danielle.x.belgrave@gsk.com, Lei Clifton lei.clifton@phc-ox-ac-uk, David Clifton david_clifton@eng.ox.ac-uk, Anshul Thakur anshul_thakur@eng.ox.ac-uk

Department of Engineering Science, University of Oxford · Nuffield Department of Primary Care, University of Oxford · GlaxoSmithKline

cs.LG

Submitted: 2026-04-23

Updated: 2026-04-23

Comments: 34 pages, 7 figures

Journal ref: Transactions on Machine Learning Research (2026)

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 91/100

The gist: The gist Dataset condensation constructs compact synthetic datasets that retain training utility, and Bézier Trajectory Matching (BTM) improves predictive performance across diverse clinical

Key concepts

Geometric Characterisation of Trajectory Matching: This concept shows that the error in matching trajectories is limited by how well the real data's parameter changes can be approximated within the gradient span reachable by a fixed synthetic dataset. This establishes a bottleneck where broad supervision signals fail when they are too different from what the student model can learn.
Bézier Trajectory Matching (BTM): BTM uses quadratic Bézier curves to create smooth surrogates between an initial and final model state instead of using raw SGD paths. These surrogates are optimized to minimize loss along the path, producing a structured, lower-rank supervision signal that is better aligned with the constraints of the fixed synthetic dataset.
Low-Rank Supervision Signal: BTM generates supervision signals that have a low effective rank because all segment displacements from a single Bézier trajectory lie in a two-dimensional subspace. This structured, low-rank signal is superior to diffuse, high-rank supervision derived from standard SGD trajectories, making it better suited for the representational constraints of fixed datasets.

Terminology

Bezier Trajectory Matching — 1× in this paper

Summary

The gist Dataset condensation constructs compact synthetic datasets that retain training utility, and Bézier Trajectory Matching (BTM) improves predictive performance across diverse clinical datasets, especially in low-prevalence regimes.

Geometric Characterisation of Trajectory Matching

The paper provides a geometric characterisation of trajectory matching showing that for any fixed synthetic dataset, the trajectory matching residual is lower bounded by how well the parameter changes induced by training on real data can be approximated within the gradient span reachable from that dataset. This leads to a conditional representability bottleneck when the resulting supervision signal is spectrally broad relative to the reachable span. The authors empirically show that the parameter changes used by trajectory matching often exhibit broad spectral structure, especially in low-prevalence settings, which explains weaker performance in rare-event clinical prediction.

Bézier Trajectory Matching (BTM)

Motivated by this mismatch, the authors propose Bézier Trajectory Matching (BTM), which replaces SGD trajectories with quadratic Bézier trajectory surrogates between initial and final model states. These surrogates are optimised to reduce average loss along the path while replacing broad SGD-derived supervision with a more structured, lower-rank signal that is better aligned with the optimisation constraints of a fixed synthetic dataset and substantially reduce trajectory storage. The resulting supervision signal is more structured and has lower effective rank, making it better aligned with the representational constraints of a fixed synthetic dataset while also substantially reducing trajectory storage.

Key Findings in Clinical Evaluation

Experiments on five clinical datasets demonstrate that BTM consistently matches or improves upon standard trajectory matching, with the largest gains in low-prevalence and low-synthetic-budget settings. The results indicate that effective trajectory matching depends on structuring the supervision signal rather than reproducing stochastic optimisation paths. Specifically, BTM achieves the best AUPRC in low-prevalence settings, such as the Portsmouth dataset, showing gains of 3.2% over the strongest baseline at ipc=500.

Structural Advantages of Bézier Surrogates

Bézier trajectories are constructed between model initialisations and their corresponding trained solutions using quadratic Bézier curves, where each control point is optimised to reduce the average loss along the path. The key structural property is that all segment displacements from a single quadratic Bézier trajectory lie in a two-dimensional subspace, meaning any displacement matrix constructed from segments of a single surrogate trajectory has rank at most 2. This yields supervision that is better aligned with the constraints of student optimisation by replacing diffuse high-rank supervision with a controlled low-rank alternative that is better aligned with the constraints of student optimisation.

Storage and Robustness

Bézier surrogates substantially reduce storage requirements across all clinical datasets, yielding approximately 33× lower storage on Oxford (NHS) and eICU, and 20× lower storage on MIMIC-III. Furthermore, the surrogate family is not only low-rank but also functionally meaningful under explicit geometric conditions, suggesting that structured supervision improves robustness to architecture shift. The training loss profiles along surrogate trajectories show that the Bézier trajectory remains consistently in lower-loss regions due to its task-optimised construction.

Conclusion

BTM provides a more structured supervision signal while preserving functionally meaningful training dynamics, and it is particularly effective in the low-data, low-prevalence regimes most relevant to clinical practice. This work suggests that effective dataset condensation depends not only on how much supervision is provided, but also on how that supervision is structured. The paper concludes by noting that BTM remains robust under architecture shift and reduces trajectory storage requirements by up to 33×, improving the practicality of trajectory-based condensation in resource-constrained clinical settings. A broader impact statement cautions that condensed or synthetic datasets are not inherently safe to share and BTM does not provide a formal privacy guarantee. The paper suggests that effective dataset condensation depends on how the supervision is structured, which is particularly important because compact synthetic datasets may help support more efficient reuse of information derived from governed datasets while reducing storage and training costs. A natural next step for future work is incorporating differential privacy into this approach. The paper also shows that the inner loop steps N can be reduced to 30 while sustaining strong AUROC and AUPRC performance, demonstrating that Bézier surrogates provide stable supervision, enabling the student to train effectively with few gradient steps. The optimal segment length is found to be ∆t = 0.2 for the Portsmouth dataset. The overall results reinforce that replacing raw SGD supervision with structured, low-rank surrogate trajectories improves the utility of condensed datasets. This effect is most evident when transferring across model classes with different inductive biases, where preserving task-relevant structure without overfitting to architecture-specific optimisation dynamics becomes especially important. The paper shows that the performance of BTM is robust under architecture shift, with absolute AUPRC gains ranging from 0.008 to 0.024 on eICU. This suggests that the supervision induced by BTM is not tightly coupled to a specific parametrisation of the MLP, but instead captures task-relevant signal that generalises across closely related architectures. The paper also shows that random initialisation remains competitive with real initialisation on eICU, suggesting that competitive condensation can still be achieved from a purely synthetic starting point. A PIPELINE OF PROPOSED METHOD is detailed in Algorithm 1. The proof section gathers the proofs of the main theoretical results, including Theorem 1 and Theorem 2 regarding the geometric lower bounds underlying trajectory matching. The final evaluation uses longer training schedules to obtain final results. The paper presents a geometric view of trajectory matching showing that, for a fixed synthetic dataset, the trajectory-matching residual is lower bounded by the component of the supervision signal lying outside the student’s reachable gradient span. This perspective clarifies a core limitation of existing trajectory-based dataset condensation methods and motivates the design of supervision signals that are better aligned with the student’s optimisation geometry. The paper is presented as arXiv:2604.21638v1 [cs.LG] 23 Apr 2026. The paper includes a broader impact statement balancing the benefits against risks such as potential bias and misuse. The paper is presented with an appendix detailing the proofs of the main theoretical results. The paper includes a list of references at the end and a section on implementation details. The paper is presented with figures illustrating the comparison between raw SGD trajectories and Bézier trajectory surrogates. The paper is presented with performance results across different synthetic budgets, showing BTM consistently outperforms baselines in low-prevalence settings.

Improvements for AI systems

Bold header: Structured Supervision via Bézier Trajectories

BTM replaces raw SGD trajectories with quadratic Bézier trajectory surrogates to replace discrete SGD trajectories, which yields a more structured supervision signal that is better aligned with the representational constraints of a fixed synthetic dataset. This allows the AI system to achieve higher utility in low-prevalence settings by focusing on what is functionally meaningful progress while avoiding diffuse high-rank variability.

Bold header: Improved Trajectory Storage Efficiency

The surrogate representation yields a substantial reduction in memory when T ≫ 1, as it requires only O(3M) parameter vectors compared to standard trajectory matching's O(TM) parameter vectors. This enables the deployment of more expert trajectories in resource-constrained clinical settings, reducing storage requirements by up to 33× on certain datasets.

Bold header: Robustness to Architecture Shift

By using BTM, the system demonstrates that it remains robust under architecture shift, suggesting that structuring the supervision signal through learned low-rank surrogate trajectories improves robustness to architecture shift, which is crucial when transferring models between different network types like TCN and LSTM.

Bold header: Optimized Synthetic Data Generation

The BTM pipeline allows for the optimization of synthetic data by differentiating LBTM through the unrolled student updates, ensuring that the resulting synthetic dataset is optimized to induce training dynamics that are better aligned with a fixed synthetic dataset.

Bold header: Enhanced Low-Data Performance in Clinical Prediction

The system consistently achieves superior performance, with the largest gains in low-prevalence and low-synthetic-budget settings, making it particularly effective at retaining positive-class signal where positive-class signal is rare and difficult to preserve under severe compression.

Abstract

Dataset condensation constructs compact synthetic datasets that retain the training utility of large real-world datasets, enabling efficient model development and potentially supporting downstream research in governed domains such as healthcare. Trajectory matching (TM) is a widely used condensation approach that supervises synthetic data using changes in model parameters observed during training on real data, yet the structure of this supervision signal remains poorly understood. In this paper, we provide a geometric characterisation of trajectory matching, showing that a fixed synthetic dataset can only reproduce a limited span of such training-induced parameter changes. When the resulting supervision signal is spectrally broad, this creates a conditional representability bottleneck. Motivated by this mismatch, we propose Bezier Trajectory Matching (BTM), which replaces SGD trajectories with quadratic Bezier trajectory surrogates between initial and final model states. These surrogates are optimised to reduce average loss along the path while replacing broad SGD-derived supervision with a more structured, lower-rank signal that is better aligned with the optimisation constraints of a fixed synthetic dataset, and they substantially reduce trajectory storage. Experiments on five clinical datasets demonstrate that BTM consistently matches or improves upon standard trajectory matching, with the largest gains in low-prevalence and low-synthetic-budget settings. These results indicate that effective trajectory matching depends on structuring the supervision signal rather than reproducing stochastic optimisation paths.