CoRTeC: A Differentially Private Release for Synthetic Tabular Data, Decodable by Any Generator

summary

The gist

The records that could train the most useful models are often those an institution cannot share, and CoRTeC proposes a method to release synthetic versions of such records under differential privacy…

In short

CoRTeC proposes a method to release synthetic versions of sensitive institutional records while protecting privacy using differential privacy. It spends one fixed budget to generate noisy counts for training classifiers, allowing reviewers to inspect every number before release. This technique achieves strong privacy guarantees while maintaining high utility for model training.

Key concepts

Differential Privacy
A mathematical framework ensuring that the output of a computation does not reveal whether any single individual's data was included in the input. It provides a formal guarantee against membership attacks, meaning an adversary cannot reliably determine if a specific record is present in the released synthetic data.
Privacy Budget
The total amount of privacy loss allowed when releasing information. CoRTeC spends this budget once on a small file of noisy counts. The mechanism is designed so that the strongest possible membership attack remains far below this allocated budget, ensuring robust protection against adversaries.
Decoding Methods
There are two ways to reconstruct the original data from the released counts. One method uses a maximum-entropy decoder or a frozen language model to sample rows. The second method involves matching these sampled rows back to the released counts and redrawing values within their specified bins, allowing for reconstruction without fine-tuning models.
Utility vs. Privacy Trade-off
The paper demonstrates that CoRTeC achieves high utility—training classifiers within 0.02 AUC of real samples on certain datasets—while adhering to strict privacy constraints. The key is that the release itself trains the model; decoded rows carry conditional rates more accurately than a real sample of the same size.

Terminology used across episodes

This episode discusses

The paper

CoRTeC: A Differentially Private Release for Synthetic Tabular Data, Decodable by Any Generator · Read on arXiv

Calister Nnona

The records that could train the most useful models, such as a hospital's patient files, are often the ones an institution cannot share. We propose CoRTeC, a method that releases synthetic versions of such records under differential privacy. Differential privacy places a proven limit on what the synthetic data can reveal about any person, and in particular on membership inference. CoRTeC spends the privacy budget once, on a small file of noisy counts designed for training a classifier. For each group of records, the file gives the group's size, its balance of outcomes, a histogram of every column for each outcome and the outcome rate in each of several non-overlapping cells. A reviewer can check every number before the file leaves the institution. Anyone can then turn the counts into synthetic rows, as often as they like and at no further privacy cost. A maximum-entropy decoder can do this without any language model, and so can each of the five frozen language models we tested. None of them ever sees a real record. A selection step then matches the rows to the released counts. Decoded without a language model, the automatically configured release at epsilon = 2 trains classifiers to within 0.02 AUC of a real sample of the same size on the Adult dataset, across three data splits. On NHANES and bank marketing, without the selection step, it stays within 0.05 AUC. On Adult, with a fixed cohort rule, the release carries the outcome rates a deployment names more closely than a real sample does, at some cost in accuracy on pairs of columns. Four controls show that most of the utility comes from the release and the selection step, and that a language model adds what it knows from the column names. No private row ever reaches a model, and no row CoRTeC generates copies a real one. The strongest membership attack stays far below the bound the budget allows.

DOI: 10.5281/zenodo.23812612

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: I'm Nadia.

Elias: And I'm Elias, here with Priya.

Nadia: We're looking at the paper: "CoRTeC: A Differentially Private Release for Synthetic Tabular Data, Decodable by Any Generator".

Elias: Here's the gist of the paper. The records that could train the most useful models are often those an institution cannot share, and CoRTeC proposes a method to release synthetic versions of such records under differential privacy The strongest membership attack stays far below the bound the budget allows CoRTeC Mechanism and Privacy Budget CoRTeC spends the privacy budget once, on a small file of noisy counts designed for training a classifier For each group of records, the file gives the group's size, its balance of outcomes, a histogram of every column for each outcome and the outcome rate in each of several non-overlapping cells A reviewer can check every number before the file leaves the institution The workload is fixed before the release is computed, either by a public rule or by a charged column ranking whose output is itself released What leaves the institution is a file of counts that anyone can inspect cell by cell The mechanism satisfies pure ε-differential privacy with a five-step proof Decoding Methods There are two ways of decoding the release (Figure 1) Stage B decodes R with either the maximum-entropy decoder or a frozen language model and passes the decoder’s output through a selection step A maximum-entropy decoder can fit the released counts and sample rows from them Alternatively, a frozen language model can decode the same release The selection step then takes the rows from either decoder, matches them to the released counts and redraws values inside their released bins Utility and Performance Decoded without a language model, the release trains classifiers to within 0.02 AUC of a real sample of the same size on the Adult dataset, across three data splits On NHANES and bank marketing, without the selection step, it stays within 0.05 AUC With the fixed cohort rule and the selection step, the decoded rows carry the conditional rates the release names more closely than a real sample of the same size does Contributions to Tabular Data Synthesis The contributions include: 1.

Nadia: A release built around what a classifier needs (Section 4.1) Stage A spends the whole budget once, on class-conditional histograms within cohorts formed by a public rule or a charged column ranking and on a conditional outcome-rate table, all fixed before the release is computed 2.

Elias: A frozen language model as a decoder of that release (Section 4.2) The model is prompted with exact per-bin, per-category and per-class counts derived from the release and never sees a record 3.

Nadia: Utility without fine-tuning (Section 5) Decoded without fine-tuning any model, the release trains classifiers close to a real sample of the same size: within 0.02 AUC on Adult across three splits with the maximum-entropy decoder Privacy Architecture and Limitations The threat model states that the adversary has unlimited computing power and sees everything that crosses the trust boundary The privacy model guarantees pure ε-differential privacy (δ = 0) in the central model under add/remove adjacency The strongest advantage is at most 0.035 on every Adult configuration and 0.026 to 0.060 across the NHANES configurations, against the 0.762 that the budget permits Limitations include fixed workload costs accuracy on pairs of columns, which the reference AIM fits more closely on Adult and NHANES (Section 5.4) The gap to a real sample is largest on the medical dataset, at up to 0.049 AUC across three splits (0.068 on the seed-42 split alone) Experimental Results The results replicate over three splits The decoded rows carry the conditional rates it names more closely than such a sample does The main argument for the design is its privacy architecture: the model never sees a row, every draw is post-processing and the backbone can change without touching the guarantee The measured membership advantage of the CoRTeC draws, 0.02 to 0.05, is a small fraction of the 0.762 the budget permits For a regulated institution, the product is the release itself Decoded without a language model, the release trains models within 0.02 AUC of a real sample of the same size on Adult and within 0.05 on NHANES and bank marketing CoRTeC is meant to sit beside the marginal methods, not to replace them For a deployment, we recommend decoding with the maximum-entropy decoder and the selection step first, and adding a language model only where the model knows the schema The release itself takes seconds, and the maximum-entropy decoder runs on a CPU Reproducibility Statement The repository (Calyie/cortec) holds the Apache-2.0 reference implementation with its tests, the research pipeline that produced every CoRTeC configuration, the paper and the frozen auto-configurations Released statistics cannot be regenerated bit for bit, because the noise comes from a cryptographically secure source that ignores any seed The reference AIM wrapper, the maximum-entropy decoder with and without the selection step, the one-shot control, the anonymisation of releases and the scorers are all in the repository Ethics Statement This work uses public benchmark datasets and a national health survey released for research No private institutional data was used The method is meant to reduce the privacy risk of sharing tabular data Compute and Cost The Qwen3 backbones and the DP fine-tuning baselines ran on rented NVIDIA A40 GPUs, using vLLM

45: for generation and Opacus

41: for DP-SGD The named and anonymised Gemini 3.6 Flash pairs cost 9.37 dollars in total, the memorisation test 0.33 dollars and the one-shot control 2.

Paper summary: Nadia: So we're looking at CoRTeC today. The paper is titled "CoRTeC: A Differentially Private Release for Synthetic Tabular Data, Decodable by Any Generator". It claims they've developed a method to release synthetic versions of records under differential privacy.

Elias: What that means is they're tackling the problem of releasing sensitive data, like hospital patient files, that institutions can't share without risking privacy. The thesis here is that you can release synthetic data while still keeping a proven limit on what any person can learn about the original records through something called membership inference.

Priya: It sounds like they are trying to find a way to create data that is useful for training models but doesn't leak private information about the individuals in those records. I wonder if this synthetic data is actually good enough for real-world applications, or if it's just theoretical noise.

Nadia: Well, the paper describes how CoRTeC spends the privacy budget once on a small file of noisy counts designed specifically for training a classifier. This file gives you things like the group size of records, its balance of outcomes, and a histogram for every single column for each outcome.

Elias: And then it also gives you the outcome rate in several non-overlapping cells, and a reviewer can check every number before that file leaves the institution. That's their initial release mechanism.

Priya: The key thing I noticed is that this whole process is fixed before the release is even computed, either by a public rule or by some charged column ranking whose output gets released too. It means the workload doesn't just discover things round by round.

Nadia: Right. And what leaves the institution isn't the actual records, but a file of counts that anyone can inspect cell by cell. The mechanism satisfies pure epsilon differential privacy with a five step proof, which is pretty rigorous for what they’re doing.

Elias: Then in stage B, you have two ways to decode this release. You can use either the maximum-entropy decoder or a frozen language model to turn those statistics into synthetic rows.

Priya: And then there's this selection step that matches those generated rows back to the original released counts, effectively redrawing values inside their released bins. It sounds like they are trying to be very precise about how much information gets out and how it can be used afterwards.

Paper summary: Nadia: The utility results show that if you decode without a language model, the release trains classifiers within point zero point two AUC of a real sample of the same size on the Adult dataset across three data splits. That’s pretty solid performance for synthetic data.

Elias: They also tested this on NHANES and bank marketing, where without that selection step, it stays within point zero five AUC. But when you use the fixed cohort rule and that selection step, the decoded rows carry conditional rates that are more closely related to a real sample of the same size than what is in a real sample itself.

Priya: That makes sense. So what's important for someone just listening to this show is that they get synthetic data that performs almost as well as real data for training purposes, but the core value here seems to be in how it handles privacy guarantees during the release process itself.

Nadia: The main contribution they highlight is building a release around what a classifier actually needs, which means Stage A spends the whole budget once on class-conditional histograms within cohorts formed by that public rule or charged column ranking and on that conditional outcome rate table, all fixed before the final release is computed.

Elias: They also introduce this frozen language model as a decoder of that release, which is prompted with exact per-bin, per-category and per-class counts derived from the release but it never sees a real record. That’s a clever way to inject knowledge without seeing the raw data.

Priya: So you're saying the utility comes from this combination—the release itself plus that selection step—and that adding a language model just helps with what it already knows about column names, rather than providing some kind of magic inference.

Nadia: Exactly. And they show that decoding without fine-tuning any model still trains classifiers close to a real sample, achieving within point zero two AUC on the Adult dataset across three splits using the maximum-entropy decoder alone.

Elias: But they also have to be careful about what this method doesn't do. The paper states that accuracy on pairs of columns is not as good, and the reference AIM fits more closely on Adult and NHANES datasets when you look at those specific comparisons.

Priya: And where does it fall short most? They flag that the gap to a real sample is largest on the medical dataset, reaching up to point zero four nine AUC across three splits, which is quite a bit of room for improvement there.

Paper summary: Nadia: The paper also details their privacy architecture. It states that the measured membership advantage of the CoRTeC draws, which they measure as point zero two to point zero five, is a small fraction of the privacy budget that they allow at point zero seven six two across the Adult configurations and point zero six to sixty across the NHANES configurations.

Elias: So, from a security standpoint, it seems robust. The strong advantage they measured against the budget that permits is relatively small, which is good for maintaining privacy guarantees under the threat model they set up.

Priya: But I want to ask about those limitations again because if you're trying to use this in a high-stakes medical setting, that gap of point zero four nine AUC on the medical data might be too big. What does that mean practically for someone building a diagnostic tool?

Nadia: It means the method is still under development and has room for refinement, especially when you move from general datasets like Adult to more complex ones like medical records. The paper also mentions they provide a reference implementation and a deployment guide, which ships both decoders and includes checks for things like memorisation tests.

Elias: And they also mention that the API only methods of Swanberg et al. did not improve on the marginal baselines, and that those authors concluded fine-tuning is necessary for tabular data, which puts CoRTeC in a different category than just training a generator under the budget.

Priya: So to sum up what we've heard about CoRTeC: it’s a method that releases synthetic data under differential privacy by spending the budget once on counts, and decoding is free with two main options, one of which is using a frozen language model.

Nadia: And the real utility comes from the release itself and the selection step, which can make the decoded rows carry conditional rates that look very much like a real sample. It’s designed to sit beside marginal methods rather than replacing them entirely.

Elias: The implication is that institutions might be able to share data in a way that offers utility for training models while keeping the privacy risk low, provided they stick to the release itself as the core product and use the maximum-entropy decoder with a selection step for deployment.

Conclusion: Nadia: So CoRTeC is about taking these records that institutions can't share and releasing synthetic versions under differential privacy so that you can still train models safely

Elias: Yeah, the authors are talking about a way to do this release process in a controlled way by spending the privacy budget once on some noisy counts

Priya: And they show how you can actually decode those counts into rows that are pretty useful for training classifiers without seeing the actual data itself

Nadia: It’s interesting because it's not just about making synthetic data; it’s about making sure the release mechanism itself is private and that the resulting data still trains models well

Elias: The main point they are driving home is that you can build a system around what a classifier needs, fixing everything up front before anything leaves the institution

Priya: So for someone who just listens to this, it means there's a method to release sensitive tabular information that doesn't just throw away privacy but actually keeps some real training utility

Nadia: Right. They show that you can get accuracy close to a real sample on datasets like Adult and NHANES without needing massive amounts of private data

Elias: But they’re clear about where it still falls short, especially when you look at more complex medical data where the gap between the synthetic training and the real thing is bigger

Priya: That gap is important. It means if you're doing something high-stakes, you can't just assume this method works perfectly across every single dataset

Nadia: Exactly. So CoRTeC isn't a magic solution that fixes everything, it’s a tool that sits beside other methods to help institutions share data more safely

Elias: It gives us a framework for how privacy budget spending can be structured when the goal is generating usable synthetic data instead of just hiding things

Priya: So while they show good numbers, the real question is how robust this entire architecture is when you push it into a real-world deployment scenario

More episodes

← Home