Embedding Foundation Model Predictions in Discrete-Choice Models with Structural Guarantees
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "Embedding Foundation Model Predictions in Discrete-Choice Models with Structural Guarantees".
Title and authors: Tom: So we're looking at this paper today, "Embedding Foundation Model Predictions in Discrete-Choice Models with Structural Guarantees." It sounds like they are taking these big foundation models—these massive AI systems—and trying to fit them into a structure that actually makes economic sense for choice prediction.
Jane: That's right, Tom. The core issue they point out is that these tabular foundation models can be very accurate at predicting choices, but the predictions often just don't follow the basic rules of economics. Like if you raise a price, demand should go up, but sometimes their estimates are totally illogical or they predict something shouldn't happen.
Lu: What they propose is this two-stage adapter. They take the probability vector that the foundation model spits out—that’s all those choice probabilities—and they embed those into a multinomial logit utility function. That's how it tries to force the prediction to respect economic logic.
Meng: And then they do this in two steps, first fitting the core structural coefficients under some constraints, and then they freeze those and fit a small neural correction on top of the foundation model's predictions. It’s like building a rigid economic framework first and then letting a tiny AI tweak the predictions slightly to match what it saw.
Lalam: The main theoretical claim here is that this two-stage setup exactly preserves the marginal rate of substitution in the multinomial logit. So, analytically computable things, like value of time, become a mathematical guarantee instead of just some messy empirical accident.
Tom: Exactly! They proved that if you use this two-stage procedure correctly, the structure holds up perfectly for those structural guarantees. But they also pointed out that if you try to do both steps at once—joint training—it breaks identifiability, meaning it gets weird and can't be reliably trained.
Jane: So what’s the practical result of this? The paper shows that this adapter, when used across three different datasets and two different foundation models, actually gains about six point four percent in test accuracy compared to just using a standard multinomial logit.
Lu: That accuracy gain is significant, but it's important to note that they maintained one hundred percent cost monotonicity across all these tests. They also made sure the resulting values of time fall within the range published by transportation economics on those specific datasets.
Meng: For me, the fact that it maintains cost monotonicity is crucial when we actually deploy this stuff at scale. We don't want a model that suddenly starts predicting things that are logically impossible for a business to operate under.
Title and authors: Lalam: And they show that even when you test against counterfactual cost changes, the adapter and the standard multinomial logit agree on the direction of share change in every single scenario. The adapter recovers forty to ninety-seven percent of the magnitude of those share changes that we would normally expect from a good model.
Tom: That's what I like to see. We’ve seen foundation models get good accuracy, but sometimes they just don't respect the underlying incentives of the decision-making process. This paper gives us a way to audit those predictions against known economic constraints using the structural properties of the multinomial logit itself.
Jane: So, what does this mean for someone who isn't an AI expert? It means that when we use these models for things like predicting how people will choose a certain route or product, we can now have a mathematical guarantee that our estimates adhere to basic economic trade-offs.
Lu: And it’s model agnostic too. The architecture is designed so it can take any predict function whose output probability vector respects the precomputed structure they set up. It essentially slots into an existing foundation model without needing to change the foundation model's internal training process at all, which is a big win for research flexibility.
Meng: From a deployment standpoint, if we use this adapter, we can use it to audit our models for violations like monotonicity or trade-off ratios. That audit pipeline lets us check compliance before we push anything live. That kind of built-in quality control is something I’d actually want to see implemented in production systems.
Lalam: The paper also mentioned some limitations on calibration, though they noted that the expected calibration error is competitive with the standard multinomial logit on two specific datasets, but it stays elevated on Swissmetro because of bias-driven issues. They also pointed out that for discrete attributes, if you want to check binary indicators, you need a specific protocol involving a discrete flip because small perturbations don't work well with the float32 precision.
Tom: So there are trade-offs. You get this structural validity and accuracy boost, but the calibration might be slightly off on certain data types or in certain scenarios. We have to be careful about where we apply this solution for maximum reliability.
Jane: It really highlights that simply getting high accuracy from a foundation model isn't enough if you need that prediction to actually make sense in the real world, especially when policy decisions are at stake. This paper gives us a tool to bridge that gap between raw prediction power and economic reality.
Lu: Overall, this work provides a solid framework for embedding complex AI outputs into established economic theory in a way that preserves those core structural properties. It’s about using the structure of the model we *want*—the multinomial logit—to constrain the predictions of the model we *have*—the foundation model.
Title and authors: Meng: So, to wrap up on practical terms, it seems like this is a way to gain about six point four percent more accuracy while adding a layer of provable economic consistency and auditability that we don't have when we just run the raw prediction. It’s an interesting path forward for production systems where economic logic matters.
Lalam: That's the gist of it. The paper, "Embedding Foundation Model Predictions in Discrete-Choice Models with Structural Guarantees," shows that a two-stage adapter can embed foundation model predictions inside a constrained multinomial logit utility, and this preserves the marginal rate of substitution exactly, while recovering most of the accuracy advantage.
Tom: And it shows that joint training fails because correction expressivity creates those one-parameter families of likelihood-equivalent solutions that break identifiability. So, stick to the two-stage approach for reliable results.
Jane: It’s a lot of machinery, but fundamentally, they are proving that we can make AI predictions more trustworthy by tethering them to established economic theory in a mathematically sound way.
Lu: Exactly. It positions the foundation model differently—not as the final decision-maker, but as an explanatory feature whose parameters never get modified, which is a very clean way to integrate these two different modeling paradigms.
Meng: For deployment, it’s about getting that audit capability. That's a strong point for me because it means we can actually test the model’s behavior against real-world constraints before we let it influence any decisions.
Lalam: And if you look at the performance across those three datasets and two foundation models, the adapter gained six point four percent on average over the standard multinomial logit, and maintained one hundred percent cost monotonicity.
Tom: So, to sum up for the listener who just wants the numbers: this adapter gives you a measurable boost in accuracy while keeping you within established economic bounds.
Jane: It's a significant contribution because it moves us away from just chasing high prediction scores and toward building systems that are structurally sound according to economic principles.
Lu: The implication is that we can use these large foundation models as powerful feature generators, constrained by the known structure of choice modeling, rather than trying to make them solve the entire economic problem from scratch.
Meng: I think for engineers in here, it’s a way to get the best of both worlds: leveraging the predictive power of massive models while keeping a hard structural guardrail on top.
Lalam: So that’s what this paper is all about: embedding foundation model predictions into a constrained utility model with two stages, proving that it preserves the marginal rate of substitution exactly, and showing how to recover most of the accuracy gain.
The paper's summary: Tom: So, we've seen that these big foundation models can get pretty accurate at predicting choices—like what someone might buy or where they might go—but those predictions often just don't follow the basic rules of economics, and this paper shows how to fix that structural problem.
Jane: Exactly. The core idea is taking the probability outputs from a foundation model and embedding them inside a specific mathematical structure called a multinomial logit utility function, which basically forces the prediction to respect economic logic from the start.
Lu: What they do is two-stage training—first, they fit the core structural coefficients using maximum likelihood under some sign constraints, and then in the second stage, they freeze those coefficients and train a small neural correction network on top of what you already have.
Meng: And that’s where it gets interesting for us. They prove that this two-stage process exactly preserves the marginal rate of substitution in the multinomial logit structure. That means analytically computable things, like how much value time has, becomes a mathematical guarantee instead of just an empirical guess based on training data.
Tom: Which is huge because it means you're not just getting a better prediction score; you're getting a mathematically consistent one that respects fundamental trade-offs. They showed this composition works across three different datasets and two different foundation models.
Jane: And the results are pretty solid, they show the adapter gains about six point four percent in test accuracy on average over the standard multinomial logit, while still keeping one hundred percent cost monotonicity.
Lu: That accuracy gain is significant because it shows you can get most of the performance edge from these foundation models without losing all your economic structure. They also mentioned that this method produces values of time that stay within the range published by transportation economics on those specific datasets.
Meng: For deployment, maintaining cost monotonicity is a big win for me. We don't want a model that suddenly predicts things that are logically impossible for a business to operate under at scale.
Tom: So, to sum it up, this adapter gives you about six point four percent more accuracy while keeping you within established economic bounds and providing provable structural validity.
Jane: It really positions the foundation model differently, not as the final decision-maker, but as a feature whose parameters are constrained by established choice theory.
Lu: They also showed that trying to train the structural coefficients and correction network together at once breaks identifiability, which is why this sequential approach is essential for reliable results.
Meng: That failure mode with joint training is something we have to watch out for when we try to implement these kinds of hybrid systems in production.
Tom: So, the implication here is that we can use these massive foundation models as powerful feature generators, constrained by the known structure of choice modeling, rather than trying to make them solve the entire economic problem from scratch.
Jane: It’s a way for us to build more trustworthy AI systems where economic logic matters for policy decisions.
Tom: Next up, we're going to look at how this adapter can be used for auditing those foundation models against real-world constraints like trade-off ratios and availability compliance.
The paper's improvements: Tom: So, we've covered how they got that structural guarantee using this two-stage adapter to keep things within economic bounds, but what are the actual fixes or improvements they suggest for making this tool better?
Jane: Well, they point out a few things. One big thing is that the architecture is model agnostic. They say it can take any foundation model prediction, as long as its output probability vector respects the precomputed structure they set up. That’s a nice piece of flexibility.
Lu: And they also address the failure mode we saw earlier—the problem with joint training—by showing that you don't need to train everything together for it to work reliably. The two-stage process is the way to go for structural integrity.
Meng: From an engineering standpoint, what they emphasize is that this method allows us to do behavioral auditing of foundation models by embedding their predictions inside a multinomial logit utility. This ensures we get one hundred percent monotonicity and exact analytical trade-off ratios by construction.
Tom: That sounds like a major win for production systems because it means we can build in quality control right into the prediction pipeline, checking for violations before anything goes live.
Jane: Exactly. And they also highlight that they can use this to audit value of time against willingness to pay for arbitrary non-cost attributes or even cluster-aware aggregation for panel data, which opens up a lot of new ways to check model behavior.
Lu: The paper is very clear about the trade-off: you get this structural validity and accuracy boost at most two point two percent, and that gain over the standard multinomial logit is positive in ten out of ten bootstrap replicates on every single cell.
Meng: That means for practical purposes, we are paying only a tiny bit of accuracy for full structural validity by construction. That's actually pretty compelling when you think about the cost to run these complex models at scale.
Tom: So, what they suggest next is that this approach can be used to recover most of the foundation model’s accuracy advantage while retaining that hard structural validity we talked about before.
Jane: And it really shows that we can get the best of both worlds: leveraging the predictive power of massive models while keeping a hard structural guardrail on top for things like policy-relevant aggregates.
Lu: They also mentioned some calibration limitations, though they noted that the expected calibration error is competitive with the standard multinomial logit on two specific datasets but stays elevated on Swissmetro because of bias-driven issues.
Meng: It’s important to remember that there are still caveats, like how the joint training cost-coefficient collapse depends on how thoroughly the foundation model has internalized cost. That's a dataset-dependent thing we have to account for.
Tom: So, to summarize these improvements: it’s flexible because it works with any prediction function, it’s reliable because of the two-stage structure, and it gives us strong audit capabilities for behavioral consistency.
Jane: And if you look at the future work they hinted at, they are looking into how this framework can be integrated into broader systems that handle more complex decision-making where these structural guarantees are essential.
Lu: They’re clearly pushing toward using this as a foundational layer for how we constrain large language models to make them more predictable and economically sensible.
Meng: So, the implication is that we can start treating foundation models less like black boxes and more like features constrained by known economic principles for critical applications.
Tom: Next up, we're going to look at some of these other papers that are dealing with how foundation models handle creative tasks versus grounded reasoning.
Conclusion: Tom: So we've covered how this two-stage adapter embeds foundation model predictions into a constrained utility model to preserve structural guarantees, and now we’re wrapping up what this paper means for the field.
Jane: Basically, they’ve shown that you can use these massive foundation models as powerful feature generators while still keeping them tethered to established economic theory in a mathematically sound way.
Lu: The big picture here is taking something that's super accurate at predicting choices and giving it a verifiable structure so we don't end up with predictions that violate the basic rules of economics.
Meng: For deployment, this means we can use these models for things where the stakes are high, like policy decisions or resource allocation, because we have a way to audit them against real-world constraints like cost monotonicity.
Lalam: I’m confident that this framework is reliable; it proves that by following the two-stage protocol of embedding predictions into a multinomial logit structure, you can exactly preserve the marginal rate of substitution.
Tom: That preservation of the marginal rate of substitution is what makes this method so unique; it turns something empirical into a mathematical guarantee for value-of-time calculations.
Jane: It’s about moving away from just chasing high prediction scores and toward building systems that are structurally sound according to economic principles.
Lu: The work on "Embedding Foundation Model Predictions in Discrete-Choice Models with Structural Guarantees" provides a clean way to integrate these two modeling paradigms without needing to retrain the entire foundation model from scratch.
Meng: I'm still watching how this scales; the paper mentioned that joint training of the structural coefficients and correction network doesn't preserve identifiability, so we have to stick to their two-stage setup for any production deployment.
Lalam: I’m sure we can use this framework to recover most of the accuracy advantage while keeping you within established economic bounds, provided you follow their two-stage training sequence correctly.
Tom: It shows that the foundation model is best used as a feature generator constrained by the known structure of choice modeling rather than trying to make it solve the entire economic problem from scratch.
Jane: It’s a solid contribution because it moves us away from just getting high accuracy scores and toward building systems that are structurally sound according to established theory.
Lu: The implications for representation learning are huge because they show how you can use existing powerful models as components within a much more robust, constrained structure.
Meng: For me, the real value is that we can now apply this kind of auditing pipeline to check model behavior against real-world constraints before anything gets pushed live.
Lalam: I’m certain that this paper sets a new standard for how we should be integrating large language models into decision-making systems with economic relevance.
Tom: So, that’s the wrap on "Embedding Foundation Model Predictions in Discrete-Choice Models with Structural Guarantees." It’s a great piece of work for anyone trying to bridge the gap between raw prediction power and real-world economic reality.
Jane: Next up, we're going to look at some papers dealing with how foundation models handle creative tasks versus grounded reasoning.
Yingshuo Wang, Xian Sun, Yanhang Li, Zhichao Fan, Zexin Zhuang
University of California, Berkeley · Duke University · Northeastern University · University of Illinois Urbana-Champaign · Southern Methodist University
cs.LG, econ.EM
Submitted: 2026-06-24
Updated: 2026-11-18
Comments: NeurIPS 2026 camera-ready version. 20 pages, 1 figure. Extends arXiv:2605.26559 (ICML 2026 FMSD Workshop). v2 adds evaluation on respondents and households unseen in training and a TasteNet baseline, runs all logit fits to convergence (results updated), and corrects references. Code: https://github.com/willw96/TabFM-Logit
License: http://creativecommons.org/licenses/by/4.0/
The gist: Tabular foundation models achieve strong accuracy on choice prediction tasks, but their predictions often violate the economic logic those tasks require.
Key concepts
- Two-Stage Adapter
- A two-step process where a foundation model's predicted probabilities are used as input for a constrained multinomial logit. Stage one fits structural coefficients, and stage two uses a small neural network to fine-tune the predictions, ensuring the final utility respects economic rules.
- Multinomial Logit Utility
- A mathematical framework used in choice modeling that represents an individual's decision as choosing among several options based on their perceived utilities. This paper embeds foundation model predictions into this utility structure to enforce economic consistency.
- Marginal Rate of Substitution (MRS)
- The rate at which consumers are willing to trade one good for another while keeping their overall satisfaction constant. The adapter is designed to exactly preserve the MRS, meaning the calculated value of time remains mathematically guaranteed and consistent with standard economic theory.
Terminology
Summary
Tabular foundation models achieve strong accuracy on choice prediction tasks, but their predictions often violate the economic logic those tasks require. p. 0 Embedding Foundation Model Predictions
in Discrete-Choice Models with Structural Guarantees
Yingshuo Wang1 Xian Sun2 Yanhang Li3 Zhichao Fan4 Zexin Zhuang5∗
Abstract
Tabular foundation models achieve strong accuracy on choice prediction tasks, but
their predictions often violate the economic logic those tasks require: raising a price
can increase predicted demand, implied willingness-to-pay estimates are frequently
negative or implausible, and unavailable alternatives receive nonzero probability. We propose a two-stage adapter that takes a foundation model’s predicted choice
probabilities as a precomputed feature and embeds them inside a multinomial
logit’s utility. Embedding Foundation Model Predictions
in Discrete-Choice Models with Structural Guarantees
Yingshuo Wang1 Xian Sun2 Yanhang Li3 Zhichao Fan4 Zexin Zhuang5∗
Abstract
Tabular foundation models achieve strong accuracy on choice prediction tasks, but
their predictions often violate the economic logic those tasks require: raising a price
can increase predicted demand, implied willingness-to-pay estimates are frequently
negative or implausible, and unavailable alternatives receive nonzero probability. We propose a two-stage adapter that takes a foundation model’s predicted choice
probabilities as a precomputed feature and embeds them inside a multinomial
logit’s utility. We propose a two-stage adapter that takes a foundation model’s predicted choice
probabilities as a precomputed feature and embeds them inside a multinomial
logit’s utility. In Stage 1, we fit the multinomial logit’s structural coefficients by
maximum likelihood with sign constraints; in Stage 2, we freeze those coefficients
and fit a small neural correction operating on the foundation model’s predictions. In Stage 1, we fit the multinomial logit’s structural coefficients by
maximum likelihood with sign constraints; in Stage 2, we freeze those coefficients
and fit a small neural correction operating on the foundation model’s predictions. We prove that this composition exactly preserves the multinomial logit’s marginal
rate of substitution, so analytically computable value-of-time becomes a mathematical guarantee rather than an empirical accident. We prove that this composition exactly preserves the multinomial logit’s marginal
rate of substitution, so analytically computable value-of-time becomes a mathematical guarantee rather than an empirical accident. Across three datasets and two
foundation models, the adapter gains 6. Across three datasets and two
foundation models, the adapter gains 6. 4 percentage points (pp) of test accuracy
on average over the multinomial logit and up to 12. Across three datasets and two
foundation models, the adapter gains 6. 4 percentage points (pp) of test accuracy
on average over the multinomial logit and up to 12. 4 percentage points (pp) of test accuracy
on average over the multinomial logit and up to 12. 8 pp, maintains 100% cost
monotonicity, and produces values of time within the published transportationeconomics range on the transportation datasets. 4 percentage points (pp) of test accuracy
on average over the multinomial logit and up to 12. 8 pp, maintains 100% cost
monotonicity, and produces values of time within the published transportationeconomics range on the transportation datasets.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Adapter Mechanism
The proposed two-stage adapter embeds the foundation model’s predicted probability vector inside a constrained multinomial logit as a precomputed feature. Stage 1 involves fitting the structural coefficients by maximum likelihood under sign constraints with the correction held at zero. p. 1 We propose a two-stage adapter that embeds the foundation model’s predicted probability vector
inside a constrained multinomial logit as a precomputed feature. In Stage 1, we fit the structural
coefficients by maximum likelihood under the usual MNL constraints with the correction held at zero;
in Stage 2, we freeze those coefficients and fit a small neural correction. We propose a two-stage adapter that takes a foundation model’s predicted choice
probabilities as a precomputed feature and embeds them inside a multinomial
logit’s utility. In Stage 1, we fit the multinomial logit’s structural coefficients by
maximum likelihood with sign constraints; in Stage 2, we freeze those coefficients
and fit a small neural correction operating on the foundation model’s predictions. where g : ∆K−1 → R
K is a small MLP (two hidden layers, width 32). To enforce cost/time
monotonicity throughout training we reparameterize each such coefficient as β = − exp(θ). In Stage 1, we fit the structural
coefficients by maximum likelihood under the usual MNL constraints with the correction held at zero;
in Stage 2, we freeze those coefficients and fit a small neural correction. We prove that this two-stage
procedure preserves the structural marginal rate of substitution exactly, and that joint training of the
structural coefficients and the correction does not: correction expressivity creates a one-parameter
family of likelihood-equivalent solutions, breaking identifiability.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Preservation of Structural Guarantees
The two-stage procedure is shown to preserve the structural marginal rate of substitution exactly under the fixed-q protocol. Proposition 1 proves that for any two attributes j, j′ that enter the model only through Vstructk(xi) = β⊤ϕk(xi), MRSj,j'(xi) ≡ β∗j/β∗j'. This means the value-of-time analogue β∗time/β∗cost is identical to the standalone multinomial logit’s value-of-time and is invariant to the choice of foundation model and to the choice of g. p. 3 For any two attributes j, j′
that enter the model
only through V
struct
k
(xi) = β
⊤ϕk
(xi), and for any xi at which ϕk
is differentiable in xij and xij′ ,
MRSj,j′ (xi) ≡
∂Vk(xi)/∂xij
∂Vk(xi)/∂xij′
=
β
∗
j
β
∗
j
′
. In particular, the value-of-time analogue β
∗
time/β∗
cost is identical to the standalone multinomial logit’s
value-of-time and is invariant to the choice of foundation model and to the choice of g.Where the paper says it
Failure Modes of Joint Training
Proposition 2 identifies that joint training of the structural coefficients and correction does not preserve structural identifiability. This failure mode is characterized by a one-parameter family of likelihood-equivalent solutions, breaking identifiability. p. 1 In Stage 1, we fit the structural
coefficients by maximum likelihood under the usual MNL constraints with the correction held at zero;
in Stage 2, we freeze those coefficients and fit a small neural correction. We prove that this two-stage
procedure preserves the structural marginal rate of substitution exactly, and that joint training of the
structural coefficients and the correction does not: correction expressivity creates a one-parameter
family of likelihood-equivalent solutions, breaking identifiability. Proposition 2 (Joint training breaks structural identifiability). Assume there exists a continuous
κk : ∆K−1 → R such that costk(x) = κk(q(x)) on the closure of the training support (costrecoverability assumption); the correction class G is dense in C
0
(∆K−1
, R
K) [Cybenko, 1989]; and
L(β, g) is minimized jointly without two-stage constraints.Where the paper says it
Where the paper says it
Empirical Performance and Audit
The adapter gains 6.4 percentage points (pp) of test accuracy on average over the multinomial logit and up to 12.8 pp, maintains 100% cost monotonicity, and produces values of time within the published transportationeconomics range on the transportation datasets. The audit pipeline allows for evaluating any predict-function by perturbing attributes and checking for violations like Monotonicity, Trade-off ratio, and Availability compliance. p. 8 The audit’s 1%-perturbation recipe is sound for continuous attributes but produces float32-rounding-precision deltas on IoT-Wearables’ binary indicators
(function and label) in Mitra cells. We use a discrete-flip protocol instead (set the indicator to its
complement, sign-correct the per-row delta), giving finite trade-off values: function flip-WTPs
span −0. 2 The adapter recovers behavioral validity while keeping most of the accuracy gain
The adapter inherits the multinomial logit’s structural utility (with a sign-constrained cost coefficient)
and availability-mask machinery, so it satisfies 100% monotonicity, exact analytical trade-off ratios,
and zero availability leak by construction on every cell. The empirical question is how much of the
foundation-model accuracy gain the adapter retains.Where the paper says it
Where the paper says it
Counterfactual Evaluation
Under counterfactual cost perturbations, the structural multinomial logit and the adapter agree on the direction of share change in every scenario. The adapter share-changes recover 40–97% of the multinomial logit’s magnitude depending on cell, whereas raw foundation models show per-observation monotonicity violations propagating to policy-relevant aggregates in 6 of 16 scenarios. p. 6 Adapter share-changes recover 40–97% of
the multinomial logit’s magnitude depending on cell (Table 2); the gap reflects softmax saturation
under the foundation-model correction, not a violation of Proposition 1. The raw foundation-model row uses the recomputed-q protocol (re-running the foundation model on
perturbed inputs) and shows the per-observation monotonicity violations of Section 5.Where the paper says it
Calibration and Limitations
The adapter’s post-calibration Expected Calibration Error (ECE) is competitive with the structural multinomial logit on LPMC and IoT-Wearables, but it stays elevated on Swissmetro due to bias-driven limitations. The audit methodology for discrete attributes requires a discrete-flip protocol for binary indicators because a 1% perturbation is too small for float32 precision. p. 8 The audit’s 1%-perturbation recipe is sound for continuous attributes but produces float32-rounding-precision deltas on IoT-Wearables’ binary indicators
(function and label) in Mitra cells. We use a discrete-flip protocol instead (set the indicator to its
complement, sign-correct the per-row delta), giving finite trade-off values: function flip-WTPs
span −0. Proposition 2 is dataset-dependent. The joint-training cost-coefficient collapse (ablation A3, §6)
reproduces on Swissmetro and LPMC and does not on IoT-Wearables.Where the paper says it
Where the paper says it
Conclusion
Discrete-choice models guide policy decisions whose economic stakes are large and whose failure modes are not subtle. We propose a two-stage adapter that embeds foundation model predictions inside a structurally constrained utility model. p. 8 Tabular foundation models raise predictive accuracy beyond what structural
utility models alone reach, but their predictions can violate basic economic logic: monotonicity in
cost, sensible willingness-to-pay, zero probability on unavailable alternatives. We propose a two-stage
adapter that embeds foundation-model predictions inside a structurally constrained utility model. We propose a two-stage
adapter that embeds foundation-model predictions inside a structurally constrained utility model. Proposition 1 shows that the two-stage procedure preserves the structural model’s marginal rate of
substitution exactly; Proposition 2 shows that joint training does not. datasets and two foundation models, the adapter pays at most 2 pp of accuracy for full structural
validity by construction, with the gain over the structural multinomial logit positive in 10 of 10
bootstrap replicates on every cell. The architecture is model-agnostic and slots in any predict-function
whose probability output respects the precomputed-q contract. datasets and two foundation models, the adapter pays at most 2 pp of accuracy for full structural
validity by construction, with the gain over the structural multinomial logit positive in 10 of 10
bootstrap replicates on every cell. The architecture is model-agnostic and slots in any predict-function
whose probability output respects the precomputed-q contract.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Two-stage adapter implementation allows for behavioral auditing of foundation models by embedding predictions inside a multinomial logit utility, ensuring
100% monotonicity
andexact analytical trade-off ratios
by construction. -
The system can recover up to 6.4 percentage points (pp) of test accuracy on average over the structural multinomial logit when using the adapter, while maintaining structural guarantees and producing values of time within the published transportation economics range on those datasets.
-
The architecture is model-agnostic, meaning it can be applied to any predict function whose probability output respects the precomputed q contract, allowing for
value-of-time auditing to willingness-to-pay for arbitrary non-cost attributes and to cluster-aware aggregation for panel data.
Abstract
Tabular foundation models achieve strong accuracy on choice prediction tasks, but their predictions often violate the economic logic those tasks require: raising a price can increase predicted demand, implied estimates of willingness to pay are frequently negative or implausible, and unavailable alternatives receive nonzero probability. We propose TabFM-Logit, an adapter that takes a foundation model's predicted choice probabilities as a precomputed feature and embeds them inside a multinomial logit's utility. In Stage 1, we fit the multinomial logit's structural coefficients by maximum likelihood with sign constraints. In Stage 2, we freeze those coefficients and fit a small neural correction operating on the foundation model's predictions. We prove that this composition exactly preserves the multinomial logit's marginal rate of substitution, so the analytically computable value of time becomes a mathematical guarantee rather than an empirical accident, and that training the two stages jointly breaks identifiability of the structural coefficients. Counterfactual derivatives and response signs are therefore the multinomial logit's by construction, and the foundation model contributes accuracy on observed choices. On respondents and households unseen in training, the adapter gains 5.7-6.8 percentage points (pp) of test accuracy on Swissmetro, 1.5 pp on LPMC, and 3.3-3.6 pp on IoT-Wearables over the multinomial logit (4.2 pp on average), stays within 1.0 pp of the raw foundation model, maintains 100% cost monotonicity, and keeps the multinomial logit's values of time exactly. The gain is largest where the multinomial logit is weakest and smallest where it is already strong. In every case it comes with the multinomial logit's guarantees, which the raw foundation models lack.
Sources
- Distilling the Knowledge in a Neural Network
- Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models
- Advancing Constrained Monotonic Neural Networks: Achieving Universal Approximation Beyond Bounded Activations
- Auditing and Fixing Economic Validity in Tabular Foundation Models for Discrete Choice
Related papers
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis
- (Mis)generalization of Helpful-Only Fine-tuning
- Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refine