Marrying Pricing and Advertising with LLMs

arXiv:2610.09985 · cs.GT, cs.LG · Submitted 2026-10-07 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Marrying Pricing and Advertising with LLMs".

Jane: The gist:

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we've seen how this online actor-critic algorithm works, combining LoRA on an LLM for ads and a demand model to guide price selection based on purchase feedback > Page four ---

Jane: And the key finding is that this joint optimization framework leads to measurable increases in expected revenue compared to just optimizing price or just optimizing ad content separately.

Lu: The paper shows that you can marry the two decisions—price and advertisement—in a sequential setting where demand is unknown, using LLMs as the mechanism for generating those ads > Page two ---

Meng: From my side, what I see is that it's a solid framework for testing how AI can handle multi-objective optimization when there are complex, coupled feedback loops > Page three ---

Tom: Exactly. The implication is that if you have data to fit a demand model and an LLM that can generate relevant content, you can jointly optimize revenue in these types of dynamic scenarios.

Jane: It moves the conversation from just one side of the coin—like just setting a price—to understanding how creative choices, like an ad, fundamentally change the pricing landscape > Page one --- <ref:2610.09985#pg1>

Lu: The paper does suggest that future work could look at making this system robust when demand shifts dramatically over time, as tracking that moving target is hard > Page four ---

Meng: And from a deployment angle, they'll need to prove that the online adaptation process doesn't introduce instability when the marketplace gets very busy and data streams become unpredictable > Page four ---

Tom: That’s the practical stuff. It’s not just about getting a high number on a test set; it’s about making sure this system stays stable when you deploy it live > Page one --- <ref:2610.09985#pg1>

Conclusion: Tom: So we’re wrapping up our look at "Marrying Pricing and Advertising with LLMs," this paper is about how you can use a large language model to handle both setting a price and picking an ad at the same time when you don't know what customers will do.

Jane: Right, so the main idea is taking something that generates ads, like an LLM, and figuring out how to pair it with pricing in this online selling thing, where you only get feedback after someone buys or not.

Lu: The authors are looking at sequential posted pricing problems where the seller has to pick a price and an ad based on some product context before seeing if the buyer actually takes the deal.

Meng: From what I see, it’s trying to solve that tricky joint optimization problem where you have two decisions—price and creative content—that both affect revenue in a dynamic setting.

Tom: Exactly, they use an online actor-critic method where one part of the system is this LLM fine-tuned with low-rank adaptation just for making those ads.

Jane: And the other part, the critic, that’s a demand model fitted to whatever data you have to estimate the purchase probability given a price and an ad.

Lu: The methodology involves the actor generating an advertisement in batches, and then the seller choosing a price guided by what that critic predicts will bring in revenue.

Meng: What I’m thinking is how they handle that uncertainty—the exploration part where you sometimes pick a random price just to see what happens—and how they balance getting better ads with finding the right price.

Tom: They use this leave-one-out baseline for their actor updates, which is interesting because it means the gradient update doesn't rely on an exact guess of the reward, which reduces some of that noise from individual purchase outcomes.

Jane: So what does this actually change for someone who just listens to the show? It suggests that a model can learn to create a better ad and set a better price simultaneously, instead of treating them as separate problems.

Lu: The results show these combined policies actually increase expected revenue across different demand scenarios, like synthetic ones and even some real-world marketplace data.

Meng: But the caveat is that it’s still an online process, so we have to worry about how stable this whole system is when the market feedback keeps changing rapidly.

Tom: It’s a solid framework for testing how you can couple generation with optimization in these kinds of sequential decision-making tasks.

Jane: And the paper points toward making this even more interesting by looking at what happens if demand itself starts to change over time, which is a big challenge for any system like this.

Alessandro Barro Francesco Bacchiocchi, Francesco Emanuele Stradi Alberto Marchesi

Politecnico di Milano

cs.GT, cs.LG

Submitted: 2026-10-07

Updated: 2026-10-07

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 83/100

The gist: The gist: We propose an online actor-critic algorithm that combines low-rank adaptation (LoRA) of a pretrained LLM with a demand model fitted to available data to jointly optimize sequential pricing

Key concepts

Sequential Pricing Problem: This is a scenario where a seller must choose both a price and an advertisement before observing the final purchase outcome. The goal is to maximize total revenue across multiple sequential interactions, which is challenging because demand remains unknown.
Actor-Critic Algorithm: The system uses two components: an 'actor' that generates advertisements (using an LLM) and a 'critic' that estimates the expected revenue based on the current price and ad. The actor tries to maximize this estimated revenue, while the critic learns to accurately predict purchase probabilities.
Low-Rank Adaptation (LoRA): LoRA is a technique used to efficiently fine-tune large pre-trained Language Models (LLMs) for generating advertisements. Instead of retraining the entire massive model, LoRA injects small, trainable matrices into the model's layers, allowing it to adapt quickly while preserving the original knowledge.

Summary

The gist: We propose an online actor-critic algorithm that combines low-rank adaptation (LoRA) of a pretrained LLM with a demand model fitted to available data to jointly optimize sequential pricing and advertisement generation for maximum revenue under unknown product demand.

Problem Formulation

The study addresses the sequential posted pricing problem where a seller must jointly post a price and an advertisement, observing only the purchase feedback, in which the buyer purchases if their private valuation exceeds the price <ref:2610.09985#pg2> The seller must specify not only a price p ∈ [0, 1], but also an advertisement x ∈ X, where X denotes the space of possible advertisements shown to buyers <ref:2610.09985#pg5> Each offer is associated with a context vector z ∈ Z representing characteristics of the product and the environment, which the seller observes before choosing the advertisement and price <ref:2610.09985#pg4> This leads to a demand function d: [0, 1] × X × Z → [0, 1] to model buyer response, where d(p, x, z) is the probability of purchase at price p, given advertisement x and context z <ref:2610.09985#pg3>

Online Actor-Critic Algorithm

The proposed algorithm utilizes an online actor-critic approach that combines adaptation of a pretrained LLM with demand estimation <ref:2610.09985#pg4> The actor is a pretrained LLM fine-tuned through low-rank adaptation (LoRA) to generate advertisements, while the critic is a demand model fitted to available data that estimates purchase probabilities given price, advertisement and product-level context <ref:2610.09985#pg2> At each round, the actor generates an advertisement, and the critic estimates purchase probabilities to guide price selection <ref:2610.09985#pg4> The seller selects a price through an exploration rule guided by the critic <ref:2610.09985#pg2>

The algorithm operates in batches of equal size indexed by τ, where the actor is adapted through low-rank adaptation (LoRA) to generate advertisements, while the critic is a demand model fitted to available data that estimates purchase probabilities given price, advertisement and product-level context <ref:2610.09985#pg2> For each advertisement generated in batch τ, the seller selects a price through an exploration rule guided by the critic <ref:2610.09985#pg4> With probability ετ, the price is sampled from a given exploration distribution ντ on [0, 1] <ref:2610.09985#pg4> Otherwise, the seller selects a price maximizing the critic’s estimate of expected revenue <ref:2610.09985#pg4> The observed purchase outcomes are used to update the critic’s demand estimates and the actor’s generative policy for the advertisement <ref:2610.09985#pg4>

The actor updates its LoRA parameters through policy gradient steps to maximize an objective that balances expected revenue with a KL penalty for deviations from the fixed pretrained reference model <ref:2610.09985#pg4> These updates use a leave-one-out baseline that averages the critic’s predicted revenues instead of observed rewards, reducing the noise introduced by individual purchase outcomes <ref:2610.09985#pg4>

Experimental Results

The approach is evaluated under three synthetic demand models and a demand simulator built from real-world marketplace data <ref:2610.09985#pg5> Comparing the Trained and Reference policies, expected revenue increases by 5.69%, 5.18% and 55.96% under the three synthetic demand models and 5.81% under the marketplace simulator <ref:2610.09985#pg4> The largest gain is fully explained by the increased frequency of the rewarded content feature under the declared demand mechanism <ref:2610.09985#pg5> Under the marketplace simulator obtained from real data, average normalized expected revenue increases by 5.81%, with improvements on 91 of 100 products across five categories <ref:2610.09985#pg4>

Related Work

Sequential pricing with unknown demand has been extensively studied in online learning and revenue management <ref:2610.09985#pg4> The interaction between pricing and advertising also has a long history in economics: Dorfman and Steiner (1954) analyzed the joint choice of price, advertising expenditure and product quality <ref:2610.09985#pg4> More recently, Agrawal et al. (2023) studied joint pricing and advertising in a sequential setting with unknown demand <ref:2610.09985#pg4> A growing body of work adapts LLMs for advertisement generation using performance feedback <ref:2610.09985#pg5>

Unbiasedness of the Actor Gradient Estimator

The proof shows that the leave-one-out baseline contributes zero to the conditional expected gradient <ref:2610.09985#pg4> The result requires neither an accurate nor an unbiased critic, as the leave-one-out baseline has zero expected contribution for any critic fixed before collecting the batch <ref:2610.09985#pg4>

Conclusion and Future Work

We study a sequential pricing problem in which a seller jointly selects a price and an LLM-generated advertisement, observing only whether the buyer purchases <ref:2610.09985#pg4> A natural extension is to allow demand to change over time <ref:2610.09985#pg4> This setting is challenging because shifting buyer preferences may alter both price sensitivity and the effect of advertisement content, so the critic must track a moving target from binary feedback while the actor keeps adapting to its estimates <ref:2610.09985#pg4>

How it works

The actor is a pretrained LLM adapted through low-rank adaptation (LoRA) to generate advertisements, while the critic is a demand model fitted to available data that estimates purchase probabilities given price, advertisement and product-level context <ref:2610.09985#pg4>
For each advertisement generated in batch τ, the seller selects a price through an exploration rule guided by the critic <ref:2610.09985#pg4>
The actor updates its LoRA parameters through policy gradient steps to maximize an objective that balances expected revenue with a KL penalty for deviations from the fixed pretrained reference model <ref:2610.09985#pg4>

The algorithm uses a leave-one-out baseline that averages the critic’s predicted revenues instead of observed rewards, reducing the noise introduced by individual purchase outcomes <ref:2610.09985#pg4> The actor surrogate loss is defined as Lactor(ψ) = −1/B X B b=1 Aτ,b − β log πθτ(xτ,b c(z)) πθref+∆θ(ψ)(xτ,b c(z)) <ref:2610.09985#pg4>

The critic loss is defined as Lcritic(ϕ) = −1/S X S s=1 h ys log ˆdϕ(ps, xs, z) + (1 − ys) log 1 − ˆdϕ(ps, xs, z) i + ρ∥ϕ - ϕ1∥2 <ref:2610.09985#pg4>

The actor update is based on REINFORCE (Williams, 1992), and the leave-one-out baseline contributes zero to the conditional expected gradient <ref:2610.09985#pg4> The critic is updated after every batch, while the actor is updated every K batches using only the seller-buyer interactions in the current batch <ref:2610.09985#pg4>

How it works

The critic estimates purchase probabilities given a price, an advertisement, and the context <ref:2610.09985#pg4> The pricing rule q: X × Z → [0, 1], model parametrization θ∈Θ and fixed z, expected revenue is u(q, θ z):= Ex∼πθ(·c(z)) q(x, z) d(q(x, z), x, z) <ref:2610.09985#pg4> The objective is regularized by uβ(q, θ z):= u(q, θ z) − β DKL(πθ(·c(z))∥ πθref (·c(z))) <ref:2610.09985#pg4>

The actor surrogate loss in Algorithm 1 satisfies Eτ h−∇ψLactor(ψ)ψ=ψτ i = ∇ψJτ (ψ z)ψ=ψτ <ref:2610.09985#pg4> This is proven using the score identity and the conditional independence of critic revenue estimates <ref:2610.

Improvements for AI systems

textbfOnline Joint Pricing and Advertisement Optimization for LLMs: Improving Revenue Maximization in Sequential Markets (Algorithm 1). This algorithm, combining LoRA adaptation of a pretrained LLM (the actor) with a data-fitted demand model (the critic), allows the seller to jointly learn price selection and advertisement generation by using purchase outcomes to update both components.
textbfRobust Policy Updates via Leave-One-Out Baseline: Reducing Noise in Actor Gradient Estimation. The algorithm employs a leave-one-out baseline, defined as Aτ,b:= rτ,b − 1/B − 1 X j∈[B]: j!=b ubϕτ,j, which the paper proves contributes zero to the conditional expected gradient when averaged over batches.
textbfControlled Policy Drift via KL Regularization: Preserving LLM Fluency During Adaptation. The actor loss incorporates a term that prevents deviation from the fixed reference model: Lactor(ψ) = − 1/B X B b=1 Aτ,b − β log πθτ(xτ,b c(z)) πθref (xτ,b c(z)) log πref+∆θ(ψ)(xτ,b c(z)). This ensures that while the model adapts its advertisements to revenue maximization, it avoids compromising the fluency and coherence of the generated content.
textbfAdaptive Exploration Strategy: Balancing Exploitation and Exploration. The algorithm utilizes an exploration rule where with probability 1 − ετ, it selects a price maximizing the critic’s estimate of expected revenue, while with probability ετ, it samples from an exploration distribution ντ on [0, 1]. This allows the system to balance expected revenue with a KL penalty for deviations from the fixed pretrained reference model.
textbfReal-World Marketplace Transfer: Achieving Expected Revenue Gains in E-commerce. The proposed algorithm demonstrates significant performance improvement over a reference policy, achieving gains such as expected revenue gains over the reference policy of 5.69%, 5.18% and 55.96% across synthetic demand models and the marketplace simulator."