Controllable Generative Trajectory Prediction via Weak Preference Alignment

arXiv:2510.10731 · cs.RO, cs.LG · Submitted 2025-10-12 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: We're looking at the paper: "Controllable Generative Trajectory Prediction via Weak Preference Alignment".

Title and authors: Dev: Let's talk about the title and who wrote this. The paper is titled "Controllable Generative Trajectory Prediction via Weak Preference Alignment," and it was put under review for a conference at ICLR two thousand twenty-six <ref:2510.10731#pg1,Controllable Generative Trajectory Prediction via Weak Preference Alignment>. The authors are Yongxi Cao, Julian F Schumann, Jens Kober, Joni Pajarinen, and Arkady Zgonnikov >

Rosa: It sounds technical because of the "preference alignment" part. Essentially, they are trying to give the latent variables in the model some actual meaning—some semantics—by looking at what we prefer between different predicted outcomes >

Taro: It's about moving beyond just getting an accurate prediction to getting a prediction that makes sense semantically for planning purposes. It suggests that for safe planning, you need context beyond just the trajectory itself >

Dev: They are addressing the issue where most trajectory prediction research focuses on perfect accuracy but ignores diversity and control, which is crucial when dealing with uncertain human behaviors in traffic >

The paper's summary: Rosa: So what does PrefCVAE actually do? It builds on the CVAE structure and uses these weakly labeled preference pairs to imbue the latent variables with semantic attributes. They are essentially using those preferences to map out a semantic latent space >

Dev: The key mechanism they use is this agent-wise preference loss. They define this as a cross entropy between the distribution of samples from the latent space and the distribution of scores given by some oracle program >

Taro: That means they are training the model so that when you pick a latent value, it aligns with what we prefer between two scenarios according to those labels >

Rosa: The paper shows that this preference loss encourages the predicted trajectory to align with that latent space. Specifically, smaller latent values are encouraged to have smaller metric values, which gives us some kind of controllability >

The paper's improvements: Dev: So what are the actual gains they claim? They show that this alignment leads to controllable prediction because we can know how a semantic factor relates to the latent value, which means we can guide the generation through stochastic sampling >

Rosa: The results are pretty interesting. They found that PrefCVAE enables AgentFormer to predict trajectories along a semantically monotonic metric by assigning latent values. A larger latent value always leads to a larger average velocity >

Taro: That's significant because it means we can control the speed of the prediction just by adjusting that one latent factor for an agent, and they showed controlling just one agent’s factor while sampling randomly for all others alters only that individual agent's behavior alone >

Conclusion: Rosa: So, to wrap up this paper on "Controllable Generative Trajectory Prediction via Weak Preference Alignment," the main thing is that they offer a way to create controllable trajectory prediction. This means a planner can make ego-plans conditioned on predefined attributes of adjacent vehicles >

Dev: The core innovation is this preference loss that regularizes the semantic meanings of latent factors through pairwise preference alignment. It's cost-effective because it uses weakly labeled pairs, and they aim to show a new way to incorporate dataset inductive bias for disentangled representation learning in these deep generative models >

Taro: I just want to say that while they showed this works with average velocity as a simple semantic factor, their evaluation is limited. They used only that one metric, which means extending it to multiple semantic dimensions is challenging because of latent factor correlations >

Rosa: That's true. And another thing they flag is the hyperparameter trade-off: increasing the weight of the preference loss improves the robustness of monotony, but there's also a tradeoff between accuracy and diversity as you increase the latent dimension >

Dev: So, in short, PrefCVAE gives us controllable trajectory prediction where we can condition ego-plans on semantic attributes of surrounding vehicles. It’s a solid step forward for using these generative models to support safer planning >

Aalto University · TU Delft

cs.RO, cs.LG

Submitted: 2025-10-12

Updated: 2026-11-18

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 79/100

The gist: Deep generative models such as conditional variational autoencoders (CVAEs) have shown great promise for predicting trajectories of surrounding agents in autonomous vehicle planning…

Key concepts

PrefCVAE
An augmented Conditional Variational Autoencoder (CVAE) framework designed for trajectory prediction. It introduces a 'preference loss' mechanism that aligns the latent space with human-labeled preference pairs, allowing latent variables to acquire meaningful semantic attributes.
Preference Loss
A loss function used during training that measures the alignment between model predictions and labeled preferences. It works by calculating a cross-entropy between the sampled latent distribution and the scores provided by an oracle program, guiding the model to ensure smaller latent values result in smaller predicted metric values.
Semantic Latent Space
The goal of structuring the latent variables so that they represent meaningful attributes, like average velocity. The preference loss enforces this by aligning preferences with these latent dimensions, meaning the model learns a space where specific latent factors correspond to predictable semantic outcomes.
Controllable Prediction
The ability to generate trajectories that can be steered by manipulating the latent variables. By controlling the latent values (e.g., from 0.1 to 0.9), users can directly influence continuous semantic factors like average velocity, enabling planners to make semantically conditioned ego-plans.

Terminology

Summary

Deep generative models such as conditional variational autoencoders (CVAEs) have shown great promise for predicting trajectories of surrounding agents in autonomous vehicle planning <ref:2510. p. 0

Where the paper says it

zgonnikov@tudelft. nl ABSTRACT Deep generative models such as conditional variational autoencoders (CVAEs) have shown great promise for predicting trajectories of surrounding agents in autonomous vehicle planning.

10731#pg2>. To address this, we propose PrefCVAE, an augmented CVAE framework that uses weakly labeled preference pairs to imbue latent variables with semantic attributes <ref:2510. p. 0
Where the paper says it

However, existing methods generally lack a scheme to generate controllably diverse trajectories, which is arguably more useful than randomly diversified trajectories, to the end of safe planning. To address this, we propose PrefCVAE, an augmented CVAE framework that uses weakly labeled preference pairs to imbue latent variables with semantic attributes.

10731#pg5>.

Pruning redundant reasoning tokens

Trajectory prediction is a key task for safe autonomous driving, forecasting the behaviors of road participants based on recent motions and accounting for complex and multimodal interactions among road agents <ref:2510.10731#pg2>. Existing CVAEs lack controllability of the prediction due to their implicit latent representations <ref:2510. p. 0

Where the paper says it

Conditional VAEs (CVAEs), in particular, excel in modeling the relationships among future trajectories, historical observations, and latent generative factors Kingma et al. Although CVAEs effectively model the causality between history and future trajectories, they lack controllability of the prediction due to their implicit latent representations Paige et al.

10731#pg3>. Most research frames trajectory prediction as a regression task, aiming to reconstruct and predict trajectories using an implicit latent code z <ref:2510. p. 0
Where the paper says it

Most research frames trajectory prediction as a regression task, with the aim of reconstructing and predicting trajectories using an implicit latent code zi . The typical goal is to predict the most likely trajectory based on the patterns of the dataset (Fig.

10731#pg3>. The goal is not just perfect accuracy but supporting safe planning, which requires richer context such as semantic attributes of predictions <ref:2510. p. 0
Where the paper says it

However, perfect accuracy on a test set is not the end goal; prediction must ultimately support safe planning Ivanovic & Pavone (2022). Planners benefit from a richer context, such as semantic attributes of predictions Chandra et al.

10731#pg3>.

How the training signal is built

The core idea of PrefCVAE is to enforce a semantic latent space by aligning the semantic of two model predictions with labelled preference of their latent generative factors <ref:2510.10731#pg5>. This involves mapping preferences to latent dimensions and using these preferences to encode semantics <ref:2510. p. 3

Where the paper says it

(2020), we call this procedure weak labelling, since it requires only ordinal signals rather than exact latent values. To inject semantic attributes into zS during training, we (i) map preferences to latent dimensions (each attribute linked to one dimension), and (ii) use these preferences to encode semantics, as detailed in this section.

10731#pg5>. Specifically, the agent-wise preference loss is defined as a cross entropy between the sampled latent distribution and the distribution of scores given by the oracle program <ref:2510. p. 4
Where the paper says it

(2017), is designed as a cross entropy between the sampled latent distribution and the distribution of scores given by the oracle program. With the ground truth preference Pˆ[yˆ 0 , yˆ 1 ] and two auxiliary sampled latent factors z0 S0 and z1 S0 , the agent-wise preference loss is defined as Lpref(z 0 S0 , z 1 S0 , yˆ 0 , yˆ 1 ) = −[Pˆ[yˆ 0 , yˆ 1 ] log( z 0 i z 0 S0 + z 1 S0 ) + (1 − Pˆ[yˆ 0 , yˆ 1 ]) log( z 1 S0 z 0 S0 + z 1 S0 )].

10731#pg6>. The loss function is formulated as L = LELBO + λbLpref, where λ is a weighting factor and b B(ν) is a Bernoulli random variable <ref:2510. p. 4
Where the paper says it

This stochastic sampling and generation process allows the creation of unseen but plausible trajectories guided by the latent semantic factors. For training, we simply use the preference loss alongside the original CVAE ELBO loss, that is, L = LELBO + λbLpref, where λ is a weighting factor, b ∼ B(ν) is a Bernoulli random variable, and the minibatch-wise loss is an aggregation of the agent-wise loss.

10731#pg5>.

Where the gains come from

The preference loss encourages the predicted trajectory to align with the latent, meaning smaller latent values are encouraged to have smaller metric values <ref:2510.10731#pg6>. This leads to controllable prediction because we know how a semantic factor aligns with latent value <ref:2510. p. 4

Where the paper says it

The prediction with a smaller latent value is encouraged to have a smaller metric value, and the converse is punished. This leads to controllable prediction because we know how a semantic factor aligns with latent value.

10731#pg6>. The method allows for the creation of unseen but plausible trajectories guided by the latent semantic factors through stochastic sampling and generation <ref:2510. p. 4
Where the paper says it

Additionally, leveraging preference, our method does not assume that latent factor boundaries are restricted to the data set. This stochastic sampling and generation process allows the creation of unseen but plausible trajectories guided by the latent semantic factors.

10731#pg6>.

The experiments demonstrate that PrefCVAE enables AgentFormer to predict trajectories along a semantically monotonic metric by assigning latent values <ref:2510.10731#pg9>. This shows that larger z value always leads to larger average velocity, as learned with the preference loss <ref:2510. p. 5

Where the paper says it

z value (horizontal dashed lines are ground truth values); Below: ADE (solid)/FDE (dashed). PrefCVAE can control the prediction: For model trained with PrefCVAE, larger z value always leads to larger average velocity, as learned with the preference loss.

10731#pg9>. Furthermore, controlling only one agent’s latent factor while randomly sampling for all other agents effectively alters that individual agent's behavior alone <ref:2510.10731#pg9>. The results show that the best accuracy occurs around latent values pertaining to ground truth velocity <ref:2510. p. 5
Where the paper says it

PrefCVAE can control the prediction: For model trained with PrefCVAE, larger z value always leads to larger average velocity, as learned with the preference loss. Also noticeably, the best accuracy occurs around the latent values that pertain to the ground truth velocity (the dashed horizontal lines).

10731#pg9>.

The analysis of hyperparameters indicates that using all preference pairs (ν = 1) does not yield the best performance, and dropping pairs has little impact <ref:2510.10731#pg10>. Surprisingly, using all pairs (ν = 1) does not yield the best performance (Tables 1, 2) <ref:2510. p. 7

Where the paper says it

Use rate For human-labeled data, collecting preferences for the entire dataset is costly, so we test the effect of randomly dropping preference pairs. Surprisingly, using all pairs (ν = 1) does not yield the best performance (Tables 1, 2).

10731#pg10>. Increasing the weight of preference loss improves the robustness of monotony <ref:2510. p. 8
Where the paper says it

, lowering the use rate). Increasing the weight of preference loss improves the robustness of monotony.

p. 8
Where the paper says it

, lowering the use rate). Increasing the weight of preference loss improves the robustness of monotony.

10731#pg10>. A tradeoff between accuracy and diversity emerges as latent dimension increases <ref:2510. p. 8
Where the paper says it

Increasing the weight of preference loss improves the robustness of monotony. A tradeoff between accuracy and diversity emerges as latent dimension increases.

p. 8
Where the paper says it

Increasing the weight of preference loss improves the robustness of monotony. A tradeoff between accuracy and diversity emerges as latent dimension increases.

10731#pg10>.

In conclusion, PrefCVAE enables controllable trajectory prediction with which a planner can make ego-plans that are conditioned on predefined attributes of adjacent vehicles <ref:2510.10731#pg7>. The core innovation is a preference loss that regularizes the semantic meanings of latent factors through pairwise preference alignment <ref:2510. p. 8

Where the paper says it

6 CONCLUSION This paper presents PrefCVAE, an augmentation to the CVAE framework that enables controllable trajectory prediction, with which a planner can make ego-plans that are conditioned on predefined attributes of adjacent vehicles, making planning more informed and safer. The core innovation is a preference loss that regularizes the semantic meanings of latent factors through pairwise preference alignment.

p. 8
Where the paper says it

6 CONCLUSION This paper presents PrefCVAE, an augmentation to the CVAE framework that enables controllable trajectory prediction, with which a planner can make ego-plans that are conditioned on predefined attributes of adjacent vehicles, making planning more informed and safer. The core innovation is a preference loss that regularizes the semantic meanings of latent factors through pairwise preference alignment.

10731#pg7>. Beyond proposing a method for controllable trajectory prediction, we aim to offer a new perspective on effectively and efficiently incorporating dataset inductive bias for disentangled representation learning in deep generative models <ref:2510. p. 8
Where the paper says it

The core innovation is a preference loss that regularizes the semantic meanings of latent factors through pairwise preference alignment. Beyond proposing a method for controllable trajectory prediction, we aim to offer a new perspective on effectively and efficiently incorporating dataset inductive bias for disentangled representation learning in deep generative models.

10731#pg7>.

Our evaluation of PrefCVAE is limited in that we used only a simple oracle-based metric (average prediction velocity) as the semantic factor <ref:2510.10731#pg10>. To enable real-world applications, future work should explore incorporating non-differentiable human-provided preferences <ref:2510. p. 8

Where the paper says it

1 LIMITATIONS AND FUTURE WORKS Our evaluation of PrefCVAE is limited in that we used only a simple oracle-based metric (average prediction velocity) as the semantic factor. To enable real-world applications, future work should explore incorporating non-differentiable human-provided preferences (e.

10731#pg10>. Another key limitation is that we evaluated our method with only one semantic dimension, and extending it to multiple factors is challenging due to latent factor correlations <ref:2510. p. 8
Where the paper says it

(2017)), as well as more practically useful semantic information for latents, such as Social Value Orientation (SVO) Schwarting et al. Another key limitation is that we evaluated our method with only one semantic dimension, and extending it to multiple factors 9

10731#pg10>. The findings regarding the choice of hyperparameters are as follows:

  1. Latent monotony persists while moderately dropping random preferences (i. p. 8

    Where the paper says it

    In conclusion, the findings regarding the choice of hyperparameters are as follows: 1. Latent monotony persists while moderately dropping random preferences (i.

    e., lowering the use rate) <ref:2510. p. 8
    Where the paper says it

    Latent monotony persists while moderately dropping random preferences (i. , lowering the use rate).

    10731#pg10>.

  2. Increasing the weight of preference loss improves the robustness of monotony <ref:2510.10731#pg10>.

  3. A tradeoff between accuracy and diversity emerges as latent dimension increases <ref:2510.10731#pg10>.

The paper is 2510.10731 <ref:2510. p. 0

Where the paper says it

However, to account for all plausible futures and make a thorough ego plan, it is often safer to predict a set of multimodal behaviors with predefined semantically controllable attributes (Fig. 1(c); the attributes 1 arXiv:2510.

10731#pg7>. The authors present PrefCVAE, an augmentation to the CVAE framework that enables controllable trajectory prediction <ref:2510. p. 8
Where the paper says it

A tradeoff between accuracy and diversity emerges as latent dimension increases. 6 CONCLUSION This paper presents PrefCVAE, an augmentation to the CVAE framework that enables controllable trajectory prediction, with which a planner can make ego-plans that are conditioned on predefined attributes of adjacent vehicles, making planning more informed and safer.

10731#pg7>. The core innovation is a preference loss that regularizes the semantic meanings of latent factors through pairwise preference alignment <ref:2510.10731#pg7>. This method is designed to be cost-effective as it uses weakly labeled preference pairs to imbue latent variables with semantic attributes <ref:2510. p. 0
Where the paper says it

However, existing methods generally lack a scheme to generate controllably diverse trajectories, which is arguably more useful than randomly diversified trajectories, to the end of safe planning. To address this, we propose PrefCVAE, an augmented CVAE framework that uses weakly labeled preference pairs to imbue latent variables with semantic attributes.

10731#pg5>. The results show the effectiveness of preference supervision as a cost-effective way to enhance samplingbased generative models <ref:2510. p. 0
Where the paper says it

Using average velocity as an example attribute, we demonstrate that PrefCVAE enables controllable, semantically meaningful predictions without degrading baseline accuracy. Our results show the effectiveness of preference supervision as a cost-effective way to enhance samplingbased generative models.

10731#pg5>. The paper is under review as a conference paper at ICLR 2026 <ref:2510. p. 0
Where the paper says it

Under review as a conference paper at ICLR 2026 CONTROLLABLE GENERATIVE TRAJECTORY PREDICTION VIA WEAK PREFERENCE ALIGNMENT Yongxi Cao Aalto University Espoo, Finland yongxi. fi Julian F.

p. 0
Where the paper says it

Under review as a conference paper at ICLR 2026 CONTROLLABLE GENERATIVE TRAJECTORY PREDICTION VIA WEAK PREFERENCE ALIGNMENT Yongxi Cao Aalto University Espoo, Finland yongxi. fi Julian F.

p. 0
Where the paper says it

Under review as a conference paper at ICLR 2026 CONTROLLABLE GENERATIVE TRAJECTORY PREDICTION VIA WEAK PREFERENCE ALIGNMENT Yongxi Cao Aalto University Espoo, Finland yongxi. fi Julian F.

p. 0
Where the paper says it

Under review as a conference paper at ICLR 2026 CONTROLLABLE GENERATIVE TRAJECTORY PREDICTION VIA WEAK PREFERENCE ALIGNMENT Yongxi Cao Aalto University Espoo, Finland yongxi. fi Julian F.

p. 0
Where the paper says it

Under review as a conference paper at ICLR 2026 CONTROLLABLE GENERATIVE TRAJECTORY PREDICTION VIA WEAK PREFERENCE ALIGNMENT Yongxi Cao Aalto University Espoo, Finland yongxi. fi Julian F.

10731#pg2>.

The authors illustrate the strengths of PrefCVAE by applying it to AgentFormer using average velocity as a simple semantic latent <ref:2510.10731#pg9>. They demonstrate that PrefCVAE enables AgentFormer to predict trajectories along a semantically monotonic metric by assigning latent values <ref:2510. p. 1

Where the paper says it

In experiments using a large-scale dataset nuScenes Caesar et al. (2020) and average velocity as a simple semantic latent, we demonstrate that PrefCVAE enables AgentFormer to predict trajectories along a semantically monotonic metric by assigning latent values.

10731#pg9>. Additionally, they show that the approximate posterior encoder maps trajectories to their ground-truth latent values with higher likelihood <ref:2510. p. 1
Where the paper says it

(2020) and average velocity as a simple semantic latent, we demonstrate that PrefCVAE enables AgentFormer to predict trajectories along a semantically monotonic metric by assigning latent values. Additionally, we show that the approximate posterior encoder maps trajectories to their ground-truth latent values with higher likelihood.

10731#pg9>. These results demonstrate the feasibility of PrefCVAE as a framework for controllable generative trajectory prediction <ref:2510. p. 1
Where the paper says it

Additionally, we show that the approximate posterior encoder maps trajectories to their ground-truth latent values with higher likelihood. These results demonstrate the feasibility of PrefCVAE as a framework for controllable generative trajectory prediction.

10731#pg9>. The paper is under review as a conference paper at ICLR 2026 <ref:2510.10731#pg10>.

The authors show that PrefCVAE can control the prediction by traversing the latent space from 0.1 to 0.9 with a step size of 0. p. 6

Where the paper says it

We evaluated the models by traversing the latent space from 0. 9 with a step size of 0.

1, where larger z value always leads to larger average velocity <ref:2510. p. 5
Where the paper says it

z value (horizontal dashed lines are ground truth values); Below: ADE (solid)/FDE (dashed). PrefCVAE can control the prediction: For model trained with PrefCVAE, larger z value always leads to larger average velocity, as learned with the preference loss.

10731#pg9>. This confirms that the latent factors are essentially learned independently for each agent <ref:2510. p. 6
Where the paper says it

Interestingly, though the latent factors of each agent are jointly regularized during training, controlling only one agent’s latent factor (Fig. 3a, column 3) while randomly sampling for all other agents in a scene effectively alters the behavior of that individual agent alone, indicating that the latent factors are essentially learned independently for each agent.

10731#pg9>. The paper is under review as a conference paper at ICLR 2026 <ref:2510.10731#pg10>.

The authors show that PrefCVAE encodes a more accurate known attribute, and the latent codes are better reconstructed when using PrefCVAE compared to the base model <ref:2510.10731#pg9>. This confirms that PrefCVAE learns model-agnostic generative factors (Fig. p. 7

Where the paper says it

With M = 32, we cross-tested the base model, λ = 8, and λ = 32. Qualitatively, the two PrefCVAE models with different λ values encode similar latent distributions, while other pairs do not, confirming that PrefCVAE learns model-agnostic generative factors (Fig.

5) <ref:2510. p. 0
Where the paper says it

However, to account for all plausible futures and make a thorough ego plan, it is often safer to predict a set of multimodal behaviors with predefined semantically controllable attributes (Fig. 1(c); the attributes 1 arXiv:2510.

10731#pg9>. The paper is under review as a conference paper at ICLR 2026 <ref:2510.10731#pg10>.

The authors show that with M = 32, the average velocity range drops below 3.3 m/s, suggesting reduced controllability as higher dimensions introduce correlations that hinder assigning distinct semantics to each factor <ref:2510. p. 8

Where the paper says it

3 m/s for M = 2, λ = 16). This suggests reduced controllability as higher dimensions introduce correlations that hinder assigning distinct semantics to each factor.

10731#pg10>. The paper is under review as a conference paper at ICLR 2026 <ref:2510.10731#pg10>.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Preference supervision enhances samplingbased generative models by using weakly labeled preference pairs to imbue latent variables with semantic attributes, enabling controllable, semantically meaningful predictions without degrading baseline accuracy.

  2. The improved AI system can generate controllably diverse trajectories that are conditioned on predefined continuous semantic factors, such as average velocity, allowing planners to reason about the interaction semantically and predict corresponding behaviors rather than relying solely on the most likely prediction.

  3. The system can enforce a semantic latent space by using a preference loss defined as a cross entropy between the sampled latent distribution and the distribution of scores given by an oracle program, ensuring that the prediction with a smaller latent value is encouraged to have a smaller metric value.

  4. The model can be trained using various use rates for preference pairs, with findings suggesting that moderately dropping random preferences (i.e., lowering the use rate) maintains latent monotony while improving robustness, and increasing the weighting factor of preference loss improves the robustness of monotony.

Abstract

Deep generative models such as conditional variational autoencoders (CVAEs) have shown great promise for predicting trajectories of surrounding agents in autonomous vehicle planning. State-of-the-art models have achieved remarkable accuracy in such prediction tasks. Besides accuracy, diversity is also crucial for safe planning because human behaviors are inherently uncertain and multimodal. However, existing methods generally lack a scheme to generate controllably diverse trajectories, which is arguably more useful than randomly diversified trajectories, to the end of safe planning. To address this, we propose PrefCVAE, an augmented CVAE framework that uses weakly labeled preference pairs to imbue latent variables with semantic attributes. Using average velocity as an example attribute, we demonstrate that PrefCVAE enables controllable, semantically meaningful predictions without degrading baseline accuracy. Our results show the effectiveness of preference supervision as a cost-effective way to enhance sampling-based generative models.

Sources

Related papers