TextGaze: Prompting Gaze Target Estimation with Textual Scene Cues
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: I'm Tom.
Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.
Tom: We're looking at the paper: "TextGaze: Prompting Gaze Target Estimation with Textual Scene Cues".
Jane: Here's the gist of the paper. Gaze target estimation aims to infer the position of a person’s gaze within a scene This paper proposes TextGaze, a unified cross-modal architecture that leverages a Large Vision-Language Model (LVLM) as scalable semantic guidance to balance the two design paradigms Problem Context and Motivation Gaze target estimation aims to infer the position of a person’s gaze within a scene Gaze target estimation Recasens et al.
2015: or video frame Recasens et al.
2017: is a task in computer vision that aims to predict a person’s gaze direction in a scene or determine if the gaze target lies within the image Multi-branch methods require extra supervision and annotations, while streamlined designs prioritize low-level visual saliency over true gaze intent The former leads to a high annotation burden and hinders domain transfer, whereas the latter causes misalignment between predicted attention and actual gaze targets Gaze following is inherently an intent-level reasoning problem rather than a purely geometric regression task Model Architecture Design TextGaze adopts a dual-stream design, shifting gaze estimation from direct geometric regression (pixels to gaze coordinates) to an intent-recognition process via a semantic layer (pixels to semantic intent to gaze target) One stream uses an LVLM with a BERT encoder to generate textual descriptions of subject behavior and plausible targets, yielding semantically enriched visual-text embeddings The other extracts fine-grained scene cues via a frozen scene encoder and utilizes positional head prompting to enhance character positioning and facial cues These complementary streams are fused by a Visual-Text Fusion Transformer for context-aware, cross-modal gaze estimation The model employs a three-layer stacked fusion transformer with post-normalization to achieve precise vision-text alignment while retaining task-critical text semantics Training Signal Construction The overall objective is defined as Lall = Lheat + λ1Lin/out + λ2Ltext, where λ1 and λ2 balance the auxiliary loss The hierarchical text supervision loss enforces semantic alignment across all transformer layers by calculating the mean squared error between reconstructed text features and target text features The visual token segment is reshaped into F ∈ R×H×W×d, where H and W denote spatial dimensions A two-layer convolutional decoder outputs the gaze probability heatmap Iheat, supervised by pixel-wise BCE loss with Gaussian targets Performance Evaluation and Contributions The method achieves competitive performance across key metrics with robust cross-dataset generalisation without extra finetuning The main contributions include introducing textual scene cues into gaze target estimation to mitigate semantic defocus We strengthen the GazeFollow and VideoAttentionTarget benchmarks by integrating scene-target-person attention fields with LVLM-oriented prompts We design a unified dual-stream cross-modal fusion framework that combines textual embeddings with finegrained visual representations, improving spatial reasoning, scene adaptability, and mimicking human top-down attention Cross-dataset results on the GOO-Real and ChildPlay datasets demonstrate strong generalization in real-world scenarios The best results are in bold and the second-best are underlined on Table 2 Ablation Study Insights The ablation studies indicate that the transformer-based fusion performs unified attention across all visual and textual tokens, allowing globally symmetric interactions The results in Table 4 demonstrate the effectiveness of this fusion strategy within our design Simultaneously activating the three components—Txt Sup.
Title and authors: Tom: So we're looking at TextGaze today, which is this new architecture that tries to solve gaze target estimation by using a large vision language model or LVLM to guide the process.
Jane: Right, so it’s trying to balance two different ways of doing things—the old way with lots of visual cues versus the newer way with semantic guidance from text. What does this actually aim to do for someone just watching a video?
Lu: Basically, the goal is to figure out exactly where a person is looking in a scene, not just which part of the image looks busy or salient. The paper talks about how traditional methods either need tons of extra labeling or they get confused between where people are looking and what’s actually there >
Meng: I worry about those multi-branch methods. If you have too many branches, it just gets complicated to deploy and the supervision you need becomes huge, which is a nightmare for production systems >
Lalam: TextGaze tries to fix that by using the LVLM as a scalable semantic guide, which means it generates textual descriptions of what the subject is doing and what targets might be plausible >
Tom: Exactly. The core idea is shifting from just predicting pixels to coordinates directly, which is hard because it’s an intent-level reasoning problem. TextGaze tries to move toward an intent recognition process using a semantic layer >
Jane: That sounds like they’re trying to bridge the gap between low-level visual cues and high-level understanding. How do they actually build this cross-modal connection?
Lu: They use a dual-stream design. One stream uses an LVLM with a BERT encoder to create semantically rich visual text embeddings, while the other extracts fine-grained scene cues using a frozen scene encoder and positional head prompting >
Meng: A frozen scene encoder sounds efficient for deployment, but how does that fine-grained visual stream actually feed into the text guidance part? What’s the fusion mechanism?
Tom: They fuse these two streams using a Visual Text Fusion Transformer. It’s a three-layer stacked fusion transformer with post-normalization, which they say achieves this precise vision text alignment >
Jane: So, if I imagine I'm driving and I need to know where someone is looking, this model is essentially asking the computer to first read what the person is doing and then combine that high-level understanding with what the camera actually sees >
Lu: It’s about getting those contextual cues together so the model can reason better about which objects are valid gaze targets instead of just following simple visual saliency >
Meng: That makes sense for robustness. If the facial cues are blurry or occluded, having high-level semantic priors from the LVLM should help it compensate for missing details >
Tom: The training signal construction is also interesting. They use a loss function Lall that balances heat loss with two auxiliary losses: Lin/out and text >
Title and authors: Jane: The text supervision loss enforces semantic alignment across all those transformer layers by calculating the mean squared error between reconstructed text features and target text features >
Lu: And the visual token segment is reshaped into an F of R times H times W times d, where H and W are the spatial dimensions, which feeds into a two-layer convolutional decoder that outputs the gaze probability heatmap, Iheat >
Meng: What about scaling this? If we have a high resolution image or a long video sequence to process, how does the complexity of this transformer fusion scale in practice?
Tom: The architecture is designed to be scalable because they are leveraging pre-trained LVLMs like GPT-4V and CLIP, which already have strong multimodal reasoning capabilities > <ref:2607.10130#pg1>
Jane: So, if we think about the real world for a user, what does this mean for them when they use this technology compared to just looking at the picture?
Lu: It means that instead of relying on brittle geometric regression that fails when things change slightly, you get a system that adapts to different scenes because it has these textual scene cues >
Meng: I’m curious about deployment cost. Training this requires running an LVLM encoder and a large fusion transformer, so what is the operational overhead compared to simpler models?
Tom: The ablation studies show that the transformer-based fusion performs unified attention across all visual and textual tokens, allowing globally symmetric interactions >
Jane: That’s a big piece of information. It suggests the way they combine text and vision isn't just localized; it’s a holistic interaction between everything in the input >
Lu: And when you look at their results on datasets like GOO-Real and ChildPlay, they show strong generalization without needing extra finetuning on those specific tasks >
Meng: Generalization across four mainstream datasets without extra finetuning is impressive, but I still need to know how reliable this holds up when we move to real traffic where the scene context is completely novel >
Tom: The main contribution here, as they state it in TextGaze: Prompting Gaze Target Estimation with Textual Scene Cues, is introducing these textual scene cues into gaze target estimation specifically to mitigate semantic defocus >
Jane: So, the paper addresses that initial problem where the visual focus might be too vague and provides a way to fix it by injecting descriptive text information >
Lu: It’s about making the prediction process more robust by giving it context that goes beyond just pixels, which is what they call semantic guidance >
Meng: Before we wrap up this part, I want to ask about their limitations. What does TextGaze explicitly say it *cannot* do reliably?
Tom: The paper points out that the traditional multi-branch frameworks introduced slow convergence problems when adding extra branches because they needed task-specific supervision on top of gaze annotations >
Title and authors: Jane: And they also mention that those older methods were essentially post-processing geometric filters, meaning they could eliminate illogical regions but couldn't actually reason about what constitutes a valid gaze target >
Lu: So the limitation is still rooted in the fact that they haven't fully resolved the problem of semantic under-specification, even with this new framework >
Meng: It sounds like it’s a step forward from those older methods, but it’s not perfect because it still relies on these auxiliary cues being effective across diverse scenarios >
Tom: Exactly. They show strong cross-dataset results on GOO-Real and ChildPlay where the best results are bolded and the second-best ones are underlined on Table two > <ref:2607.10130#pg1>
Jane: And for someone just listening, what this means is that we’re getting a system that's much better at understanding intent than just looking at raw video frames alone >
Lu: It suggests that by using foundation models to generate those textual scene cues, we can achieve high performance without the massive annotation burden associated with the old methods >
Meng: So, to summarize, TextGaze is a unified cross-modal architecture that uses an LVLM for semantic guidance and a dual-stream fusion transformer to improve spatial reasoning and scene adaptability >
Tom: It’s about shifting from direct geometric regression to an intent recognition process via a semantic layer, which makes the model much more flexible >
Jane: We've talked about how it works, but what does this mean for the future of gaze estimation research? Where does this lead next?
Lu: It points toward using accessible, interpretable auxiliary semantic cues to achieve high performance in gaze target estimation through effective cross-modal fusion >
Meng: From a deployment standpoint, it means we can build systems that are more resilient to visual noise because they have these high-level semantic priors compensating for missing facial details >
Tom: It’s about balancing the need for low-level visual cues with the need for top-down reasoning, which is something real human attention does >
Jane: So, TextGaze is a solid step in using large vision language models to inject richer context into computer vision tasks like gaze estimation >
Lu: It’s a framework that combines textual embeddings with fine-grained visual representations to improve spatial reasoning across different types of scenes >
Meng: I still see the challenge being making sure these semantic cues are truly reliable when we move beyond controlled datasets to unpredictable real-world traffic where things get messy >
Tom: That’s the natural next step for any model, but TextGaze shows a way to build a more adaptable foundation than what we had before >
Jane: So, that’s our take on TextGaze today. It’s about using language models to give vision models better reasoning about human attention >
The paper's summary: Tom: So we just finished looking at TextGaze, and now let's see what the authors actually say about it in their summary.
Jane: Right, so they're saying this whole TextGaze thing is basically an attempt to stop models from getting confused about where someone is looking by adding some text information.
Tom: Exactly. They’re taking this idea of using a large vision language model, or LVLM, and training it to generate descriptions of what the person is doing and what targets are plausible.
Lu: It’s like they are giving the visual system a set of semantic clues that help it understand the intent behind the gaze, instead of just guessing based on pixels.
Jane: That makes sense. They are shifting from a direct guess about where your eyes landed to an intent-recognition process guided by those textual descriptions.
Tom: They also talk about how they’re using two different streams for this, which is pretty smart because it balances the visual data with the text guidance.
Meng: I see. So one part of the model is looking at the raw scene cues, and another part is processing that text to give context to what it sees.
Jane: Precisely. The visual stream grabs fine-grained details from the scene, like facial features or objects nearby, while the text stream provides that high-level semantic background.
Tom: And they fuse those two streams using a special transformer—a Visual Text Fusion Transformer—to make sure they align well with each other, which is pretty important for keeping things accurate.
Lu: The goal of that fusion is to get this unified cross-modal representation, which improves the model’s spatial reasoning and how it handles different scenes.
Jane: So instead of needing separate models for vision and text, they build one framework that uses both together to get a better picture of attention.
Tom: And when they look at the results on those datasets like GOO-Real and ChildPlay, they show that this method generalizes well without needing extra finetuning on those specific tasks.
Meng: That’s good for deployment because it means we don't have to spend months labeling data just to make it work for a new environment.
Lu: The authors point out that their main contribution is introducing these textual scene cues specifically to fight what they call semantic defocus, which is when the visual focus is too vague.
Jane: So, the big idea here is using language models to inject richer context into vision tasks so the model can make better predictions than it could on its own.
Tom: It’s about making the system more adaptable because it uses these high-level semantic priors to compensate when lower-level visual details are missing or unclear.
Lu: If you look at how they structured the loss function, Lall, they have this hierarchical text supervision loss that forces semantic alignment across all the layers of that transformer.
Jane: That means even deep in the model, it’s constantly checking if what it’s seeing matches what the text says about what should be there.
Tom: It sounds like they’re trying to bridge that gap between low-level visual processing and high-level reasoning in a structured way.
Meng: From my side, I'm still wondering how much computation this fusion transformer adds to the real runtime compared to just running a standard vision model.
Lu: The ablation studies suggest that the unified attention across all tokens means the interaction between text and vision is globally symmetric, which is a strong point for performance.
Jane: So, what this really changes for us as listeners is that we are seeing systems that aren't just reacting to what’s in front of them, but actually trying to understand *why* things are in front of them.
The paper's improvements: Tom: So we’ve seen how TextGaze works, and now let's talk about what the authors suggest they improved in this architecture.
Jane: They’re basically saying they took those initial ideas and refined them to make them more robust in real-world use, especially when things get messy.
Tom: Right. One big improvement is how they handle those unreliable visual cues; they're using LVLM-based behavioral descriptions to compensate for missing details with high-level semantic priors.
Lu: So if the face is blurry or something else blocks the view, the model doesn't fail completely because it has a strong semantic understanding of what should be happening.
Jane: That’s useful for safety, isn't it? It means we can predict gaze even when the visual signal isn't perfect.
Tom: Exactly. They also introduced a unified dual-stream cross-modal fusion framework that combines text embeddings with fine-grained visual representations to improve spatial reasoning and scene adaptability.
Meng: That sounds like a big architectural win for deployment because it means the model is better at handling different scenes without needing specific retraining for every new environment.
Lu: The authors also argue that by shifting from direct geometric regression, which is just pixels to gaze coordinates, to an intent-recognition process via a semantic layer, they solve the problem of just following low-level visual saliency.
Jane: So it’s not about finding the brightest spot in an image; it’s about figuring out what the person *intends* to look at based on context.
Tom: And they used prompt engineering, like instructing the LVLM to describe only the direction of looking without detailing exactly what they are looking at, which helps keep those auxiliary cues effective across diverse scenarios.
Meng: I’m still thinking about the cost here. Does that extra text processing and fusion layer add a lot of latency when we push this to a real-time application?
Lu: The fusion transformer is three layers deep with post-normalization, but they claim it achieves precise vision text alignment while retaining all the critical text semantics, which is the trade-off they’re accepting.
Jane: So for a user watching a video, what does this actually change in their experience?
Tom: It means better spatial reasoning and scene adaptability. The system can handle more complex scenes because it understands the bigger picture provided by the text cues.
Lu: They also showed strong generalization on datasets like GOO-Real and ChildPlay without needing extra finetuning on those specific tasks, which is a key result for their method’s efficiency.
Jane: It shows that this approach isn't just for a lab setting; it actually works well in scenarios that look different from the training data.
Meng: So, the implication is that we can build systems that are more resilient to visual noise and don't break easily when things get unexpected in production.
Tom: The authors emphasize that they’ve created a streamlined design to avoid the high annotation burden and subjectivity of older multi-branch methods, balancing those two needs really well.
Lu: It boils down to using foundation models like GPT or CLIP to inject better reasoning into vision tasks, which is a scalable way forward for cross-modal understanding.
Conclusion: Tom: So we're wrapping up on TextGaze, summarizing what they actually proved in this paper and where this research is heading next.
Jane: Basically, they showed that using a large vision language model to provide textual scene cues can significantly improve gaze target estimation by making the model reason about intent instead of just looking at pixels.
Tom: Right. It’s a unified cross-modal architecture that uses both textual embeddings and fine-grained visual representations fused by a transformer to get better spatial reasoning.
Lu: The main implication is that we can build systems that are more resilient to visual noise because they have these high-level semantic priors compensating for missing facial details.
Meng: From a deployment standpoint, it means we can skip the massive annotation burden of multi-branch methods and still get competitive performance without needing extra finetuning on those specific datasets.
Jane: It’s about shifting from a direct guess about where your eyes landed to an intent recognition process guided by those textual descriptions.
Tom: They also demonstrated that this method performs well across different datasets, showing strong generalization without needing extra finetuning on GOO-Real and ChildPlay.
Lu: The paper points toward using accessible, interpretable auxiliary semantic cues to achieve high performance in gaze target estimation through effective cross-modal fusion.
Jane: So, the future seems to be about using language models not just for chat, but as a way to give vision models better reasoning about human attention.
Tom: It’s about balancing the need for low-level visual cues with the need for top-down reasoning that mimics how real human attention works.
Meng: I still see the challenge being making sure these semantic cues are truly reliable when we move beyond controlled datasets to unpredictable real-world traffic where things get messy.
Lu: That’s the natural next step, but TextGaze shows a way to build a more adaptable foundation than what we had before in this area.
Jane: So, that’s our take on TextGaze today—using large vision language models to inject richer context into computer vision tasks like gaze estimation.
University of Science and Technology of China · Hefei University of Technology · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center · Anhui University
cs.CV
Submitted: 2026-07-11
Updated: 2026-11-11
Comments: Accepted at IJCAI-ECAI 2026 (IJCAI-26, Main Track), pp. 1586-1594. 9 pages, 5 figures. All contents are available at: https://github.com/idremo/TextGaze-IJCAI2026
Journal ref: In Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-26), pp. 1586-1594, 2026
Code: https://github.com/idremo/TextGaze-IJCAI2026
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 86/100
The gist: Gaze target estimation aims to infer the position of a person’s gaze within a scene This paper proposes TextGaze…
Key concepts
- Gaze Target Estimation
- This is a computer vision task that tries to figure out exactly where a person's eyes are focused within an image or video frame. The goal is to predict the specific location of the object or area they are looking at, moving beyond just general visual attention.
- Large Vision-Language Model (LVLM)
- An LVLM is a powerful AI model that can understand both images and text simultaneously. In this system, it's used to generate detailed textual descriptions of what people in the scene are doing and what plausible targets might be, providing rich semantic guidance for gaze prediction.
- Dual-Stream Cross-Modal Fusion
- This is the model's core design where two different information streams—one from the LVLM (textual) and one from a visual encoder (visual)—are combined. This fusion happens through a specialized transformer that allows the model to integrate high-level semantic meaning with fine-grained visual details for better, context-aware predictions.
- Intent Recognition Process
- Instead of just mapping pixels directly to coordinates, this method treats gaze estimation as an intent problem. The model first interprets the scene and text to understand *why* a person is looking somewhere, allowing it to reason about the target's meaning rather than just tracking low-level visual features.
Terminology
Summary
Gaze target estimation aims to infer the position of a person’s gaze within a scene This paper proposes TextGaze, a unified cross-modal architecture that leverages a Large Vision-Language Model (LVLM) as scalable semantic guidance to balance the two design paradigms
Problem Context and Motivation
Gaze target estimation aims to infer the position of a person’s gaze within a scene Gaze target estimation Recasens et al. [2015] or video frame Recasens et al. p. 1 [2025] aims to predict
the location a person is looking at in a scene image Recasens
et al. [2015] or video frame Recasens et al.Where the paper says it
Model Architecture Design
TextGaze adopts a dual-stream design, shifting gaze estimation from direct geometric regression (pixels to gaze coordinates) to an intent-recognition process via a semantic layer (pixels to semantic intent to gaze target) One stream uses an LVLM with a BERT encoder to generate textual descriptions of subject behavior and plausible targets, yielding semantically enriched visual-text embeddings The other extracts fine-grained scene cues via a frozen scene encoder and utilizes positional head prompting to enhance character positioning and facial cues These complementary streams are fused by a Visual-Text Fusion Transformer for context-aware, cross-modal gaze estimation The model employs a three-layer stacked fusion transformer with post-normalization to achieve precise vision-text alignment while retaining task-critical text semantics
Training Signal Construction
The overall objective is defined as Lall = Lheat + λ1Lin/out + λ2Ltext, where λ1 and λ2 balance the auxiliary loss The hierarchical text supervision loss enforces semantic alignment across all transformer layers by calculating the mean squared error between reconstructed text features and target text features The visual token segment is reshaped into F ∈ R×H×W×d, where H and W denote spatial dimensions A two-layer convolutional decoder outputs the gaze probability heatmap Iheat, supervised by pixel-wise BCE loss with Gaussian targets
Performance Evaluation and Contributions
The method achieves competitive performance across key metrics with robust cross-dataset generalisation without extra finetuning The main contributions include introducing textual scene cues into gaze target estimation to mitigate semantic defocus We strengthen the GazeFollow and VideoAttentionTarget benchmarks by integrating scene-target-person attention fields with LVLM-oriented prompts We design a unified dual-stream cross-modal fusion framework that combines textual embeddings with finegrained visual representations, improving spatial reasoning, scene adaptability, and mimicking human top-down attention Cross-dataset results on the GOO-Real and ChildPlay datasets demonstrate strong generalization in real-world scenarios The best results are in bold and the second-best are underlined on Table 2
Ablation Study Insights
The ablation studies indicate that the transformer-based fusion performs unified attention across all visual and textual tokens, allowing globally symmetric interactions The results in Table 4 demonstrate the effectiveness of this fusion strategy within our design Simultaneously activating the three components—Txt Sup.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
TextGaze can be used to produce
semantically rich textual cues that align with visual focus
by leveraging a frozen Large Vision-Language Model to generate descriptions of subject behavior and plausible targets, addressing the issue ofsemantic defocus.
-
The system can achieve a
unified dual-stream cross-modal fusion framework that combines textual embeddings with finegrained visual representations, improving spatial reasoning, scene adaptability, and mimicking human top-down attention
by integrating an LVLM stream with a frozen scene encoder and positional head prompting. -
Gaze target estimation can be improved by shifting from
direct geometric regression (pixels to gaze coordinates) to an intent-recognition process via a semantic layer (pixels to semantic intent to gaze target),
enabling the model to reason about which objects constitute valid gaze targets rather than just following low-level visual saliency. -
The model can better handle unreliable visual cues by generating
context-aware auxiliary cues that remain effective across diverse scenarios
through prompt engineering, such as instructing the LVLMto describe only the direction in which the target person is looking without detailing the object/person they are viewing.
-
The system gains robustness against occlusion or low resolution by using
LVLM-based behavioural descriptions that compensate for missing details with high-level semantic priors,
ensuring reliable predictions even when facial cues are unreliable. -
The architecture can be streamlined to avoid the
high annotation burden and subjectivity
of multi-branch methods, resulting in a system that balancesmulti-branch methods and streamlined designs
while achieving competitive performance without extra finetuning on four mainstream datasets.
Abstract
Gaze target estimation aims to infer the position of a person's gaze within a scene. Within mainstream design logic, multi-branch methods require extra supervision and annotations, while streamlined designs prioritize low-level visual saliency over true gaze intent. The former leads to a high annotation burden and hinders domain transfer, whereas the latter causes misalignment between predicted attention and actual gaze targets. To address this issue, we propose TextGaze, a unified cross-modal architecture that leverages a Large Vision-Language Model (LVLM) as scalable semantic guidance to balance the two design paradigms. The model extracts visual features from a frozen encoder and utilizes an LVLM to obtain gaze-aligned textual cues. We design a transformer-based fusion module with hierarchical text supervision to preserve task semantics. Lightweight decoding heads enable the joint prediction of gaze heatmaps and in-/out-of-frame status. We evaluate our method on four mainstream datasets, and the results show competitive performance across key metrics with robust cross-dataset generalisation without extra fine-tuning. Overall, we provide a streamlined alternative to traditional designs and highlight the potential of LVLMs as accessible auxiliary guidance for gaze estimation.
Sources
- Qwen2.5-VL Technical Report
- Towards Seamless Interaction: Causal Turn-Level Modeling of Interactive 3D Conversational Head Dynamics
- What Does BERT Look At? An Analysis of BERT's Attention
- The Llama 3 Herd of Models
- MA-Bench: Towards Fine-grained Micro-Action Understanding
- Decoupled Weight Decay Regularization
- DINOv2: Learning Robust Visual Features without Supervision
- AI Governance and Accountability: An Analysis of Anthropic's Claude
- Emu3: Next-Token Prediction is All You Need
- The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)
- Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs