DAT: Dual-Aware Adaptive Transmission with Small-Large Model Collaboration for Edge-Cloud Anomaly Detection

arXiv:2604.05375 · cs.MM · Submitted 2026-04-07 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "DAT: Dual-Aware Adaptive Transmission with Small-Large Model Collaboration for Edge-Cloud Anomaly Detection".

Title and authors: Tom: So we’re looking at this paper called "DAT: Dual-Aware Adaptive Transmission with Small-Large Model Collaboration for Edge-Cloud Anomaly Detection". It sounds pretty technical, but basically, it tackles the problem of using these multimodal large language models, or MLLMs, on continuous video streams.

Jane: Right. The title tells you a lot about what they're trying to do: dual awareness and adaptive transmission with small and large models collaborating at the edge and in the cloud for anomaly detection.

Lu: It seems like they are focusing on reducing the huge computational cost of running these big MLLMs constantly on video data, which is a real bottleneck when you’re dealing with continuous streams in bandwidth-constrained systems.

Meng: I’m curious about what that means for deployment. If you have to run this massive model every time something happens, it just won't work at scale.

Lalam: This paper proposes a collaboration between a small model on the edge and the large MLLM inference side, which they call a cascade mechanism where the small model acts as a gating module to filter out non-target frames and only trigger the expensive MLLM inference for suspicious frames.

The paper's summary: Tom: So what’s the core idea here? They are proposing this collaborative small large model cascade where a lightweight edge-side small model does rapid screening to reduce unnecessary large-model invocations, and then they design an efficient fine-tuning strategy with visual guidance and semantic prompting.

Jane: That makes sense. They are trying to get the big understanding from the MLLM only when it really needs to look at something specific, which saves a ton of computation on the edge.

Lu: And they also introduce this efficient fine-tuning strategy, using the boxed image and a task prompt built from small model priors as input to the MLLM, which produces structured semantic output. This is supported by a parameter efficient adaptation technique called LoRA for fine-tuning.

Meng: LoRA sounds good for reducing training cost, but how does that affect the actual quality of the structured understanding they are trying to get?

Lalam: They retain a priority output qt which gives coarsegrained prioritization through lt and finegrained semantic severity through rt, which is important for their transmission strategy later on.

The paper's improvements: Tom: The paper points out several improvements they made. First, they jointly optimize cascade inference and elastic transmission—meaning they link how the model runs with how the data gets sent over the network.

Jane: That’s a big move, linking inference efficiency with transmission efficiency to solve the problem of low-latency alerting and effective visual evidence delivery under bandwidth constraints.

Lu: The adaptive transmission strategy formulates multimodal data stream uploading as a lexicographic optimization problem, with minimizing weighted semantic alert delay being the primary objective and maximizing effective visual evidence delivery being the secondary objective.

Meng: A lexicographic optimization problem sounds complicated for an edge device to solve in real time; how do they actually handle that complexity?

Lalam: They use an online hierarchical greedy approximation to solve this, meaning they first prioritize structured semantic information for high-value events and then supplement visual evidence when the residual budget allows. This approach shows that DAT reduces weighted semantic alert delay by up to seventy-seven point five percent under severe congestion and delivers ninety-eight point three three percent of visual evidence within zero point five seconds, according to Table three >

Conclusion: Tom: So, to wrap up, the key contribution of "DAT: Dual-Aware Adaptive Transmission with Small-Large Model Collaboration for Edge-Cloud Anomaly Detection" is this dual optimization approach. They manage the inference cascade and the transmission scheduling together.

Jane: It shows that you can get high-quality semantic generation and low latency event alerting, plus good visual evidence, all while cutting down on the heavy MLLM reasoning costs on the edge.

Lu: The use of LoRA for fine-tuning helps improve structured output compliance, showing they can achieve a parse, schema, rule and violation rate of one point zero for the first three metrics and zero for violation when generating structured semantics from small model guided inputs using limited training cost > <ref:2604.05375#pg1>

Meng: From a deployment standpoint, the fact that it handles severe congestion by prioritizing semantic carriers over visual evidence under dynamic bandwidth constraints is what makes this useful; it’s not just about getting high accuracy in a perfect network.

Lalam: Exactly. The paper concludes that DAT is an efficient framework for MLLM inference and transmission in edge-cloud systems, proving that coordinating cascade inference and elastic transmission works well together >

Institute of Computing Technology, Chinese Academy of Science

cs.MM

Submitted: 2026-04-07

Updated: 2026-11-17

Comments: 9 pages, 7 figures. Submitted to AAAI 2027

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 83/100

The gist: Multimodal large language models (MLLMs) have shown strong capability in semantic understanding and visual reasoning…

Key concepts

Small-Large Model Cascade
This involves using a lightweight, small model on the edge to first filter video frames for potential targets. Only suspicious frames are then passed to the computationally expensive MLLM for deep reasoning. This reduces unnecessary MLLM usage and saves computation time.
Efficient Fine-Tuning Strategy
This technique improves structured understanding and output consistency by fine-tuning the MLLM using a combination of visual guidance (the image) and semantic prompting derived from small models. It uses LoRA for parameter-efficient adaptation, saving training costs.
Semantics and Bandwidth-Aware Adaptive Transmission
This method optimizes how data is streamed. It formulates the upload process as an optimization problem to minimize alert delay while maximizing visual evidence delivery, dynamically adjusting transmission based on current link state and event priority.

Terminology

Summary

Multimodal large language models (MLLMs) have shown strong capability in semantic understanding and visual reasoning, yet their use on continuous video streams in bandwidth-constrained edge-cloud systems incurs prohibitive computation and communication overhead and hinders low-latency alerting and effective visual evidence delivery. p. 0 To reduce unnecessary deep reasoning costs, the paper proposes a collaborative smalllarge model cascade where a lightweight edge-side small model acts as a gating module to filter non-target frames and perform object detection, triggering MLLM inference only for suspicious frames. p. 0 Building on this, an efficient fine-tuning strategy with visual guidance and semantic prompting is introduced to improve structured event understanding, object detection, and output consistency. p. 0 Finally, a semantics and bandwidth-aware multi-stream adaptive transmission optimization method is devised to ensure low-latency semantic alerting and effective visual evidence supplementation under bandwidth constraints. p. 0

Pruning redundant reasoning tokens

The paper introduces DAT as the first edge-cloud collaborative transmission method that jointly supports efficient cascaded semantic understanding on the inference side and low-latency adaptive multi-stream transmission on the transmission side. DAT is designed with two complementary objectives: reducing redundant MLLM invocations while enhancing task-oriented structured semantic generation through cascaded small–large model collaboration and task-adaptive fine-tuning, and enabling low-latency alerting and timely supplementation of visual evidence via a semanticbandwidth aware adaptive multi-stream transmission strategy. p. 1

How the training signal is built

The efficient fine-tuning strategy involves taking the boxed image ˜It together with a task prompt pt constructed from small-model priors as input to the MLLM, which produces structured semantic output yt=fl(˜It, pt; θ∗), where θ∗ = θ + Δθ. To support downstream transmission scheduling, a priority output qt = l t, r t is retained. p. 3 This output provides coarsegrained prioritization through lt and fine-grained semantic severity through rt. p. 3 To avoid the cost of full fine-tuning, DAT adopts LoRA for parameter-efficient adaptation. p. 3

Where the gains come from

The primary gains stem from jointly optimizing cascade inference and elastic transmission. The adaptive transmission strategy formulates multimodal data stream uploading as a lexicographic optimization problem with minimizing weighted semantic alert delay as the primary objective and maximizing effective visual evidence delivery as the secondary objective. p. 1 The online hierarchical greedy approximation is used to solve this problem due to computational constraints at the edge. p. 4

Transmission optimization mechanism

The adaptive transmission strategy formulates multi-stream upload in interval τ as an online optimization problem jointly determined by the link state Bτ and the event semantic priority Si. The core of the method is a two-stage greedy prioritization where the semantic stage selects events based on priority gain per unit semantic cost, and the visual stage allocates residual budget to events with higher gain per unit visual cost. p. 5 This hierarchical greedy scheduling approximates the lexicographic optimization problem by first prioritizing structured semantic information for high-value events and then supplementing visual evidence when the residual budget permits. p. 4 The resulting performance shows that DAT reduces weighted semantic alert delay by up to 77. p. 05% under severe congestion and delivers 98. p. 033% of visual evidence within 0. p. 0 p. 05 s. The overall time and space complexities are O(Nτ log Nτ) and O(Nτ). p. 5

Experimental validation

In experiments using the Accidents Detection Dataset, DAT achieves 98.83% recognition accuracy and 100% output consistency. p. 0 Under severe congestion, it reduces weighted semantic alert delay by up to 77. p. 05% and delivers 98. p. 033% of visual evidence within 0.5 s. The method maintains high semantic recognition accuracy and reliable localization while substantially improving transmission efficiency under severe congestion. p. 1 DAT achieves the best results across all four metrics—Parse, Schema, Rule, and Viol. p. 7—reaching 1. p. 70 for the first three and 0. p. 70 for violation rate in structured output compliance. p. 6 The method outperforms competing methods on all VTR-based visual timeliness metrics under burst conditions. p. 7 The advantage of DAT lies in how it coordinates multi-stream transmission under dynamic bandwidth constraints rather than relying on the accuracy of pre-assigned priorities alone. p. 8 The results confirm that the proposed dual-aware scheduling can more precisely prioritize semantic carriers and subsequently supplement visual evidence, thus improving transmission efficiency under severe congestion. p. 7 The method demonstrates its effectiveness in joint optimization of cascade inference and elastic transmission. p. 0 The paper concludes that DAT is an efficient framework for MLLM inference and transmission in edge-cloud systems. p. 8 The work was supported by the National Key R&D Program of China (2023YFB4502805), the National Natural Science Foundation of China (62072440), and the Beijing Natural Science Foundation (L221004). p. 8 The corresponding author is Wen Ji. p. 8

--- Page 1 ---

DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Trovato et al.

Qi Guo, Zheming Yang, Yunqing Hu, Chang Zhao, and Wen Ji. p. 0 p. 0 2018. DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems. p. 0 p. 0 Keywords Edge-Cloud Collaboration, Bandwidth-Aware Scheduling, Multimodal Large Language Models, Video Transmission. p. 0

--- Page 2 ---

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Trovato et al.

Figure 1: Comparison of Traditional and DAT Service Pipelines. p. 1 p. 1

--- Page 3 ---

DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Trovato et al.

Figure 2: Overview of DAT. p. 2 p. 2

--- Page 4 ---

DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Trovato et al.

Figure 3: Workflow of DAT: Semantic-Bandwidth Aware Multi-Stream Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems. p. 3 p. 3

--- Page 5 ---

DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Trovato et al.

Figure 6: Validation under Different Priority Sources. p. 7 p. 7 p. 7

--- Page 6 ---

DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Trovato et al.

Algorithm 1 Semantic-Priority and Bandwidth-Aware Adaptive Transmission Optimization. p. 5 p. 5 p. 5

--- Page 7 ---

DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Trovato et al.

Table 3: Overall Performance of Semantics-Bandwidth Aware Multi-Stream Adaptive Transmission Method. p. 7 p. 7

--- Page 8 ---

DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems Conference acronym ’XX, June 03–05–2018, Woodstock, NY Trovato et al.

Table 2: Structured Output Format Compliance Evaluation. p. 7

--- Page 9 ---

DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems Conference acronym ’XX, June 03–05–2018, Woodstock, NY Trovato et al.

Figure 6: Validation under Different Priority Sources.

--- Page 10 ---

DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems Conference acronym ’XX, June 03–05–2018, Woodstock, NY Trovato et al.

Algorithm 1 Semantic-Priority and Bandwidth-Aware Adaptive Transmission Optimization.

--- Page 1 ---

DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems Conference acronym ’XX, June 03–05–2018, Woodstock, NY Trovato et al.

Qi Guo, Zheming Yang, Yunqing Hu, Chang Zhao, and Wen Ji. 2018. DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems.

--- Page 2 ---

Conference acronym ’XX, June 03–05–2018, Woodstock, NY Trovato et al.

Figure 1: Comparison of Traditional and DAT Service Pipelines.

--- Page 3 ---

DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems Conference acronym ’XX, June 03–05–2018, Woodstock, NY Trovato et al.

Figure 2: Overview of DAT.

--- Page 4 ---

DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems Conference acronym ’XX, June 03–05–2018, Woodstock, NY Trovato et al.

Figure 3: Workflow of DAT: Semantic-Bandwidth Aware Multi-Stream Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems.

--- Page 5 ---

DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems Conference acronym ’XX, June 03–05–2018, Woodstock, NY Trovato et al.

Figure 6: Validation under Different Priority Sources.

--- Page 6 ---

DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems Conference acronym ’XX, June 03–05–2018, Woodstock, NY Trovato et al.

Algorithm 1 Semantic-Priority and Bandwidth-Aware Adaptive Transmission Optimization.

--- Page 7 ---

DAT: Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems Conference acronym ’XX, June 03–05–2018, Woodstock, NY Trovato et al.

Table 3: Overall Performance of Semantics-Bandwidth Aware Multi-Stream Adaptive Transmission Method.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Dual-Aware Adaptive Transmission for Efficient Multimodal LLM Inference in Edge-Cloud Systems can achieve high-quality semantic generation, low latency event alerting, and effective visual evidence supplementation by reducing unnecessary deep reasoning costs through a collaborative smalllarge model cascade.

  2. The system can improve structured event understanding, object detection, and output consistency by employing an efficient fine-tuning strategy with visual guidance and semantic prompting that uses LoRA for parameter-efficient adaptation.

  3. The AI system can deliver low latency alerting and timely supplementation of visual evidence under bandwidth constraints by devising a semantics and bandwidth-aware multi-stream adaptive transmission optimization method that formulates multimodal data stream uploading as a lexicographic optimization problem.

  4. Under severe congestion, the system can reduce weighted semantic alert delay by up to 77.5% and deliver 98.33% of visual evidence within 0.5 s, demonstrating the effectiveness of jointly optimizing cascade inference and elastic transmission through an online hierarchical greedy scheduling process.

  5. The system can perform superior structured output compliance, achieving a Parse, Schema, Rule, and Viol rate of 1.0 for the first three metrics and 0.0 for violation rate when generating structured semantics from small-model-guided inputs at limited training cost using LoRA.

Sources

Related papers