ORACLE: Anticipating Scams from Partial Trajectories in Streaming App Usage
summary
The gist
Smartphone scams are increasingly prevalent and typically manifest as multi-stage, cross-application processes with gradually emerging intent <ref:2605.16363#pg2>.
In short
ORACLE is an agentic framework designed to predict smartphone scams early by analyzing streaming app usage trajectories. It tackles the challenge of fragmented, long-term evidence by using a self-evolving context manager and an on-policy self-distillation scheme. This approach allows the system to detect subtle, multi-stage threats hidden within normal user behavior before they become explicit.
Key concepts
- Streaming App Usage Trajectories
- This refers to the sequence of applications a user interacts with over time. The paper uses this continuous stream of data—like opening apps in a specific order—as the primary evidence to detect scams that unfold gradually across different apps and days.
- Self-Evolving Context Manager
- This module adaptively consolidates historical interactions over time. Instead of looking at isolated events, it learns to build a richer, entity-centric memory by refining its understanding of what is happening in the user's overall activity history.
- On-Policy Self-Distillation Scheme
- This technique involves training a student model using knowledge distilled from a teacher model. The teacher uses anti-scam reflections to guide the student, enabling it to recognize faint, early signals of fraud that are otherwise hidden in partial or normal user actions.
Terminology used across episodes
This episode discusses
- ORACLE: Anticipating Scams from Partial Trajectories in Streaming App Usage · Paper Radio
- An Overview of 7726 User Reports: Uncovering SMS Scams and Scammer Strategies
- Qwen2-Audio Technical Report
- CASE: An Agentic AI Framework for Enhancing Scam Intelligence in Digital Payments
- XSkill: Continual Learning from Experience and Skills in Multimodal Agents
- ScamGPT-J: Inside the Scammer's Mind, A Generative AI-Based Approach Toward Combating Messaging Scams
- Anticipate, Simulate, Reason (ASR): A Comprehensive Generative AI Framework for Combating Messaging Scams
- Kimi K2.5: Visual Agentic Intelligence
- SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning · Paper Radio
- Qwen3 Technical Report
- Online Experiential Learning for Language Models
- On-Policy Context Distillation for Language Models
- GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
- Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
The paper
ORACLE: Anticipating Scams from Partial Trajectories in Streaming App Usage · Read on arXiv
Wenbo Gao, Songbai Tan, Zhongan Wang, Fei Shen, Gang Xu, Huiping Zhuang, ¥yunyun Yang*, 世 Li‡† Ming Li受,
Harbin Institute of Technology, Shenzhen · Shenzhen University · Zhejiang University · National University of Singapore · Guangming Laboratory · South China University of Technology · Hainan University
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "ORACLE: Anticipating Scams from Partial Trajectories in Streaming App Usage".
Jane: Here's the gist of the paper. Smartphone scams are increasingly prevalent and typically manifest as multi-stage, cross-application processes with gradually emerging intent <ref:2605.16363#pg2>.
Tom: This paper proposes ORACLE Online Reasoning for Anticipating Cross-temporal Latent threats, the first agentic framework for early scam anticipation from streaming app-usage trajectories <ref:2605.16363#pg2>.
Jane: Problem Formulation The investigation reveals that decisions must rely on partial trajectories with temporally distributed evidence <ref:2605.16363#pg3>.
Tom: Real-world scams rarely appear as isolated malicious events, but typically unfold as a temporal multi-stage multi-application process <ref:2605.16363#pg3>.
Paper summary: Tom: So this paper is about this system called ORACLE, and it's trying to predict scams before they even fully become obvious. Basically, the authors are looking at how people use their streaming apps over time and trying to spot patterns that mean a scam is coming soon.
Jane: That sounds really important because scams aren't just one quick thing anymore; they're these multi-stage things that happen across different apps and spread out over a long time, so you need to see the whole picture to catch it.
Lu: Exactly, the core idea is that decisions have to be made based on partial trajectories where the evidence isn't all in one place or at one time <ref:2605.16363#pg3>. They propose this agentic framework for early scam anticipation from streaming app usage trajectories <ref:2605.16363#pg2>.
Meng: So what’s the actual claim here? Are they saying this is going to stop scams before they start, or just give us a heads-up when things get sketchy? Because we need something that works on real traffic, not just clean data.
Tom: They claim it's the first agentic framework for early scam anticipation from streaming app usage trajectories <ref:2605.16363#pg2>. And they set up a really realistic benchmark, covering twelve scam types and spanning extended periods on average of fifteen days <ref:2605.16363#pg1>.
Jane: Fifteen days is a long time for a scam to unfold, which explains why the evidence is so scattered across different apps and applications <ref:2605.16363#pg2>. They are testing this on ninety-five different apps too, mixing normal behavior with scam behaviors <ref:2605.16363#pg1>.
Lu: Their main mechanism to handle that fragmented evidence is a self-evolving context manager <ref:2605.16363#pg5>. This thing adaptively consolidates entity related interactions over time to rebuild the picture from partial observations <ref:2605.16363#pg5>.
Meng: A context manager sounds complicated when you think about deploying it at scale, but if it can effectively reconstruct cross temporal evidence, that's useful. How does it actually handle the sheer volume of interactions?
Tom: To make sure we catch those subtle early signals, they also introduced an on policy self distillation scheme <ref:2605.16363#pg5>. This involves a teacher model that gets trained on summarized anti scam reflections and clues from skills, which then supervises a student model without seeing those reflections directly <ref:2605.16363#pg5>.
Jane: So they are essentially teaching the system to learn from expert knowledge about scams—the distilled lessons—so it can be more sensitive to those subtle early stage patterns where the intent is still hidden in normal behavior <ref:2605.16363#pg2>.
Paper summary: Lu: That distillation process allows the agent to learn from what experts have already figured out about deception, which helps sharpen its recognition of emerging fraud patterns from partial trajectories <ref:2605.16363#pg5>. This is how they bridge the gap between scattered clues and a coherent risk judgment.
Meng: If that works, it means we don't need perfect information at the start to make a decent prediction, just good enough structure to piece things together over time <ref:2605.16363#pg3>. But what about the training part? How do they actually get this system trained on this long horizon data?
Tom: They have two stages for training. First, supervised finetuning on shorter scam and normal traces to build up basic risk recognition <ref:2605.16363#pg5>. Then, they use on policy self distillation on those longer trajectories where the scam segments are mixed in with normal histories <ref:2605.16363#pg5>.
Jane: That sounds like a smart way to build it up, starting simple and then refining it by forcing it to learn from the complex, long-term scenarios <ref:2605.16363#pg9>. It’s about using the generated anti scam reflections as privileged information for the student model <ref:2605.16363#pg5>.
Lu: The training process is designed to leverage those expert reflections to supervise a student model without giving it direct access to those reflections <ref:2605.16363#pg5>. It’s about distilling the knowledge into the model parameters so it learns the pattern itself <ref:2605.16363#pg9>.
Meng: From an engineering standpoint, that sounds like a solid approach for improving sensitivity, but how much computational cost is this distillation process adding to a live system? We need something lean for production.
Tom: The experiments show that this method consistently improves early scam anticipation <ref:2605.16363#pg8>. They compared it against GPT-five point one, and ORACLE improved the precision area from seventy-seven point three to ninety-eight point two and reduced the false positive rate from forty-six point four to twenty-nine point five <ref:2605.16363#pg8>.
Jane: That’s a big jump in precision, which is great for catching the right threats, but reducing false positives means we aren't bothering people with too many warnings that turn them off <ref:2605.16363#pg8>. The ablation study points to the evolving skill consolidated from memory as achieving the best results across all metrics <ref:2605.16363#pg8>.
Lu: I agree, the consolidation of skills from memory seems to be where they get their highest HR and lowest far and par values, which suggests that cross-temporal reasoning through that mechanism is really doing the heavy lifting <ref:2605.16363#pg8>.
Meng: So, what does this mean for deploying something like ORACLE in a real environment? It seems to perform very well on the benchmark data they built, but how stable are these predictions when you move to live traffic that isn't perfectly aligned with that benchmark?
Paper summary: Tom: The conclusion is that ORACLE raises earlier and more reliable warnings than strong large language model baselines <ref:2605.16363#pg8>. It suggests that practical scam intervention needs to move past just classifying isolated content toward reasoning across time over partial behavioral trajectories <ref:2605.16363#pg11>.
Jane: It’s about using the temporal aspect of the data, not just looking at what's happening in one app session <ref:2605.16363#pg2>. It reduces the amount of sensitive information you need to gather for detection while still enabling that cross app temporal reasoning <ref:2605.16363#pg14>.
Lu: The implication is that we can detect intent by analyzing these sequences rather than just single events, which makes the detection process less data intensive in terms of the raw sensitive information needed <ref:2605.16363#pg14>.
Meng: I do have to bring up one thing: they mention that responsible deployment needs privacy preserving mechanisms and safeguards against misuse <ref:2605.16363#pg15>. That’s a necessary caveat for any system that has access to this level of behavioral data.
Tom: Right, so we’ve seen how they tackle the fragmented evidence with the context manager and how distillation helps them learn from expert reflections <ref:2605.16363#pg5>. But the real win here is using partial trajectories to anticipate threats before they escalate <ref:2605.16363#pg1>.
Jane: It’s a big step because it moves the focus from immediate detection to proactive anticipation across time <ref:2605.16363#pg8>. So, ORACLE is proposing that for scams, the danger isn't the single click or message, but the sequence of interactions itself <ref:2605.16363#pg2>.
Lu: And that sequence requires a model capable of bridging those scattered fraudulent clues using cross temporal reasoning <ref:2605.16363#pg3>. It’s about learning how intent evolves over many days and many different applications <ref:2605.16363#pg1>.
Meng: It sounds like a lot of complexity to get running smoothly in the real world, but if it can consistently yield better precision than GPT-five point one on this kind of task, maybe the engineering trade-off is worth it <ref:2605.16363#pg8>.
Tom: That’s what the numbers show, Meng. It improves precision significantly while keeping false positives down compared to the baselines we tested <ref:2605.16363#pg8>. So the takeaway is that for complex, multi-stage threats like these scams, temporal reasoning on partial data is a way to get better early warning signals <ref:2605.16363#pg1>.
Jane: It gives us a new lens for thinking about threat detection in apps—we need to stop looking at isolated events and start looking at the entire journey <ref:2605.16363#pg2>. That’s what ORACLE is suggesting we do now <ref:2605.16363#pg1>.
Conclusion: Tom: So we’ve been looking at ORACLE, which is this new framework for anticipating scams based on how people use their streaming apps over time.
Jane: That's right, Tom, and the core idea is that instead of just looking at one weird thing happening now, we look at the whole timeline of app usage to see if a scam is building up gradually.
Lu: They built this agentic framework specifically because decisions usually rely on partial trajectories with evidence spread out over time.
Meng: So, what does this mean for us in terms of actual deployment? Is this something we can actually put into a live system that handles real traffic?
Tom: Well, the authors found it consistently improves early scam anticipation compared to models like GPT-five point one, showing improvements in precision and reducing false alerts.
Jane: So, what’s the big picture here for someone just listening? It suggests that catching scams isn't just about spotting one single message or click; it’s about understanding the entire sequence of interactions across different apps.
Lu: The method they used to handle that scattered evidence is a self-evolving context manager that consolidates entity interactions over time to reconstruct the evidence better.
Meng: That sounds complicated to run at scale, but if it can effectively rebuild what's happening over days, that’s valuable for catching those multi-stage scams.
Tom: They also used this on policy self distillation scheme where a teacher model, trained on anti-scam reflections, supervises a student model without seeing those reflections directly.
Jane: That means they distilled the knowledge from what experts have already figured out about deception into a way that helps the system learn those early stage signals even when they are hidden within normal behavior.
Lu: The training involved two stages, starting with supervised finetuning and then moving to on policy self distillation on long horizon trajectories.
Meng: I'm still worried about running that kind of complex training pipeline in production, but the results on the benchmark data are pretty compelling.
Tom: The conclusion is that ORACLE raises earlier and more reliable warnings than strong large language model baselines, suggesting we need to move toward cross temporal reasoning over partial behavioral trajectories for scam intervention.
Jane: It’s a shift from reacting to immediate content to proactively anticipating threats based on the whole journey of app usage.
Lu: The paper points out that this approach reduces the amount of sensitive information needed for detection while still enabling that cross app temporal reasoning, which is pretty efficient.
Meng: But they also flag that responsible deployment requires privacy preserving mechanisms and safeguards against misuse because this is deep behavioral data.
Tom: So it’s about using the sequence of events over time to get better early warning signals, even when the intent is subtle.
Jane: And that brings us to what’s next—how do we make sure this kind of temporal reasoning works reliably when it's not just on a test dataset?
More episodes
- 2603.28595-Optimistic Actor-Critic with Parametric Policies for Linear Markov Decision Processes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound
- 2605.15085-From Data to Action: Accelerating Refinery Optimization with AI