Towards LLM-Based Analysis of Virtualization-Obfuscated Code through Automated Data Generation
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: We're looking at the paper: "Towards LLM-Based Analysis of Virtualization-Obfuscated Code through Automated Data Generation".
Elias: Here's the gist of the paper. Virtualization-based obfuscation produces extremely large and structurally complex binaries, posing challenges for LLM-based analysis due to input size limits and the need for large-scale labeled data.
Nadia: We address this by focusing on structural rather than full semantic analysis.
Elias: Obfuscated binaries are decomposed into the largest semantically coherent units that fit within LLM constraints and are labeled according to their structural roles.
Nadia: We implement a static analysis framework to automate labeling and enable large-scale dataset generation.
Title and authors: Nadia: So we're looking at the paper, "Towards LLM-Based Analysis of Virtualization-Obfuscated Code through Automated Data Generation," and it seems like they’re tackling a real headache for anyone trying to analyze these heavily protected binaries.
Priya: But what they really want is for the AI to learn the structure itself rather than just trying to read every single line of assembly instruction.
Nadia: Right, and they achieve this through three stages: first, getting ground-truth labels automatically during obfuscation to solve that data scarcity problem, then training a BERT model to classify those blocks based on context, and finally connecting those blocks back together for visualization.
Elias: That BERT model part sounds like the key component here because it's what actually learns the contextual relationships between the assembly instructions and assigns them block roles.
Priya: Before we get into how they do that, I want to ask what the actual output looks like for a person trying to reverse engineer something complicated.
Nadia: So, what does this automated labeling framework actually generate for someone who is trying to de-obfuscate code?
Elias: It generates these labels—like Dispatch Start, Handler, VM Start, and VM End—which are then used to reconstruct the virtualized execution flow and create a control flow graph.
Priya: That visualization part sounds really useful because it gives an analyst a visual map of where the interpreter is jumping around in the code.
Nadia: It does, and they claim this allows analysts to directly grasp the core execution logic of obfuscated programs, which is much better than just looking at raw assembly.
The paper's summary: Elias: Now we’re digging into the summary of "Towards LLM-Based Analysis of Virtualization-Obfuscated Code through Automated Data Generation." What they are saying is that virtualization obfuscation makes static analysis really hard because it turns original instructions into virtual ones executed by an interpreter.
Nadia: They point out that understanding this internal logic requires identifying three main parts: the Virtual Program Counter, the Dispatcher, and the Handler, with the dispatcher acting as a central hub and handlers doing the actual work.
Priya: It sounds like they are focusing on these core VM components rather than trying to analyze every single instruction individually at first.
Elias: Precisely; their framework is designed to distinguish between those VM-related blocks and other mixed control flow regions, which is a big step in making sense of the structure.
Nadia: The main contribution they highlight is designing this whole framework—it’s a pipeline with three stages: automated labeling for data generation, BERT classification for block roles, and then visualization to reconstruct the execution flow.
Priya: I'm curious about that first stage they mentioned; they generate ground-truth labels during the obfuscation process itself so they can make this large dataset.
Elias: That’s the part that addresses the data scarcity problem for LLM analysis, so you don't have to spend all your time manually annotating code.
Nadia: The framework defines four specific block types within these regions: Dispatch Start, Handler, VM Start, and VM End. These are then used to define the structural roles of each part of the virtualized region.
The paper's improvements: Nadia: Moving on to what they actually improved in this work, they focus on the methodology—the way they build that framework. They introduced this three-stage process as a way to overcome the lack of labeled data for these types of binaries.
Elias: And then there’s the BERT-based identification model, which they claim achieves up to ninety-nine point eight percent accuracy in classifying those key virtualization components, even when the binary is already optimized <ref:2605.09961#pg1>.
Priya: Ninety-nine point eight percent sounds high, but I want to know what that means for real-world analysis; does it hold up when you look at actual complex obfuscators?
Nadia: It shows they've built a model that can learn those contextual relationships between assembly instructions and assign the correct block roles with very high precision.
Elias: They also improved the visualization part, providing a tool that reconstructs the logical execution flow of virtualized code, which significantly cuts down on the manual effort reverse engineers usually have to do.
Priya: So it’s not just about classification accuracy; it’s about giving analysts something concrete they can look at and actually use to understand the program's behavior.
Conclusion: Nadia: So wrapping up the "Towards LLM-Based Analysis of Virtualization-Obfuscated Code through Automated Data Generation" paper, they’ve put forward a system that automates the creation of large datasets for this type of analysis and uses BERT to identify core VM components with very high accuracy.
Elias: They also provide this visualization tool that lets analysts see the logical execution flow, which helps them understand things like how the dispatcher talks to the handlers.
Priya: I just want to stress that while they've achieved high classification performance, we need to keep in mind what they say about their limitations.
Nadia: They admit that their framework is designed around distinguishing VM-related blocks from mixed control flow regions, but they also mention that the performance relies on the quality of the ground-truth labels generated during obfuscation.
Elias: Another limitation they point out is that while they handle different structures, like threaded designs in dispatchers, there are still various modes of virtualization—like Switch or Direct structures—that might require different tuning for their framework to work optimally.
Priya: So the core limitation seems to be that the quality of the input data directly impacts how good the AI's classification becomes.
Nadia: Exactly, and that's where we’ll need to watch future work to see if they can make those ground-truth labels easier or more scalable to generate for every single type of obfuscator out there.
Elias: They are definitely setting a strong foundation for structural analysis, so the next step is seeing how this framework handles even more complex or different virtualization schemes.
Sangjun An, Hyeyeon Park, Yejin Son, Seoksu Lee, Eun-Sun Cho
Chungnam National University
cs.CR
Submitted: 2026-05-11
Updated: 2026-11-09
Comments: arXiv admin note: This submission has been withdrawn because it does not meet arXiv's research content quality standards
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 88/100
The gist: Virtualization-based obfuscation produces extremely large and structurally complex binaries, posing challenges for LLM-based analysis due to input size limits and the need for large-scale labeled…
Key concepts
- Virtualization-based Obfuscation
- This technique transforms original program instructions into a set of virtual instructions that an interpreter executes. This makes static analysis very difficult because the actual logic is hidden inside this custom virtual machine structure.
- Structural Labeling Framework
- A pipeline that automatically analyzes assembly code at the intermediate representation level to identify core structural components like Dispatchers and Handlers. This process segments large binaries into smaller, semantically coherent blocks for easier LLM processing.
- BERT-based Component Classification
- A deep learning model trained to learn the contextual relationships between assembly instructions and their functional roles within the virtual machine. It accurately identifies key parts of the obfuscated code, such as Dispatchers and Handlers, with high precision.
- CFG-based Visualization
- A method that reconstructs and visualizes the logical execution flow of virtualized code using Control Flow Graphs. This helps analysts directly see how the program executes by highlighting relationships between different parts of the virtual machine.
Terminology
Summary
Virtualization-based obfuscation produces extremely large and structurally complex binaries, posing challenges for LLM-based analysis due to input size limits and the need for large-scale labeled data. p. 0 We address this by focusing on structural rather than full semantic analysis. p. 0 Obfuscated binaries are decomposed into the largest semantically coherent units that fit within LLM constraints and are labeled according to their structural roles. p. 0 We implement a static analysis framework to automate labeling and enable large-scale dataset generation. p. 0 Our prototype shows strong performance on real-world virtualization obfuscators. p. 0
Pruning redundant reasoning tokens
Virtualization obfuscation is considered one of the most powerful approaches. It translates original instructions into virtual instructions executed by an interpreter, significantly complicating static analysis. p. 0 Efficiently understanding the internal logic of virtualized binaries has become a critical challenge. p. 0 We propose a framework for statically identifying the core structures of virtualization-obfuscated binaries. p. 0 This shift in perspective enables not just detection, but a granular structural identification of internal Virtual Machine (VM) components—such as dispatchers and handlers—which is essential for deep reverse engineering and de-obfuscation. p. 0
How the training signal is built
The framework consists of three stages. First, ground-truth labels are automatically generated during obfuscation to enable large-scale dataset construction. p. 2 Second, a BERT-based model is trained to learn contextual relationships between assembly instructions and classify block roles. p. 2 Finally, the identified blocks are connected to reconstruct and visualize the virtualized execution flow. p. 2 The automated structural labeling pass analyzes control flows at the intermediate representation level to capture virtualization-specific control patterns. p. 2
Where the gains come from
The framework defines four core block types within virtualized regions: Dispatch Start, Handler, VM Start, and VM End. The automated structural labeling pass identifies the Dispatch Start as the block with the maximum out-degree in Algorithm 1. p. 2 Handler blocks are labeled as those directly reachable from the dispatcher. p. 3 VM Start is marked by the predecessor of the dispatcher, and VM End marks exits from the virtualization region. p. 3 This strategy reduces input length by an average of 98. p. 353% by segmenting assembly code into basic-block units and merging adjacent blocks sharing the same label. p. 3
Analysis of Virtualization Obfuscation Logic and Framework Design
Virtualization obfuscation transforms original instructions into virtual instructions executed by an interpreter[5, 11]. Understanding its internal organization requires identifying three core components: VPC (Virtual Program Counter), Dispatcher, and Handler. p. 1 The dispatcher serves as a central hub, while handlers implement the decomposed execution logic. p. 2
Experimental Results and Analysis
The BERT model achieved 99.8% accuracy in classifying key virtualization components (Dispatcher, Handler, etc. p. 1), even in optimized binaries. p. 1 BertTokenizer outperformed Palmtree (91. p. 57% vs. 83.8%) while maintaining comparable sub-label performance. p. 4 The macro F1-score achieved by BertTokenizer was 0. p. 4998, slightly outperforming Palmtree (0. p. 4996). CFG-based visualization allows analysts to directly grasp the core execution logic of obfuscated programs. p. 6
Conclusion
In this study, we proposed an LLM-based framework for automatically identifying and visualizing the execution structures of virtualization-obfuscated code. We established an automated structural labeling pipeline that enables large-scale dataset construction without manual intervention. p. 6 We demonstrated high identification performance using a BERT-based model that classifies core components with up to 99. p. 68% accuracy. p. 1 Finally, we enhanced analysis interpretability through CFG-based visualization of identified virtualization regions. p. 6 The proposed structure-oriented learning framework is extensible to other virtualization-based obfuscators such as VMProtect and Code Virtualizer. p. 6
REFERENCES
[1] C. Collberg, C. p. 6 Thomborson, and D. p. 6 Low, “A taxonomy of obfuscating transformations,” Department of Computer Science, University of Auckland, New Zealand, Tech. p. 6 Rep. 148.
[2] V. J. M. Salis and A. p. 6 S. Sani, “A survey of code obfuscation techniques,” ACM Comput. p. 6, vol 53, no 3, pp 1–36, 2020. p. 6
[3] N. Zhang, et al. p. 6, “Inspecting Virtual Machine Diversification Inside Virtualization Obfuscation,” IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA, 2025, pp 3051–3069. p. 6
[4] S. Li, et al., “Chosen-Instruction Attack Against Commercial Code Virtualization Obfuscators,” 29th Annual Network and Distributed System Security Symposium, NDSS 2022, San Diego, California, USA, April 24–28, 2022. p. 6
[5] J. Teschke and T. p. 6 Kurth, “Virtualization-based obfuscation: A survey,” in Proc. p. 6 12th Int. Conf. Availability, Rel. p. 6 Secur., (ARES), 2017, pp 1–10. p. 6
[6] B. Kinder, “The tigress diversifying c compiler,” 2023. p. 6
[7] C. Lattner and V. p. 6 Adve, “LLVM: A compilation framework for lifelong program analysis & transformation,” in Proc. p. 6 Int. Symp. Code Gener. p. 6 Optim., (CGO), 2004, pp 75–86. p. 6
[8] J. Devlin, M. W. Chang, K. Lee, and K. p. 0 Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” 2018. p. 7
[9] B. Li, S. Feng, and G. p. 7 p. 7 Tan, “Palmtree: Learning an assembly language model for instruction embedding,” in Proc. p. 7 2021 ACM SIGSAC Conf. p. 7 Comput. Commun. Secur., (CCS), 2021, pp 3236–3251. p. 7
[10] H. Pei, B. G. Chun, and S. p. 7 Kim, “DeepBinDiff: Learning program-wide code representations for binary diffing,” in Proc. p. 7 27th Network Distrib. p. 7 Syst. Secur. Symp., (NDSS), 2020. p. 7
[11] G. Luan, S. Feng, and G. Tan, “Identifying virtualized code via deep learning,” in Proc. p. 7 36th IEEE/ACM Int. p. 7 Conf. Autom. Softw Eng., (ASE), 2021, pp 1178–1189. p. 7
[12] X. Wang, J. Yang, and Z. p. 7 Lin, “BinBert: Binary code understanding with BERT,” in Proc. p. 7 32nd IEEE Int. p. 7 Conf. Softw Maint. Evol., (ICSME), 2022. p. 7
[13] S. Vaswani et al. p. 7, “Attention is all you need,” in Proc. p. 7 31st Int. Conf. Neural Inf Process. p. 7 Syst., (NIPS), 2017, pp 5998–6008. p. 7
[14] D. Yoo et al., “CNN based Virtualized Obfuscated Malware Detection Technique,” in Proc. p. 7 Symposium of the Korean Institute of Communications and Information Sciences, 2024, pp 637–638. p. 7
[15] S. Park and Y. p. 0 Park, “Analysis of anti-reversing functionalities of vmprotect and bypass method using pin,” KIPS Trans. p. 7 Comp. Comm. Syst., vol 10, no 11, pp 297–304, 2021. p. 7
[16] D. Xu et al., “VMHunt: A verifiable approach to partially-virtualized binary code simplification,” in Proc. p. 7 ACM CCS, 2018, pp 442–458. p. 7
[17] A. Kalysch et al. p. 7, “VMAttack: Deobfuscating virtualization-based packed binaries,” in Proc. p. 7 ARES, 2017, pp 1–10. p. 6
[2605.09961v1] arXiv:2605. p. 009961v1 [cs. p. 0CR] 1 May 2026. p. 0
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Automated Ground-Truth Dataset Generation: The system can generate
large-scale ground-truth datasets for virtualization analysis
by implementing anautomated labeling framework that overcomes the data scarcity problem.
This allows AI systems to train on vast amounts of labeled data for structural identification, moving beyond manual annotation costs. -
Contextual Component Classification: A BERT-based model can be developed to classify key virtualization components such as
Dispatcher, Handler, etc., with up to 99.8% accuracy in classifying key virtualization components (Dispatcher, Handler, etc.), even in optimized binaries.
This enables AI systems to perform granular structural identification of internal Virtual Machine (VM) components automatically. -
CFG-Based Visualization for Interpretability: The system can reconstruct and visualize the
logical execution flow of virtualized code
by using BERT inference results to create acolor-coded CFG.
This allows analysts todirectly grasp the core execution logic of obfuscated programs
by highlightingVM entry/exit points and dispatcher-handler relationships.
-
Robustness Across Obfuscation Types: The framework is designed to handle different virtualization structures, as it can identify dispatch structures across various modes including
Switch, Direct, and Indirect,
demonstrating robustness even when facing thread designs or compiler optimizations.
Sources
Related papers
- From Strategy Search to Policy Learning: MCTS-Guided Adversarial Co-Evolution for LLM Safety Alignment
- Privacy Leakage from Recurrent State in LLM Inference
- Incrementally Verifiable Oracle Aided Computation
- Privilege and confidentiality in generative AI workflows
- BadMoE: Backdooring Mixture-of-Experts LLMs via Routing-Aware Triggers and Task-Underutilized Experts
- CoVault: Secure, Scalable Analytics of Personal Data