Identifier-Free Code Embedding Models for Scalable Search

arXiv:2605.05251 · cs.CR, cs.LG, cs.SE · Submitted 2026-05-05 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.

Nadia: Today's paper: "Identifier-Free Code Embedding Models for Scalable Search".

Elias: The gist The paper presents a framework for evaluating how well embedding models can associate source code functions with their decompiled, stripped counterparts that lack identifiers like variable, parameter,

Nadia: First, who's behind it and why it matters.

Title and authors: Nadia: We started by looking at the paper "Identifier-Free Code Embedding Models for Scalable Search" and who was behind this work. The title itself tells us right away that they are focused on models that don't need identifiers to do their job.

Elias: It’s about moving beyond traditional methods where you have to rely on symbols, and this paper is looking at how far AI can go in associating source code functions with their stripped counterparts without those names.

Nadia: The authors are Eric Wolos and Michael Doyle from the MITRE Corporation. They set out to bridge a gap where existing search tools exist but don't use the full range of capabilities that AI-enabled search provides for this kind of association.

Priya: So, when you read that title, it sounds like they’re tackling a problem common in reverse engineering where you have code without any human-readable labels attached.

Elias: That's right. They formally define the function association problem and evaluate how far embedding models can bidirectionally associate between source code and decompiled, stripped code representations.

Nadia: The paper sets up a bidirectional search scenario, meaning they aren't just checking if source matches decompiled, but also if decompiled matches source using their embeddings.

Priya: That seems like a heavy lift because you're dealing with two completely different ways of looking at the same underlying logic structure.

Elias: It is heavy, but it’s necessary because standard preprocessing requirements for these models often make it difficult to handle this bidirectional connection effectively without these new techniques.

The paper's summary: Nadia: So, diving into the actual summary of "Identifier-Free Code Embedding Models for Scalable Search," they outline the motivation behind tackling this problem head-on.

Elias: The motivation is that binary reverse engineering often involves reasoning about how a particular binary works because many binaries are compiled with debug and symbol information stripped, leading to little naming information for understanding the code they analyze.

Nadia: They state that function association can be accomplished with existing search tools like BSim or FLIRT, but AI-enabled function association may supplement these rules-based methods.

Priya: So, this isn't just about building a new tool; it’s about seeing if AI can actually provide better results when the human naming information is gone.

Elias: They seek to examine search in both directions between source code and decompiled, stripped code representations to see what the models can find.

Nadia: To make this search transferable to applications, they use transformer-based embedding models with text inputs because that avoids sophisticated input preprocessing for any text that can be tokenized.

Priya: So, by using generic text embeddings instead of specialized binary parsers first, they are trying to make the system more versatile across different kinds of code analysis.

Elias: They focus on source code and the symbol-stripped, pseudo-C decompiled code that tools like Ghidra and IDA Pro generate as their specific representations.

The paper's improvements: Nadia: Now let's look at what they suggest is improved in this work, specifically the improvements they make to the model itself.

Elias: The main improvement is fine-tuning a Qwen3-Embedding model with contrastive learning to improve its ability to bidirectionally associate between source code and decompiled, stripped code.

Nadia: This fine-tuning uses an InfoNCE Loss objective where the anchor representations are the decompiled functions, and the positive and negative representations are the source functions.

Priya: That's a very specific training signal; they’re essentially telling the model what a match looks like by pairing something it should find with something it shouldn't.

Elias: They fine-tune Qwen3-Embedding-0 point 6B over ninety-six four steps using AdamW as their optimizer, which gives them an effective batch size of five hundred twelve for that single epoch.

Nadia: The result of this tuning is that their new model outperforms other models on all function association baselines by a substantial margin across every metric and all pool configurations.

Priya: So the authors claim this contrastive learning approach is what lets them beat the existing models when dealing with stripped code representations.

Elias: Beyond just beating baselines, they show that the combined search pools present significantly harder tasks for models that rely on identifier names, and their model reduces this reliance.

Conclusion: Nadia: So, to wrap up this discussion on "Identifier-Free Code Embedding Models for Scalable Search," the main implications are about how much better we can expect function association to get when we use AI.

Elias: They found that their new model performs substantially better, achieving approximately five times improvement in MRR over the next best model on combined search pools during Decompiled to Source association.

Priya: Five times improvement is a big number; it means the system can find those correct function pairs much more reliably than previous methods.

Elias: They also showed that this model generalizes to a constant-algorithm association task it was not explicitly trained on, which suggests it learns meaningful features that aren't just memorizing Assemblage’s data format.

Nadia: The limitation they state is that these results might be limited by the underlying data distribution of the Assemblage dataset, as compilation only for xeighty-six means performance could deviate on binaries compiled for different architectures.

Priya: That’s a fair caveat; if you test this on a binary compiled for ARM, the performance might not hold up as well.

Elias: The intent of this work is to empower systems that might help binary reverse engineers perform defensive activities, but such systems may also be used for malicious purposes, so they recommend refraining from using high-sensitivity data for models and evaluations in future works.

Nadia: So, the paper "Identifier-Free Code Embedding Models for Scalable Search" gives us a framework to evaluate how well embedding models can associate source code functions with their decompiled counterparts lacking identifiers like variable, parameter, and function names.

Priya: It’s a solid framework for testing the limits of what these models can actually do when they have to work without explicit naming conventions.

Elias: Yeah, it really shows that learning structural relationships is possible even when you take away all the human-readable identifiers from the code.

The MITRE Corporation

cs.CR, cs.LG, cs.SE

Submitted: 2026-05-05

Updated: 2026-10-15

Code: https://github.com/JetBrains-Research/astminer

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 83/100

The gist: The gist The paper presents a framework for evaluating how well embedding models can associate source code functions with their decompiled, stripped counterparts that lack identifiers like variable,

Key concepts

Function Association: This is the process of finding matching functions between two different code representations. The paper focuses on associating standard source code functions with their corresponding decompiled, stripped versions where names like variable or function labels are missing, which is common in binary analysis.
Bidirectional Association: Instead of just searching one way (e.g., source to stripped), this concept tests the model's ability to associate both directions: source code functions with their decompiled counterparts, and vice versa. This makes the association framework more robust for analyzing binary code.
Contrastive Learning: This is a training method where the model learns by comparing different pieces of data. In this context, it involves setting 'anchor' representations (like decompiled functions) and 'positive/negative' pairs (like source functions) to teach the model what makes two codes similar or different.
Qwen3-Embedding Model: This is a large language model specifically adapted to create numerical representations (embeddings) of text. The researchers fine-tuned this specific model using contrastive learning to make its embeddings better at capturing the semantic relationship between source code and stripped code functions.

Terminology

Contrastive Learning — 1× in this paper · explained in 7 episodes

Summary

The gist The paper presents a framework for evaluating how well embedding models can associate source code functions with their decompiled, stripped counterparts that lack identifiers like variable, parameter, and function names.

Background and Motivation

Binary reverse engineering often involves reasoning about how a particular binary works because many binaries are compiled with debug and symbol information stripped, leading to little naming information for understanding the code they analyze Function association can be accomplished with existing search tools like BSim or FLIRT, but AI-enabled function association may supplement these rules-based methods. The authors seek to examine search in both directions between source code and decompiled, stripped code representations. To make this search transferable to applications, they use transformer-based embedding models with text inputs to avoid sophisticated input preprocessing. This work focuses on source code and the symbol-stripped, pseudo-C, decompiled code that tools like Ghidra and IDA Pro generate. They focus on text embedding transformers because they allow them to relax constraints on model inputs to any text that can be tokenized.

Methodology

The authors formalize the function association problem and evaluate the extent to which embedding models can bidirectionally associate between these two representations. To improve model performance at this task, they fine-tune a Qwen3-Embedding model with contrastive learning. The training involves defining anchor representations as decompiled functions and the positive/negative representations as source functions using the InfoNCE Loss objective. They set τ = 0.05 and use AdamW as their optimizer for fine-tuning Qwen3-Embedding-0.6B with an effective batch size of 512 over a single epoch of 964 steps.

Experimental Setup

The experiments utilize the Assemblage dataset, which contains nearly 500,000 function pairs (over 1M total functions across both representations). They formalize two search scenarios over two distinct search pools: Source → Decompiled, Decompiled → Source, and Filtered Search Pool (Source Candidates Only, Decompiled Candidates Only). The Combined Search Pools contain both source and decompiled functions. They also test Signsrch experiments involving algorithm names and corresponding constant values, measuring performance on two tasks: Algorithm Group Name → Constants and Constant → Other Group Constants.

Results and Discussion

The authors find that their new model outperforms other models on all function association baselines by a substantial margin. In the Assemblage experiments, their model exhibits the best performance across every metric and all pool configurations. The results show that the combined search pools present significantly harder tasks for models that rely on identifier names, and their model reduces this reliance. Furthermore, the Signsrch experiments demonstrate model generalization to a new task because their model achieves improvement over its base model and others at the Constant → Other Group Constants task despite not being explicitly trained for it. Quantization does not entirely hinder performance, as their FP8-quantized model performs comparably to the unquantized version on most benchmarks.

Conclusion

The authors present a framework for evaluating how well embedding models can associate source code functions with their decompiled, stripped counterparts that lack identifiers like variable, parameter, and function names. They demonstrate that contrastive learning on a paired corpus such as Assemblage can substantially improve function association by achieving approximately 5x improvement in MRR over the next best model on combined search pools during Decompiled → Source association. Their results show that the combined search pools, which include both representations, present significantly harder tasks for models that rely on identifier names, and that their model reduces this reliance. Finally, they show that their model generalizes to a separate constant-association task it was never trained on, indicating that it learns broader algorithmic features not dependent on identifier names.

Limitations and Impact

These results may be limited by the underlying data distribution of the Assemblage dataset, as compilation only for x86 means that model performance may deviate on binaries compiled for different architectures. The intent of this work is to empower systems that might help binary reverse engineers perform defensive activities, but such systems may also be used for malicious purposes. To mitigate such harm, they recommend future works to refrain from using high-sensitivity data for models and evaluations.

Acknowledgments

Thank you to Peter Lucia, Corre Steele, Dr. Chris Niessen, and Marissa Dotter for their helpful feedback during peer review. Thank you to Dr. Stan Barr for guiding this work, thoroughly reviewing it, and supporting funding acquisition. This research was funded by MITRE’s Independent Research and Development Program.

References

[1] National Security Agency. BSim: Tutorial, 2026. URL https://ghidra-sre.org/GhidraClass/BSim/README.html

[2] Hex-Rays. FLIRT: Fast library identification and recognition technology, 2024. URL https://docs.hex-rays.com/user-guide/signatures/flirt

[3] Igor Popov. Malware detection using machine learning based on word2vec embeddings of machine code instructions. In 2017 Siberian Symposium on Data Science and Engineering (SSDSE), pages 1–4, 2017. doi: 10.1109/SSDSE.2017.8071952

[4] Bolun Zhang, Zeyu Gao, Hao Wang, Yuxin Cui, Siliang Qin, Chao Zhang, Kai Chen, and Beibei Zhao. Binquery: A novel framework for natural language-based binary code retrieval. Proc. ACM Softw. Eng., 2(ISSTA), June 2025. doi: 10.1145/3728927

[5] Guoqiang Chen, Lingyun Ying, Ziyang Song, Daguang Liu, Qiang Wang, Zhiqi Wang, Li Hu, Shaoyin Cheng, Weiming Zhang, and Nenghai Yu. Cross-modal retrieval models for stripped binary analysis

[6] Yuanda Wang, Ji Zhou, Xinhui Han, and Chao Zhang. Bdlf-qwen3: Enhanced cross-architecture binary function similarity detection through binary dynamic layer fusion. Proceedings of the AAAI Conference on Artificial Intelligence, 40(2):1222–1230, Mar. 2026. doi: 10.1609/aaai/v40i2.37094

[7] Yifan Zhang, Chen Huang, Yueke Zhang, Huajie Shao, Kevin Leach, and Yu Huang. Pretraining representations of binary code using contrastive learning

[8] Xuezixiang Li, Yu Qu, and Heng Yin. Palmtree: Learning an assembly language model for instruction embedding. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, CCS ’21, page 3236–3251. ACM, November 2021. doi: 10.1145/3460120.3484587

[9] Y. Jia, Z. Yu, and Z. Hong. Semantic aware-based instruction embedding for binary code similarity detection

[10] Zeping Yu, Wenxin Zheng, Jiaqi Wang, Qiyi Tang, Sen Nie, and Shi Wu.

Improvements for AI systems

Bold header: Contrastive Learning for Bidirectional Association

This improvement involves fine-tuning a Qwen3-Embedding model with contrastive learning using an InfoNCE loss to improve its ability to bidirectionally associate between source code and decompiled, stripped code. This allows the improved system to perform more robust function association by leveraging functional equivalence even when identifier information is lost, as demonstrated by the finding that our new model outperforms other models on all function association baselines by a substantial margin.

Bold header: Enhanced Discrimination Between Representations

The system will be improved to better discriminate between different code representations, particularly in combined search pools where decompiled functions share similar or identical variable names due to naming conventions like Var1, Var2, param 1, param 2. This addresses the limitation where models that rely too heavily on identifier names may match on naming similarity rather than functional equivalence, leading to better performance across Both Representations Search Pool tasks.

Bold header: Generalization to Unseen Association Tasks

The model will be enhanced to generalize its learned features beyond the training data format by achieving performance on tasks it was not explicitly trained on, such as the Signsrch experiments where our model achieves improvement over its base model and others at the Constant → Other Group Constants task. This suggests the system learns meaningful features and has not simply overfit to Assemblage’s data format and composition.

Bold header: Effective Model Compression for Deployment

By testing inference with FP8 quantization, the system can be optimized for low-compute environments where conserving VRAM enables larger batch sizes and higher throughput. This allows the improved AI system to maintain high performance while being viable in production settings requiring low-compute environments.

Abstract

Function association is a useful process for binary reverse engineers. Search tools exist to perform association at scale, but they do not utilize the full range of capabilities that AI-enabled search provides. Prior work has explored the development of embedding models for association between certain reverse engineering code representations, but that work does not cover bidirectional association between source code and decompiled, stripped code with standard preprocessing requirements. To bridge this gap, we formalize this function association problem and evaluate the extent to which embedding models can bidirectionally associate between these two representations. To improve model performance at this task, we fine-tune a Qwen3-Embedding model with contrastive learning. We find that our new model outperforms other models on all function association baselines by a substantial margin and generalizes to a constant-algorithm association task it is not explicitly trained on.