Evaluating Language Models on Cross-Language Code Functional Equivalence

arXiv:2608.23961 · cs.SE, cs.AI, cs.CL · Submitted 2026-08-25 · Read on arXiv

cs.SE, cs.AI, cs.CL

Submitted: 2026-08-25

Updated: 2026-08-25

Comments: 20 pages. To appear in the 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026), LIPIcs vol. 394, Article No. 41

Code: https://github.com/dflook/python-min

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

Accuracy — 26× in this paper · explained in 1 episode
Benchmark — 19× in this paper · explained in 2 episodes
Abstraction — 16× in this paper · explained in 1 episode
Failure Modes — 4× in this paper · explained in 3 episodes
Semantic Understanding — 4× in this paper · explained in 2 episodes
Benchmarking — 2× in this paper · explained in 4 episodes
Foundation Model — 2× in this paper · explained in 3 episodes
Foundation Models — 2× in this paper · explained in 7 episodes
Large Language Models (LLMs) — 2× in this paper · explained in 8 episodes
Multi-Agent System — 2× in this paper · explained in 7 episodes
Symposium on Empirical Software Engineering and Measurement — 2× in this paper
Systematic Bias — 2× in this paper · explained in 3 episodes
Trace — 2× in this paper · explained in 2 episodes
Verification — 2× in this paper · explained in 2 episodes
AI Foundation Model — 1× in this paper
Acceptance Bias — 1× in this paper · explained in 1 episode
Chain-of-Thought (CoT) — 1× in this paper · explained in 5 episodes
Fingerprinting — 1× in this paper · explained in 2 episodes
Generative AI — 1× in this paper · explained in 2 episodes
Hallucination — 1× in this paper · explained in 4 episodes
Human-in-the-Loop — 1× in this paper · explained in 3 episodes
Semantic Similarity — 1× in this paper · explained in 2 episodes

Evaluating Language Models on Cross-Language Code Functional Equivalence

Terminology

Sources

Related papers