Evaluating Neural Decompilation of Dart AOT Binaries: Fine-Tuning, Metric Validity, Specification Leakage, and Reliability
Researchers tested six fine-tuned models for neural decompilation of Dart code using three metrics: CodeBLEU, compile@k, and pass@k. The main finding is that fine-tuning does not improve functional correctness (pass@k), although surface metrics like CodeBLEU and compile@k can improve significantly while pass@k regresses. This suggests surface metrics can mislead users about a model's true performance.