T2J-BENCH: Benchmarking Coding Agents on Execution-Grounded PyTorch-to-JAX Repository Migration
The paper introduces T2J-Bench, a benchmark for migrating PyTorch code to JAX by framing conversion as transfer under a fixed equivalence contract. It tests coding agents on whole-repository migrations across 63 tasks. Findings show that existing benchmarks fail because they don't require actual behavioral equivalence, and current agents overestimate success significantly when validated against the fixed evaluator.