Goal Retrieval versus Action-Value Ordering: Contrastive Critics under Best-of- K Selection

Goal-conditioned action generators can produce many plausible actions. Best-of- K selection then uses a critic to rank those actions for the current state and goal. Standard contrastive critics are trained and evaluated on a different ranking: distinguishing future goals from unrelated negatives for a fixed state--action pair. The unnormalized contrastive score is monotone in behavior-policy value at the population optimum. Cosine-normalized scores lack this guarantee, and finite trained networks may fail to recover the correct ordering. We audit the resulting gap using shared state--goal queries and frozen candidate pools, so critics differ only in their action ordering. Raw and cosine critics reach retrieval AUCs of 0.96 -- 1.00 across eight OGBench tasks. Their progress ordering on navigation tasks is much weaker and sometimes inverted. In the exact-value controlled problem, all three contrastive selectors have higher mean regret than random selection. Their regret can increase with K. TD-Q stays close to the pool oracle. PointMaze is the only task with a reliable learned value reference. TD-Q gives substantially better fixed-query ordering there, though the differences in executed return are smaller and vary by task. Matched controls suggest that graded value supervision accounts for most of the improvement. Changing score bounds or model capacity alone has much less effect. High retrieval performance does not establish whether a critic ranks actions correctly for best-of- K selection. That requires a separate action-ordering evaluation before deployment.