Does Centralized Learning Help Embodied Agents Coordinate, and When Can a Tabular Study Tell?
The study compared Independent Q-Learning (IQL) and Centralized Q-Learning (CQL) in an 8x8 predator-prey gridworld with speed and stamina limits. It found that coordination benefits depend heavily on agent roles and constraints, not just centralized learning. Mismatched pairings caused coordination breakdowns due to temporal synchronization locks when agents had different capabilities or resource limitations.