Decentralized Bandit Learning of Markov Perfect Equilibria
This work describes a decentralized learning procedure for solving discounted infinite-horizon Markov games with continuous actions. Agents learn strategies by iteratively updating their beliefs using stochastic gradient ascent based on observed rewards and continuation values. Under specific regularity conditions, this process converges almost surely to the unique stationary Markov perfect equilibrium.