General-Sum Markov Games with Borel Spaces: Existence of Approximate Nash Equilibria via Finite Model Approximations under Setwise or Weak Continuity
The paper proves that approximate Markov and stationary Nash equilibria exist for nonzero-sum stochastic games defined over Borel spaces. The method is constructive: it approximates the complex Borel game with simpler finite state-action models, showing their equilibria are close to the original game's true equilibria under mild continuity conditions. This provides a way to learn these solutions.