Continuity Properties in Competitive Markov Decision Processes

Eilon Solan

We provide a bound for the variation of the function that assigns to every competitive Markov decision process and every discount factor its discounted value.

This bound implies that the undiscounted value of a competitive Markov decision process is continuous in the relative interior of the space of transition rules.