Richard Mayr ; Eric Munday - Strategy Complexity of Point Payoff, Mean Payoff and Total Payoff Objectives in Countable MDPs

lmcs:9216 - Logical Methods in Computer Science, March 6, 2023, Volume 19, Issue 1 - https://doi.org/10.46298/lmcs-19(1:16)2023
Strategy Complexity of Point Payoff, Mean Payoff and Total Payoff Objectives in Countable MDPsArticle

Authors: Richard Mayr ; Eric Munday

    We study countably infinite Markov decision processes (MDPs) with real-valued transition rewards. Every infinite run induces the following sequences of payoffs: 1. Point payoff (the sequence of directly seen transition rewards), 2. Mean payoff (the sequence of the sums of all rewards so far, divided by the number of steps), and 3. Total payoff (the sequence of the sums of all rewards so far). For each payoff type, the objective is to maximize the probability that the $\liminf$ is non-negative. We establish the complete picture of the strategy complexity of these objectives, i.e., how much memory is necessary and sufficient for $\varepsilon$-optimal (resp. optimal) strategies. Some cases can be won with memoryless deterministic strategies, while others require a step counter, a reward counter, or both.


    Volume: Volume 19, Issue 1
    Published on: March 6, 2023
    Accepted on: January 24, 2023
    Submitted on: March 15, 2022
    Keywords: Computer Science - Computational Complexity,Computer Science - Artificial Intelligence,Computer Science - Computer Science and Game Theory,Mathematics - Probability

    Classifications

    Mathematics Subject Classification 20201

    Consultation statistics

    This page has been seen 1844 times.
    This article's PDF has been downloaded 314 times.