Nathanaël Fijalkow ; Stefan Kiefer ; Mahsa Shirmohammadi - Trace Refinement in Labelled Markov Decision Processes

lmcs:5002 - Logical Methods in Computer Science, June 3, 2020, Volume 16, Issue 2 - https://doi.org/10.23638/LMCS-16(2:10)2020
Trace Refinement in Labelled Markov Decision ProcessesArticle

Authors: Nathanaël Fijalkow ORCID; Stefan Kiefer ; Mahsa Shirmohammadi

    Given two labelled Markov decision processes (MDPs), the trace-refinement problem asks whether for all strategies of the first MDP there exists a strategy of the second MDP such that the induced labelled Markov chains are trace-equivalent. We show that this problem is decidable in polynomial time if the second MDP is a Markov chain. The algorithm is based on new results on a particular notion of bisimulation between distributions over the states. However, we show that the general trace-refinement problem is undecidable, even if the first MDP is a Markov chain. Decidability of those problems was stated as open in 2008. We further study the decidability and complexity of the trace-refinement problem provided that the strategies are restricted to be memoryless.


    Volume: Volume 16, Issue 2
    Published on: June 3, 2020
    Accepted on: May 16, 2020
    Submitted on: November 30, 2018
    Keywords: Computer Science - Logic in Computer Science
    Funding:
      Source : OpenAIRE Graph
    • Challenges for Logic, Transducers and Automata; Funder: French National Research Agency (ANR); Code: ANR-16-CE40-0007
    • The Alan Turing Institute; Funder: UK Research and Innovation; Code: EP/N510129/1

    3 Documents citing this article

    Consultation statistics

    This page has been seen 1296 times.
    This article's PDF has been downloaded 220 times.