Model Checking Probabilistic Timed Automata with One or Two Clocks

Probabilistic timed automata are an extension of timed automata with discrete probability distributions. We consider model-checking algorithms for the subclasses of probabilistic timed automata which have one or two clocks. Firstly, we show that PCTL probabilistic model-checking problems (such as determining whether a set of target states can be reached with probability at least 0.99 regardless of how nondeterminism is resolved) are PTIME-complete for one-clock probabilistic timed automata, and are EXPTIME-complete for probabilistic timed automata with two clocks. Secondly, we show that, for one-clock probabilistic timed automata, the model-checking problem for the probabilistic timed temporal logic PCTL is EXPTIME-complete. However, the model-checking problem for the subclass of PCTL which does not permit both punctual timing bounds, which require the occurrence of an event at an exact time point, and comparisons with probability bounds other than 0 or 1, is PTIME-complete for one-clock probabilistic timed automata.


Introduction
Model checking is an automatic method for guaranteeing that a mathematical model of a system satisfies a formally-described property [CGP99].Many real-life systems, such as multimedia equipment, communication protocols, networks and fault-tolerant systems, exhibit probabilistic behaviour.This leads to the study of model checking of probabilistic models based on Markov chains or Markov decision processes [Var85, HJ94, CY95, BdA95, Model checking of (non-probabilistic) continuous-time systems against properties of timed temporal logics, which can refer to the time elapsed along system behaviours, has been studied extensively in, for example, the context of timed automata [ACD93,AD94], which are automata extended with clocks that progress synchronously with time.Finally, certain systems exhibit both probabilistic and timed behaviour, leading to the development of model-checking algorithms for such systems [ACD91, HJ94, dA97a, KNSS02, BHHK03, LS05, AB06, BCH + 07, DHS07].
In this paper, we aim to study model-checking algorithms for probabilistic timed automata [Jen96,KNSS02], which can be regarded as a variant of timed automata extended with discrete probability distributions, or (equivalently) Markov decision processes extended with clocks.Probabilistic timed automata have been used to model systems such as the IEEE 1394 root contention protocol, the backoff procedure in the IEEE 802.11Wireless LANs, and the IPv4 link local address resolution protocol [KNPS06].The temporal logic that we use to describe properties of probabilistic timed automata is Ptctl (Probabilistic Timed Computation Tree Logic) [KNSS02].The logic Ptctl includes operators that can refer to bounds on exact time and on the probability of the occurrence of events.For example, the property "a request is followed by a response within 5 time units with probability 0.99 or greater" can be expressed by the Ptctl property request ⇒ P ≥0.99 (F ≤5 response).The logic Ptctl extends the probabilistic temporal logic Pctl [HJ94,BdA95], and the real-time temporal logic Tctl [ACD93].
In the non-probabilistic setting, timed automata with one clock have recently been studied extensively [LMS04,LW05,ADOW05].In this paper we consider the subclasses of probabilistic timed automata with one or two clocks.While probabilistic timed automata with a restricted number of clocks are less expressive than their counterparts with an arbitrary number of clocks, they can be used to model systems with simple timing constraints, such as probabilistic systems in which the time of a transition depends only on the time elapsed since the last transition.Conversely, one-clock probabilistic timed automata are more natural and expressive than Markov decision processes in which durations are associated with transitions (for example, in [dA97b,LS05]).We note that the IEEE 802.11Wireless LAN case study has two clocks [KNPS06], and that an abstract model of the IEEE 1394 root contention protocol can be obtained with one clock [Sto02].
After introducing probabilistic timed automata and Ptctl in Section 2 and Section 3, respectively, in Section 4 we show that model-checking properties of Pctl, such as the property P ≥0.99 (Ftarget ) ("a set of target states is reached with probability at least 0.99 regardless of how nondeterminism is resolved"), is PTIME-complete for one clock probabilistic timed automata, which is the same complexity as for probabilistic reachability properties on (untimed) Markov decision processes [PT87].We also show that, in general, model checking of Ptctl on one clock probabilistic timed automata is EXPTIMEcomplete.However, inspired by the efficient algorithms obtained for non-probabilistic one clock timed automata [LMS04], we also show that, restricting the syntax of Ptctl to the sub-logic in which (1) punctual timing bounds and (2) comparisons with probability bounds other than 0 or 1, are disallowed, results in a PTIME-complete model-checking problem.In Section 5, we show that reachability properties with probability bounds of 0 or 1 are EXPTIME-complete for probabilistic timed automata with two or more clocks, implying EXPTIME-completeness of all the model-checking problems that we consider for this class of models.Our complexity results are summarized in Table 1, where 0/1 denotes the sublogics of Ptctl with probability bounds of 0 and 1 only, and [≤, ≥] denotes the sub-logics of Ptctl in which punctual timing bounds are disallowed.The EXPTIME-hardness results are based on the concept of countdown games, which are two-player games operating in discrete time in which one player wins if it is able to make a state transition after exactly c time units have elapsed, regardless of the strategy of the other player.We show that the problem of deciding the winning player in countdown games is EXPTIME-complete.We believe that countdown games are of independent interest, and note that they have been used to show EXPTIME-hardness of model checking punctual timing properties of timed concurrent game structures [LMO06].Finally, in Section 6, we consider the application of the forward reachability algorithm of Kwiatkowska et al. [KNSS02] to one-clock probabilistic timed automata, and show that the algorithm computes the exact probability of reaching a certain state set.This result is in contrast to the case of probabilistic timed automata with an arbitrary number of clocks, for which the application of the forward reachability algorithm results in an upper bound on the maximal probability of reaching a state set, rather than in the exact maximal probability.Note that, throughout the paper, we restrict our attention to probabilistic timed automata in which positive durations elapse in all loops of the system.

Probabilistic Timed Automata
2.1.Preliminaries.We use R ≥0 to denote the set of non-negative real numbers, Q to denote the set of rational numbers, N to denote the set of natural numbers, and AP to denote a set of atomic propositions.A (discrete) probability distribution over a countable set Then for an uncountable set Q we define Dist(Q) to be the set of functions µ : Q → [0, 1], such that support(µ) is a countable set and µ restricted to support(µ) is a (discrete) probability distribution.In this paper, we make the additional assumption that distributions assign rational probabilities only; that is, for each µ ∈ Dist(Q) and q ∈ Q, we have µ(q) ∈ [0, 1] ∩ Q.
We now introduce timed Markov decision processes, which are Markov decision processes in which rewards associated with transitions are interpreted as time durations.
Definition 2.1.A timed Markov decision process (TMDP) T = (S, s, → , lab) comprises the following components: • A (possibly uncountable) set of states S with an initial state s ∈ S.
The transitions from state to state of a TMDP are performed in two steps: given that the current state is s, the first step concerns a nondeterministic selection of (s, d, ν) ∈ → , where d corresponds to the duration of the transition; the second step comprises a probabilistic choice, made according to the distribution ν, as to which state to make the transition to (that is, we make a transition to a state s ′ ∈ S with probability ν(s ′ )).We often denote such a completed transition by s An infinite path of the TMDP T is an infinite sequence of transitions ω = s 0 that the target state of one transition is the source state of the next.
Similarly, a finite path of T is a finite sequence of consecutive transitions ω = s 0 The length of ω, denoted by |ω|, is n (the number of transitions along ω).We use Path ful to denote the set of infinite paths of T, and Path fin the set of finite paths of T. If ω is a finite path, we denote by last(ω) the last state of ω.For any path ω and i ≤ |ω|, let ω(i) = s i be the (i + 1)th state along ω.Let Path ful (s) and Path fin (s) refer to the sets of infinite and finite paths, respectively, commencing in state s ∈ S.
In contrast to a path, which corresponds to a resolution of nondeterministic and probabilistic choice, an adversary represents a resolution of nondeterminism only.Formally, an adversary of a TMDP T is a function A mapping every finite path ω ∈ Path fin to a transition (last (ω), d, ν) ∈ → .Let Adv T be the set of adversaries of T (when the context is clear, we write simply Adv ).For any adversary A ∈ Adv , let Path A ful and Path A fin denote the sets of infinite and finite paths, respectively, resulting from the choices of distributions of A, and, for a state s ∈ S, let Path A ful (s) = Path A ful ∩ Path ful (s) and Path A fin (s) = Path A fin ∩ Path fin (s).Note that, by defining adversaries as functions from finite paths, we permit adversaries to be dependent on the history of the system.Hence, the choice made by an adversary at a certain point in system execution can depend on the sequence of states visited, the nondeterministic choices taken, and the time elapsed from each state, up to that point.
Given an adversary A ∈ Adv and a state s ∈ S, we define the probability measure Prob A s over Path A ful (s) in the following way.We first define the function A : Next, for any finite path ω fin ∈ Path A fin (s) such that |ω fin | = n, we define the probability P A s (ω fin ) as follows: Then we define the cylinder of a finite path ω fin as: and let Σ A s be the smallest sigma-algebra on Path A ful (s) which contains the cylinders cyl A (ω fin ) for ω fin ∈ Path A fin (s).Finally, we define Prob A s on Σ A s as the unique measure such that Prob A s (cyl (ω fin )) = P A s (ω fin ) for all ω fin ∈ Path A fin (s).An untimed Markov decision process (MDP) (S, s, → , lab) is defined as a finite-state TMDP, but for which → ⊆ S × Dist(S) (that is, the transition relation → does not contain timing information).Paths, adversaries and probability measures can be defined for untimed MDPs in the standard way (see, for example, [BK98]).
In the remainder of the paper, we distinguish between the following classes of TMDP.
• Continuous TMDPs are infinite-state TMDPs in which any transition s 2.2.Syntax of probabilistic timed automata.Let X be a finite set of real-valued variables called clocks, the values of which increase at the same rate as real-time.The set CC (X ) of clock constraints over X is defined as the set of conjunctions over atomic formulae of the form x ∼ c, where x, y ∈ X , ∼ ∈ {<, ≤, >, ≥}, and c ∈ N. Definition 2.2.A probabilistic timed automaton (PTA) P = (L, l, X , inv , prob, L) is a tuple consisting of the following components: • A finite set L of locations with the initial location l ∈ L.
• A finite set X of clocks.
• A function inv : L → CC (X ) associating an invariant condition with each location.
A probabilistic edge (l, g, p) ∈ prob is a triple containing (1) a source location l, (2) a clock constraint g, called a guard, and (3) a probability distribution p which assigns probabilities to pairs of the form (X, l ′ ) for some clock set X ⊆ X and target location l ′ .The behaviour of a probabilistic timed automaton takes a similar form to that of a timed automaton [AD94]: in any location time can advance as long as the invariant holds, and a probabilistic edge can be taken if its guard is satisfied by the current values of the clocks.However, probabilistic timed automata generalize timed automata in the sense that, once a probabilistic edge is nondeterministically selected, then the choice of which clocks to reset and which target location to make the transition to is probabilistic.We require that the values of the clocks after taking a probabilistic edge satisfy the invariant conditions of the target locations.

init, x<3
wait, x<8 error, x≤100 1 < x < 3 5 < x < 6 7 < x < 8 x = 100 x := 0 x := 0 (0.8) (0.2) x := 0 (0.9) (0.1) Figure 1: A probabilistic timed automaton P Example 2.3.A PTA P is illustrated in Figure 1.The PTA represents a simple communication protocol, in which the sender can wait for between 5 and 6 time units before sending the message, at which point the message is delivered successfully with probability 0.8, or can wait for between 7 and 8 time units before sending the message, which corresponds to the message being sent successfully with probability 0.9.From location wait, there are two probabilistic edges: the upper one has the guard 5 < x < 6, and assigns probability 0.8 to ({x}, init ) and 0.2 to (∅, error ), whereas the lower one has the guard 7 < x < 8, and assigns probability 0.9 to ({x}, init ) and 0.1 to (∅, error ).
The size |P| of the PTA P is |L| + |X | + |inv | + |prob|, where |inv | represents the size of the binary encoding of the constants used in the invariant condition, and |prob| includes the size of the binary encoding of the constants used in guards and the probabilities used in probabilistic edges.As in the case of TMDPs, probabilities are expressed as a ratio between two natural numbers, each written in binary.
In the sequel, we assume that at least 1 time unit elapses in all structural loops within a PTA.Formally, a PTA is structurally non-Zeno [TYB05] if, for every sequence X 0 , (l 0 , g 0 , p 0 ), X 1 , (l 1 , g 1 , p 1 ), • • • , X n , (l n , g n , p n ), such that p i (X i+1 , l i+1 ) > 0 for 0 ≤ i < n, and p n (X 0 , l 0 ) > 0, there exists a clock x ∈ X and 0 ≤ i, j ≤ n such that x ∈ X i and g j ⇒ x ≥ 1 (that is, g j contains a conjunct of the form x ≥ c for some c ≥ 1).
We also assume that there are no deadlock states in a PTA.This can be guaranteed by assuming that, in any state of a PTA, it is always possible to take a probabilistic edge, possibly after letting time elapse, a sufficient syntactic condition for which has been presented in [Spr01].First, for a set X ⊆ X of clocks, and clock constraint ψ ∈ CC (X ), let [X := 0]ψ be the clock constraint obtained from ψ by letting, for each x ∈ X, each conjunct of the form x > c or x ≥ c ′ where c ′ ≥ 1 be equal to false.For a clock constraint ψ ∈ CC (X ), let upper(ψ) be the clock constraint obtained from ψ by substituting constraints of the form x < c with x > c − 1 ∧ x < c, and constraints of the form x ≤ c with x ≥ c ∧ x ≤ c.Then, for an invariant condition inv (l) of a PTA location, the clock constraint upper(inv (l)) represents the set of clock valuations for which a guard of a probabilistic edge must be enabled, otherwise the clock valuations correspond to deadlock states from which it is not possible to let time pass and then take a probabilistic edge.Then a PTA has non-deadlocking invariants if, for each location l ∈ L, we have upper(inv (l)) ⇒ (l,g,p)∈prob (g∧ (X,l ′ )∈support(p) [X := 0]inv (l ′ )).The condition of non-deadlocking invariants usually holds for PTA models in practice [KNPS06].
We use 1C-PTA (respectively, 2C-PTA) to denote the set of structurally non-Zeno PTA with non-deadlocking invariants, and with only one (respectively, two) clock(s).
2.3.Semantics of probabilistic timed automata.We refer to a mapping v : X → R ≥0 as a clock valuation.Let R X ≥0 denote the set of clock valuations.Let 0 ∈ R X ≥0 be the clock valuation which assigns 0 to all clocks in X .For a clock valuation v ∈ R X ≥0 and a value d ∈ R ≥0 , we use v + d to denote the clock valuation obtained by letting (v + d)(x) = v(x) + d for all clocks x ∈ X .For a clock set X ⊆ X , we let v[X := 0] be the clock valuation obtained from v by resetting all clocks within X to 0; formally, we let v[X := 0](x) = 0 for all x ∈ X, and let v[X := 0](x) = v(x) for all x ∈ X \ X.The clock valuation v satisfies the clock constraint ψ ∈ CC (X ), written v |= ψ, if and only if ψ resolves to true after substituting each clock x ∈ X with the corresponding clock value v(x).
We now present formally the semantics of PTA in terms of continuous TMDPs.The semantics has a similar form to that of non-probabilistic timed automata [AD94], but with the addition of rules for the definition of a timed, probabilistic transition relation from the probabilistic edges of the PTA.

Probabilistic timed temporal logic
We now proceed to describe a probabilistic, timed temporal logic which can be used to specify properties of probabilistic timed automata [KNSS02].
We identify the following sub-logics of Ptctl.Qualitative reachability properties are those reachability properties for which ζ ∈ {0, 1}.
The size |Φ| of a Ptctl formula Φ is defined in the standard way as the number of symbols in Φ, with each occurrence of the same subformula of Φ as a single symbol.
We now define the satisfaction relation of Ptctl for discrete TMDPs.Given the infinite path ω = s 0 Definition 3.2.Given a discrete TMDP T = (S, s, → , lab) and a Ptctl formula Φ, we define the satisfaction relation |= T of Ptctl as follows: and ω(j) |= T φ 1 , ∀j < i .
We proceed to define the satisfaction relation of Ptctl for continuous TMDPs.Given When clear from the context, we omit the T subscript from |= T .We say that the TMDP T = (S, s, → , lab) satisfies the Ptctl formula Φ, denoted by T |= Φ, if and only if s |= Φ.Furthermore, the PTA P satisfies Φ, denoted by P |= Φ, if and only if Complexity of Ptctl model checking for PTA.Given an arbitrary structurally non-Zeno PTA P, model checking Ptctl formulae is in EXPTIME [KNSS02] (the algorithm consists of executing a standard polynomial-time model-checking algorithm for finite-state probabilistic systems [BdA95,BK98] on the exponential-size region graph of P).The problem of model checking qualitative reachability formulae of the form ¬P <1 (Fa) is EXPTIMEhard for PTA with an arbitrary number of clocks [LS07].Hence Ptctl model checking for structurally non-Zeno PTA with an arbitrary number of clocks is EXPTIME-complete.
Example 3.4.Consider the PTA P of Figure 1.The formula P >0 (F ≤9 error ) holds for the configuration (init, 0): for every non-deterministic choice, the probability to reach error within 9 time units is strictly positive.The formula P <0.1 (F ≤6 error ) does not hold for (init, 0): if the adversary chooses to delay until x = 5.4 in wait , and then performs the probabilistic edge with the guard 5 < x < 6, then the probability to reach error is 0.2.Note also that the formula P ≥0.1 (F ≤6 error ) is not true either in (init, 0): the adversary can choose to delay in wait until x = 7.8 and then perform the second probabilistic edge, in which case the probability to reach error within 6 time units is zero.

Model Checking One-Clock Probabilistic Timed Automata
In this section we consider the case of 1C-PTA.We will see that model checking Pctl and Ptctl 0/1 [≤, ≥] for 1C-PTA is P-complete, but remains EXPTIME-complete for the logic Ptctl 0/1 .4.1.Model Checking Pctl on 1C-PTA.First we present the following result about the model checking of Pctl formulae.
Proof.The problem is P-hard because model checking formulae of the form ¬P <1 (Fa) in finite MDPs is P-hard [PT87].Here we show P-membership.For this we adapt the encoding for showing NLOGSPACE-membership of reachability in one-clock timed automata [LMS04] in order to obtain an untimed MDP which is polynomial in the size of the 1C-PTA.This untimed MDP is then subject to the established polynomial-time Pctl model-checking algorithm [BdA95].
Let P = (L, l, {x}, inv , prob, L) be a 1C-PTA.A state of P is a control location and a value v for x.The exact value of x is not important to solve the problem: we just need to know in which interval (with respect to the constants occurring in the guards and invariants of P) is x.Let Cst(P) be the set of integer values used in the guards and invariants of P, and let B = Cst(P)∪{0}.We use b 0 , b 1 , . . ., b k to range over B, where 0 We also define a total order on the set I B , where The configuration (l, v) is then encoded by the pair (l, n(v)) such that v belongs to the n(v)-th interval in I B : note that the length of the binary representation of the number of an interval is log(2(k + 1)).We then build an untimed MDP M[P] whose states are the pairs (l, n(v)) and the transitions simulate those of P. Note that we can easily decide whether a guard is satisfied by the clock values of the n(v)-th interval.A step of P from (l, v) consists in choosing a duration d and a distribution µ (as represented by the transition ((l, v), d, µ)), and finally making a probabilistic choice.Such a step is simulated in M[P] by a transition ((l, n(v)), ν), which corresponds to choosing the appropriate interval n(v + d) in the future (i.e., n(v + d) ≥ n(v)), then making a probabilistic choice according to the distribution ν from (l, n(v • → M is the least set such that ((l, B), ν) ∈ → M if there exists an interval B ′ ∈ I B and a probabilistic edge (l, g, p) ∈ prob such that: (1) and 0] and ν 0 (l ′ , B ′′ ) = 0 otherwise, and where Given a Pctl formula Φ and a state (l, v) of T[P], we then have that (l, v) |= T[P] Φ if and only if (l, n(v)) |= M[P] Φ, which can be shown by induction on the length of the formula.The cases of atomic propositions and boolean combinators are straightforward, and therefore we concentrate on the case of a formula P ⊲⊳λ (Φ 1 UΦ 2 ).We can show that, for each adversary A of T[P], it is possible to construct an adversary Conversely, we can show that, for each adversary A of M[P], it is possible to construct an adversary A ′ of T[P] such that, for each state (l, v) of T[P], we have By the definition of the semantics of Pctl, given (l, v), we have (l, v) |= T Because Pctl model checking is polynomial in the size of the MDP [BdA95], we have obtained a polynomialtime algorithm for Pctl model checking for PTA.

Model checking Ptctl
In this section, inspired by related work on discrete-time concurrent game structures [LMO06], we first show that modelchecking Ptctl 0/1 [≤, ≥] properties of discrete TMDPs can be done efficiently.Then, in Theorem 4.3, using ideas from the TMDP case, we show that model checking Ptctl 0/1 [≤, ≥] on 1C-PTA can also be done in polynomial time.Proof sketch.The model-checking algorithm is based on several procedures to deal with each modality of Ptctl 0/1 [≤, ≥].The boolean operators and the Pctl modalities (without timed subscripts) can be handled in the standard manner, with the Pctl properties verified on the untimed MDP T u corresponding to T. For formulae P ⊲⊳ζ (Φ 1 U ∼c Φ 2 ), we assume that the truth values of subformulae Φ 1 and Φ 2 are known for all states of T. First, given that the TMDP is structurally non-Zeno, we have the equivalences: where E (respectively, A) stands for the existential (respectively, universal) quantification over paths which exist in the logic Tctl.Thus we can apply the procedure proposed for model checking Tctl formulae -running in time O(|S| • | → |) -over weighted graphs [LMS05] (in the case of P ≥1 (Φ 1 U ≥c Φ 2 ), by first obtaining the set of states satisfying The problem of verifying the remaining temporal properties of Ptctl 0/1 [≤, ≥] can be considered in terms of turn-based 2-player games.Such a game is played over the space S ∪ → , and play proceeds as follows: from a state s ∈ S, player P n (representing nondeterministic choice) chooses a transition (s, d, ν) ∈ → ; then, from the transition (s, d, ν), player P p (representing probabilistic choice) chooses a state s ′ ∈ support(ν).The duration of the move from s to s ′ via (s, d, ν) is d.Notions of strategy of each player, and winning with respect to (untimed) path formulae of the form Φ 1 UΦ 2 , are defined as usual for 2-player games.
Using the fact that the TMDP is structurally non-Zeno, for any state s ∈ S, we can obtain the following equivalences: The functions α, β, γ, δ can be computed on the 2-player game by applying the same methods as in [LMO06] for discrete-time concurrent game structures: for each temporal operator We decompose the proof into the following four cases, which depend on the form of the formula to be verified.Φ = P >0 (Φ 1 U ≤c Φ 2 ).To compute the value α(s), we introduce the coefficients α i (s) defined recursively as follows.Let α 0 (s) = 0 if s |= Φ 2 , let α 0 (s) = ∞ otherwise, and let: Fact 1.If α i (s) < ∞, the value α i (s) is the minimal duration that player P p can ensure from s with respect to Φ 1 UΦ 2 in at most 2i turns.If α i (s) = ∞, player P p cannot ensure Φ 1 UΦ 2 in 2i turns.
Proof of Fact 1.The proof proceeds by induction over i.The result is immediate for i = 0. Now assume the property holds up to i. Consider α i+1 (s).The cases for α i+1 (s) = 0, and α i+1 (s) = ∞ with s |= ¬Φ 1 ∧ ¬Φ 2 , are trivial.Now assume α i+1 (s) = ∞ and s |= Φ 1 ∧ ¬Φ 2 : by the definition of α i+1 (s), there exists a transition (s, , ν) from s such that any possible successor s ′ ∈ support(ν) verifies α i (s ′ ) = ∞.By the induction hypothesis this entails that there is no strategy for P p to ensure Φ 1 UΦ 2 in less than 2i turns from any s ′ ∈ support(ν), and then there is no strategy for P p from s for games with 2(i + 1) turns.
Assume α i+1 (s) ∈ N. Let θ be the minimal duration that player P p can ensure with respect to Φ 1 UΦ 2 , for games with at most 2(i + 1) turns.This duration θ is obtained from a choice of transition (s, d, ν) of P n and a choice of state s ′ ∈ support(ν) of P p , where, by the induction hypothesis, we have θ = d + α i (s ′ ).We also have that this s ′ is the best (minimal) choice for P p among all states in support(ν); that is, α i (s ′ ) = min s ′′ ∈support(ν) {α i (s ′′ )}.Given the definition of α i+1 (s), we have that α i+1 (s) equals: However, as θ corresponds to the best (maximal) choice for P n , we cannot have α i+1 (s) > θ, and therefore α i+1 (s) = θ.
We claim that α |S| (s) = α(s).First note that we clearly have α |S| (s) ≥ α(s).Now assume α(s) < α |S| (s): this value α(s) is obtained by a strategy (for P p ) that uses more than 2|S| turns.Therefore, along some path generated by this strategy there will be at least one occurrence of a state s ′ .However, as the TMDP is structurally non-Zeno, this loop has a duration strictly greater than 0, and it can be removed by applying earlier in the path the last choice done for state s ′ along the path1 .Such a looping strategy is clearly not optimal for P p and need not be considered when computing α(s).Hence the computation of α |S| , and thus α, can be done in time In order to establish the set of states satisfying Φ, we first compute the sets of states satisfying two untimed, auxiliary formulae.The first formula we consider is P >0 (Φ 1 UΦ 2 ): obtaining the set of states satisfying this formula relies on qualitative Pctl analysis of the underlying untimed MDP T u of T, which can be done in time O(|Edges( → )|).The second formula we consider is P >0 (Φ 1 U ≥1 Φ 2 ), where, for any infinite path ω ∈ Path ful , we have ω |= Φ 1 U ≥1 Φ 2 if and only if there exists i ≥ 1 such that ω(i) |= Φ 2 , and ω(j) |= Φ 1 for all j < i.The set of states satisfying P >0 (Φ 1 U ≥1 Φ 2 ) can be obtained through a combination of the usual "next" temporal operator of Pctl (see [HJ94,BdA95]) and the formula P >0 (Φ 1 UΦ 2 ), and can be computed in time O(|Edges( → )|).
We then proceed to compute, for each state s of T satisfying P >0 (Φ 1 UΦ 2 ), the maximal duration γ(s) that player P p can ensure with respect to Φ 1 U(P >0 (Φ 1 UΦ 2 )).We compute γ using the following recursive rules: We have the following fact, the proof of which is similar to that of Fact 1.
Assume γ i+1 (s) ∈ N. Let θ be the maximal duration that player P p can ensure with respect to Φ 1 UΦ 2 , for games with at most 2(i + 1) turns.This duration θ is obtained from a choice of (s, d, ν) of P n and a choice of s ′ ∈ support(ν) of P p , where, by the induction hypothesis, we have θ = d + γ i (s ′ ).We also have that this s ′ is the best (maximal) choice for P p among all states in support(ν); that is, γ i (s ′ ) = max s ′′ ∈support(ν) {γ i (s ′′ )}.We have that γ i+1 (s) equals: However, as θ corresponds to the best (minimal) choice for P n , we cannot have γ i+1 (s) < θ, and therefore γ i+1 (s) = θ.
As in the case of the function α, we claim that γ |S| (s) = γ(s).We clearly have γ |S| (s) ≥ γ(s) (indeed we can prove by induction over i that γ i (s) ≥ γ(s) for any i ≥ 0).Assume that γ(s) < γ |S| (s); then as in the case of α, the value γ(s) is obtained by a strategy for P p which generates a path whose length is greater than |S| along which a state is visited twice.The assumption of structural non-Zenoness means that, if the strategy can choose to repeat s ′ an arbitrary number of times, the elapsed duration along the path becomes arbitrarily large and γ(s) = γ |S| (s) = ∞.Hence, there is no need to explore further the path.Therefore the computation of γ |S| , and thus γ, can be done in time O(|S| • | → |).Φ = P <1 (Φ 1 U ≤c Φ 2 ).This case can be treated in a similar manner as the case of Φ = P >0 (Φ 1 U ≤c Φ 2 ).Here we aim at computing the minimum duration β(s) that player P n can ensure with respect to Φ 1 UΦ 2 .Then Φ holds for s if and only if β(s) > c.We compute the following values β i (s) with β 0 (s) = 0 if s |= Φ 2 , β 0 (s) = ∞ otherwise, and: Fact 3. If β i (s) < ∞, the value β i (s) is the minimal duration that player P n can ensure from s with respect to Φ 1 UΦ 2 in at most 2i turns.If The proof of Fact 3 proceeds in a similar manner to that of Fact 1, but with the roles of players P n and P p reversed, and therefore we omit it.Furthermore, we have β |S| (s) = β(s) for similar reasons that we had α |S| = α(s) (again, with the roles of P n and P p reversed), and hence the computation of β can be done in time ).This property is true when player P n has no strategy to ensure Φ 1 U ≥c Φ 2 .Similarly to the case of P >0 (Φ 1 U ≥c Φ 2 ), we first compute the sets of states satisfying two untimed formulae, namely P <1 (Φ 1 UΦ 2 ) and P <1 (Φ 1 U ≥1 Φ 2 ), the complexity of which is in O(|Edges( → )| |Edges( → )|) [CJH03].We then compute, for each state s of T satisfying ¬P <1 (Φ 1 UΦ 2 ), the maximal duration δ(s) that player P n can ensure with respect to Φ 1 U(P <1 (¬Φ 1 UΦ 2 )).Then s |= Φ if and only if δ(s) < c.We compute δ using the following recursive rules: is the maximal duration that player P n can ensure from s with respect to Φ 1 U(P >0 (Φ 1 UΦ 2 )) in at most 2i turns.If We can adapt the reasoning used in Fact 2 to prove this fact (as in the case of Fact 3).Finally, with similar reasoning to that used in the case of P >0 (Φ 1 U ≥c Φ 2 ), we can show that δ |S| (s) = δ(s), and therefore δ can be computed in time Finally we obtain an algorithm running in time We use Proposition 4.2 to obtain an efficient model-checking algorithm for 1C-PTA.
Proof sketch.Our aim is to label every state (l, v) of T[P] with the set of subformulae of Φ which it satisfies (as |X | = 1, recall that v is a single real value).For each location l ∈ L and subformula Ψ of Φ, we construct a set Sat[l, Ψ] ⊆ R ≥0 of intervals such that v ∈ Sat[l, Ψ] if and only if (l, v) |= Ψ.We write Sat[l, Ψ] = j=1,...,k c j ; c ′ j with ∈ {[, (} and ∈ {], )}.We consider intervals which conform to the following rules: for 1 ≤ j ≤ k, we have c j < c ′ j and c j , c ′ j ∈ N ∪ {∞}, and for 1 ≤ j < k, we have c ′ j < c j+1 .We will see that |Sat[l, Ψ]| -i.e., the number of intervals corresponding to a particular location -is bounded by |Ψ| • 2 • |prob|.
The cases of obtaining the sets Sat[l, Ψ] for boolean operators and atomic propositions are straightforward, and therefore we concentrate on the verification of subformulae Ψ of the form P ⊲⊳ζ (Φ 1 U ∼c Φ 2 ).Assume that we have already computed the sets Sat[ , ] for Φ 1 and Φ 2 .Our aim is to compute Sat[l, Ψ] for each location l ∈ L.
There are several cases depending on the constraint "⊲⊳ ζ".The equivalence , which holds from the structural non-Zenoness property, can be used to reduce the "≤ 0" case to the appropriate polynomial-time labeling procedure for ¬ (EΦ 1 U ∼c Φ 2 ) on one-clock timed automata [LMS04], where the 1C-TA is obtained by converting the probabilistic choice of prob to nondeterministic choice.In the "≥ 1" case, the equivalence )) relies on first computing state set satisfying P ≥1 (Φ 1 UΦ 2 ), which can be handled using a qualitative Pctl modelchecking algorithm, applied to a discrete TMDP built from P, Sat[l, , and second verifying the formula A (Φ 1 U ∼c (P ≥1 (Φ 1 UΦ 2 ))) using the aforementioned method for one-clock timed automata.
For the remaining cases, our aim is to construct a (finite) discrete TMDP T r = (S r , , → r , lab r ), which represents partially the semantic TMDP T[P], for which the values of the functions α, β, γ and δ of the proof of Proposition 4.2 can be computed, and then use these functions to obtain the required sets Sat[ , Ψ] (the initial state of T r is irrelevant for the model-checking procedure, and is therefore omitted).The TMDP T r will take a similar form to the region graph MDP of PTA [KNSS02], but, as in the case of the MDP M[P] constructed in the proof of Proposition 4.1, will be of reduced size.More precisely, the size of T r will be independent of the magnitude of the constants used in invariants and guards, and will ensure a procedure running in time polynomial in |P|.
We now describe the construction of T r .In the following we assume that the sets Sat[l, Φ i ] contain only closed intervals (and possibly intervals of the form [b; ∞)) and that the guards and invariant of the PTA contain non-strict comparisons: the general case is explained in Appendix A.
Transitions of T r : We now define the set → r of transitions of T r as the smallest set such that ((l, λ), d, ν) ∈→ r , where for some b j ∈ C, and (l, g, p) ∈ prob such that: Now we can apply the algorithms defined in the proof of Proposition 4.2 and obtain the value of the coefficients α, β, γ or δ for the states of T r .Our next task is to define functions α, β, γ, δ : S → R ≥0 , where S is the set of states of T[P], which are analogues of α, β, γ or δ defined on T[P].Our intuition is that we are now considering an infinite-state 2-player game with players P n and P p , as in the proof of Proposition 4. For intervals of the form (b i ; b i+1 ), the functions α and δ decrease (with slope -1) throughout the interval, because, for all states of the interval, the optimal choice of player P n is to delay as much as possible inside any interval.Hence, the value α(l, v) Next we consider the values of β and γ over intervals (b i ; b i+1 ).In this case, the functions will be constant over a portion of the interval (possibly an empty portion, or possibly the entire interval), then decreasing with slope -1.The constant part corresponds to those states in which the optimal choice of player P n is to take a probabilistic edge, whereas the decreasing part corresponds to those states in which it is optimal for player P n to delay until the end of the interval.The value otherwise.An analogous definition holds also for γ.From the functions α, β, γ and δ defined above, it becomes possible to define Sat[l, Ψ] by keeping in this set of intervals only the parts satisfying the thresholds ≤ c, > c, ≥ c and < c, respectively, as in the proof of Proposition 4.2.We can show that the number of intervals in Sat[l, Ψ] is bounded by 2 • |Ψ| • |prob|.For the case in which a function α, β, γ or δ is decreasing throughout an interval, then an interval in Sat[l, Φ 1 ] which corresponds to several consecutive intervals in T r can provide at most one (sub)interval in Sat[l, Ψ], because the threshold can cross at most once the function in at most one interval.For the case in which a function β or γ combines a constant part and a part with slope -1 within an interval, the threshold can cross the function in several intervals (b i ; b i+1 ) contained in a common interval of Sat[l, Φ 1 ].However, such a cut is due to a guard x ≥ k of a given transition, and thus the number of cuts in bounded by |prob|.Moreover a guard x ≤ k may also add an interval.Thus the number of new intervals in Sat[q, Ψ] is bounded by 2 • |prob|.
In addition to these cuts, any interval in Sat 4.3.Model checking Ptctl 0/1 on 1C-PTA.We now consider the problem of modelchecking Ptctl 0/1 properties on 1C-PTA.An EXPTIME algorithm for this problem exists by the definition of an MDP analogous to the region graph used in non-probabilistic timed automata verification [KNSS02].We now show that the problem is also EXPTIME-hard by the following three steps.First we introduce countdown games, which are a simple class of turn-based 2-player games with discrete timing, and show that the problem of deciding the winner in a countdown game is EXPTIME-complete.Secondly, we reduce the countdown game problem to the Ptctl 0/1 model-checking problem on TMDPs.Finally, we adapt the reduction to TMDPs to reduce also the countdown game problem to the Ptctl 0/1 model-checking problem on 1C-PTA.
A countdown game C consists of a weighted graph (S, T), where S is the set of states and T ⊆ S × N \ {0} × S is the transition relation.If t = (s, d, s ′ ) ∈ T then we say that the duration of the transition t is d.A configuration of a countdown game is a pair (s, c), where s ∈ S is a state and c ∈ N. A move of a countdown game from a configuration (s, c) is performed in the following way: first player 1 chooses a number d, such that 0 < d ≤ c and (s, d, s ′ ) ∈ T, for some state s ′ ∈ S; then player 2 chooses a transition (s, d, s ′ ) ∈ T of duration d.The resulting new configuration is (s ′ , c − d).There are two types of terminal configurations, i.e., configurations (s, c) in which no moves are available.If c = 0 then the configuration (s, c) is terminal and is a winning configuration for player 1.If for all transitions (s, d, s ′ ) ∈ T from the state s, we have that d > c, then the configuration (s, c) is terminal and it is a winning configuration for player 2. The algorithmic problem of deciding the winner in countdown games is, given a weighted graph (S, T) and a configuration (s, c), where all the durations of transitions in (S, T) and the number c are given in binary, to determine whether player 1 has a strategy to reach a winning configuration, regardless of the strategy of player 2, from the configuration (s, c).If the state from which the game is started is clear from the context then we sometimes specify the initial configuration by giving the number c alone.
Theorem 4.5.Deciding the winner in countdown games is EXPTIME-complete.
Proof sketch.Observe that every configuration of a countdown game played from a given initial configuration can be written down in polynomial space and every move can be computed in polynomial time; hence the winner in the game can be determined by a straightforward alternating PSPACE algorithm.Therefore the problem is in EXPTIME because APSPACE = EXPTIME.
We now prove EXPTIME-hardness by a reduction from the problem of the acceptance of a word by a linearly-bounded alternating Turing machine [CKS81].Let M = (Σ, Q, q 0 , q acc , Q ∃ , Q ∀ , ∆) be an alternating Turing machine, where Σ is a finite alphabet, Q = Q ∃ ∪Q ∀ is a finite set of states partitioned into existential states Q ∃ and universal states Q ∀ , q 0 ∈ Q is an initial state, q acc ∈ Q is an accepting state, and ∆ ⊆ Q×Σ×Q×Σ×{L, R} is a transition relation.Let us explain the interpretation of elements of the transition relation.Let t = (q, σ, q ′ , σ ′ , D) ∈ ∆ be a transition.If machine M is in state q ∈ Q and its head reads letter σ ∈ Σ, then it rewrites the contents of the current cell with the letter σ ′ , it moves the head in direction D (either left if D = L, or right if D = R), and it changes its state to q ′ .Let G > 2 • |Q × Σ| be an integer constant and let w ∈ Σ n be an input word.Without loss of generality, we can assume that the alternating Turing machine M uses exactly n tape cells when started on the word w, and hence a configuration of machine M is a word We first define countdown games which have the role of checking the contents of the tape; these countdown games will be used as gadgets later in the overall reduction.Let i ∈ N, 0 ≤ i < n, be a tape cell position, and let a ∈ Σ∪Q×Σ.We define a countdown game Check i,a , such that for every configuration u = b 0 • • • b n−1 of machine M , player 1 has a winning strategy from the configuration (s i,a 0 , N (u)) of the countdown game Check i,a if and only if b i = a.The game Check i,a has states { s i,a 0 , . . ., s i,a n }, and for every k, 0 ≤ k < n, we have a transition (s i,a k , d, s i,a k+1 ) ∈ T, if: There are no transitions from the state s i,a n .Observe that if b i = a then the winning strategy for player 1 in game Check i,a from N (u) is to choose the transitions (s i,a k , b k • G k , s i,a k+1 ), for all k, 0 ≤ k < n.If, however, b i = a then there is no way for player 1 to count down from N (u) to 0 in the game Check i,a .Now we define a countdown game C M , such that machine M accepts a word w = σ 0 σ 1 . . .σ n−1 if and only if player 1 has a winning strategy in C M from configuration (q 0 , N (u)), where u = (q 0 , σ 0 )σ 1 . . .σ n−1 is the initial configuration of tape contents of machine M with input w.The main part of the countdown game C M is a gadget that allows the countdown game to simulate one step of the Turing machine M .Note that one step of a Turing machine makes only local changes to the configuration of the machine: if the configuration is of the form u = a 0 . . .a n−1 = σ 0 . . .σ i−1 (q, σ i )σ i+1 . . .σ n−1 , then performing one step of M can only change entries in positions i − 1, i, or i + 1 of the tape.For every tape position i, 0 ≤ i < n, for every triple τ = (σ i−1 , (q, σ i ), σ i+1 ) ∈ Σ × (Q × Σ) × Σ, and for every transition t = (q, σ, q ′ , σ ′ , D) ∈ ∆ of machine M , we now define the number d i,τ t , such that if σ i = σ and performing transition t at position i of configuration u yields configuration u ′ = b 0 . . .b n−1 , then N (u) − d i,τ t = N (u ′ ).For example, assume that i > 0 and that D = L; from the above comment about locality of Turing machine transitions we have that b k = a k = σ k , for all k ∈ { i − 1, i, i + 1 } and b i+1 = a i+1 = σ i+1 .Moreover we have that b i−1 = (q ′ , σ i−1 ), and b i = σ ′ .We define d i,τ t as follows: The gadget for simulating one transition of Turing machine M from a state q ∈ Q\{q acc } has three layers.In the first layer, from a state q ∈ Q \ { q acc }, player 1 chooses a pair (i, τ ), where i, 0 ≤ i < n, is the position of the tape head, and τ = (a, b, c) ∈ Σ × (Q × Σ) × Σ is his guess for the contents of tape cells i − 1, i, and i + 1.In this way the state (q, i, τ ) of the gadget is reached, where the duration of this transition is 0. Intuitively, in the first layer player 1 has to declare that he knows the position i of the head in the current configuration as well as the contents τ = (a, b, c) of the three tape cells in positions i − 1, i, and i + 1.In the second layer, in a state (q, i, τ ) player 2 chooses between four successor states: the state (q, i, τ, * ) and the three subgames Check i−1,a , Check i,b , and Check i+1,c .The four transitions are of duration 0. Intuitively, in the second layer player 2 verifies that player 1 declared correctly the contents of the three tape cells in positions i − 1, i, and i + 1.Finally, in the third layer, if q ∈ Q ∃ (respectively, q ∈ Q ∀ ), then from a state (q, i, τ, * ) player 1 (respectively, player 2) chooses a transition t = (q, σ, q ′ , σ ′ , D) of machine M , such that b = (q, σ), reaching the state q ′ ∈ Q of the gadget, with a transition of duration d i,τ t .Note that the gadget described above violates some conventions that we have adopted for countdown games.Observe that durations of some transitions in the gadget are 0 and the duration d i,τ t may even be negative, while in the definition of countdown games we required that durations of all transitions are positive.In order to correct this we add the number G n to the durations of all transitions described above.This change requires a minor modification to the subgames Check i,a : we add an extra transition (s i,a n , G n , s i,a n ).We need this extra transition because instead of starting from (q 0 , N (u)) as the initial configuration of the countdown game C M , where u is the initial configuration of M running on w, we start from the configuration (q 0 , G 3n + N (u)).In this way the countdown game can perform a simulation of at least G n steps of machine M ; note that G n is an upper bound on the number of all configurations of machine M .
Without loss of generality, we can assume that whenever the alternating Turing machine M accepts an input word w then it finishes its computation with blanks in all tape cells, its head in position 0, and in the unique accepting state q acc ; we write u acc for this unique accepting configuration of machine M .Moreover, assume that there are no transitions from the accepting state q acc in machine M .In order to complete the definition of the countdown game G M , we add a transition of duration N (u acc ) from the state q acc of game C M .
Proof.An EXPTIME algorithm can be obtained by employing the algorithms of [LS05].We now prove EXPTIME-hardness of Ptctl 0/1 model checking on discrete TMDPs by a reduction from countdown games.Let C = (S, T) be a countdown game and (s, c) be its initial configuration.We construct a TMDP T C,(s,c) = (S, s, → , lab) such that player 1 wins C from (s, c) if and only if T C,(s,c) |= ¬P <1 (F =c true).Let S = S and s = s.We define → to be the smallest set satisfying the following: for each s ∈ S and d ∈ N >0 , if (s, d, s ′ ) ∈ T for some s ′ ∈ T, we have (s, d, ν) ∈ → , where ν is an arbitrary distribution over S such that support(ν) = {s ′ | (s, d, s ′ ) ∈ T}.The labelling condition lab is arbitrary.Then we can show that player 1 wins C from the configuration (s, c) if and only if there exists an adversary of T C,(s,c) such that a state is reached from s = s after exactly c time units with probability 1.The latter is equivalent to s |= ¬P <1 (F =c true).
We now show that the proof of Proposition 4.6 can be adapted to show the EXPTIMEcompleteness of the analogous model-checking problem on 1C-PTA.
Proof.Recall that there exists an EXPTIME algorithm for model-checking Ptctl 0/1 properties on structurally non-Zeno PTA [KNSS02]; hence, it suffices to show EXPTIMEhardness for Ptctl 0/1 and 1C-PTA.Let C be a countdown game with an initial configuration (s, c).We construct the 1C-PTA P 1C C,(s,c) = (L, l, {x}, inv , prob, L) which simulates the behaviour of the TMDP T C,(s,c) of the proof of Proposition 4.6 in the following way.Each state s ∈ S of T C,(s,c) corresponds to two distinct locations l 1 s and l 2 s of P 1C C,(s,c) .
, and let l = l 1 s .For every transition (s, d, ν) ∈ → of T C,(s,c) , we have the probabilistic edges (l 1 s , x = 0, p 1 ), (l 2 s , x = d, p 2 ) ∈ prob, where p 1 ({x}, l 2 s ) = 1, and p 2 ({x}, l 1 s ′ ) = ν(s ′ ) for each location s ′ .For each state s ∈ S, let inv (l 1 s ) = (x ≤ 0) and inv (l 2 s ) = true.Therefore the PTA P 1C C,(s,c) moves from the location l 1 s to l 2 s instantaneously.Locations in L 1 are labelled by the atomic proposition a, whereas locations in L 2 are labelled by ∅.Then we can observe that P 1C C,(s,c) |= ¬P <1 (F =c a) if and only if T C,(s,c) |= ¬P <1 (F =c true).As the latter problem has been shown to be EXPTIMEhard in the proof of Proposition 4.6, we conclude that model checking Ptctl 0/1 on 1C-PTA is also EXPTIME-hard.
In Figure 2, we illustrate the transformation from countdown games to TMDP, then to 1C-PTA, for a fragment of a countdown game.For simplicity, we omit guards of the form x = 0 and invariant conditions of the form true.

Model Checking Two-Clocks Probabilistic Timed Automata
We now show EXPTIME-completeness of the simplest problems that we consider on 2C-PTA.
Proof.EXPTIME algorithms exist for probabilistic reachability problems on structurally non-Zeno PTA [KNSS02], and therefore it suffices to show EXPTIME-hardness.We proceed by reduction from deciding the winner in countdown games.Let C be a countdown game with initial configuration (s, c), and let P 1C C,(s,c) = (L, l, {x}, inv , prob, L) be the 1C-PTA constructed in the proof of Theorem 4.7.We define the 2C-PTA P 2C C,(s,c) = (L ∪ {l ⋆ }, l, {x, y}, inv ′ , prob ′ , L ′ ) from P 1C C,(s,c) in the following way.The set of probabilistic edges prob ′ is obtained by adding to prob the following: for each location l ∈ L 1 , we extend the set of outgoing probabilistic edges of l with (l, y = c, p l ⋆ ), where p l ⋆ (∅, l ⋆ ) = 1; we also add (l ⋆ , true, p l ⋆ ) to prob ′ .For each l ∈ L, let inv ′ (l) = inv (l), and let inv ′ (l ⋆ ) = true.Finally, we let L ′ (l ⋆ ) = a, and L(l) = ∅ for all l ∈ L. Then P 2C C,(s,c) |= ¬P <1 (Fa) if and only x := 0 x := 0 x := 0 x := 0 x := 0 x := 0 In Figure 2 we illustrate the reduction from countdown games to 2C-PTA (via the reduction to TMDPs and 1C-PTA).
Corollary 5.2.The Pctl, Ptctl 0/1 [≤, ≥], Ptctl 0/1 , Ptctl[≤, ≥] and Ptctl modelchecking problems for 2C-PTA are EXPTIME-complete.6. Forward Reachability for One-Clock Probabilistic Timed Automata Model-checking tools for non-probabilistic timed automata such as Uppaal [BDL + 06] are generally based on algorithms for forward reachability through the state space: such algorithms start from the initial state and explore the state space by executing transitions either in a depth-first or breadth-first manner, and representing sets of clock valuations symbolically using zones.Forward reachability algorithms can be used for verifying reachability properties, such as "the location error is reachable from the initial state".
We recall that the zone-based forward reachability approach has been adapted for PTA by Kwiatkowska et al. [KNSS02], and can be used to reason about the maximal probability of reaching a certain set of locations.More precisely, an (untimed) MDP is constructed by exploring the state space of the PTA from its initial state.Then the maximal probability of reaching a set of locations is computed on the MDP.The appeal of this approach is its practical applicability [DKN04].A disadvantage of the approach is that, in general, it can be used only to obtain an upper bound on the maximal probability of reaching a set of locations of a PTA, rather than the actual maximal probability of reaching the locations.In particular, Kwiatkowska et al. [KNSS02] present an example of a 2C-PTA in which the forward reachability approach does not compute the actual maximal probability of reaching a set of locations.
In this section, we consider the application of the forward reachability approach of Kwiatkowska et al. [KNSS02] to 1C-PTA, and show that the maximal and minimal probabilities computed on the untimed MDP corresponds to the actual maximal and minimal probabilities of reaching a set of locations of the 1C-PTA. 2 First we introduce some notation.Consider the 1C-PTA P = (L, l, {x}, inv , prob, L), which we assume to be fixed throughout this section.As in the proof of Proposition 4.1, we use B = Cst(P) ∪ {0} to refer to the set of constants used in the guards and invariants of P (and 0).Let I FR be the set of intervals of the form b; b ′ , where b ∈ B, b ′ ∈ B ∪ {∞}, ∈ {(, [} and ∈ {), ]}.The aim of forward exploration is to compute state sets represented by pairs of the form (l, I), where l ∈ L is a location and I ∈ I FR is an interval of the above form.The pair (l, I) represents all states (l, v) of T[P] such that v ∈ I.
We define the operator post, which maps a location-interval pair, a probabilistic edge, a reset set and a location, to a location-interval pair.Intuitively, post returns the set of states obtained after executing a probabilistic edge (including making the probabilistic choice concerning the target location and clock reset) and then letting time pass. 2 Readers familiar with Kwiatkowska et al. [KNSS02] will note that the presentation below is simplified with regard to that for PTA with an arbitrary number of clocks.In particular, to ease notation, we consider that forward reachability can consider states reached after reaching the target set of locations.
Lemma 6.5 then allows us to construct, for any adversary A of M[P], an adversary A ′ of 1st[P] such that the probability of reaching a given set of locations from the initial state is the same for A and A ′ (this fact also follows by noting that ( ) −1 is a probabilistic simulation [SL95]).The converse result, which states that, for any adversary A of 1st[P], an adversary A ′ of M[P] such that the probability of reaching a given set of locations from the initial state is the same for A and A ′ , follows from the fact that 1st[P] is a restriction of M[P].We then obtain the following corollary.Combining Corollary 6.4 and Corollary 6.6, and using the proof of Proposition 4.1, which states that the results of model checking a Pctl formula (including reachability properties of the form P ∼λ (Fa)) on M[P] correspond to the satisfaction of the formula on T[P], we conclude with the following corollary.

Conclusion
We have shown that probabilistic model-checking problems for 1C-PTA can be performed efficiently if qualitative properties with non-punctual timing bounds are considered.If the temporal logic features punctual timing bounds, the problem becomes EXPTIMEcomplete.We have also shown that the forward reachability algorithm of Kwiatkowska et al. [KNSS02] can be used to compute the exact probability of reaching a state set for 1C-PTA.For future work, we intend to consider the complexity of model checking 1C-PTA against quantitative properties without punctual timing bounds (that is, properties of Ptctl[≤, ≥]).On the other hand, we have shown that model-checking problems for 2C-PTA are EXPTIME-complete, regardless of the probability threshold and timing bounds used.
where p ′ ({x}, l ′′ ).Moreover the only state satisfying Φ 2 is (l ′′ , 0), and all states satisfy Φ 1 .The value α corresponds to the duration between the current state and (l ′′ , 0).This example is sufficient to illustrate the problem of strict and non-strict values.
Let us consider the structure of the function α.For the singular points (l, b i ) the value can be of the form "< k", "= k", "> k", or ∞ when there exists a strategy for P n to avoid Φ 2 forever.Note that the case "> k" can occur for a state (l, b j ) when the property Φ 2 holds for an interval (l, (b i ; b i+1 )): reaching this interval from (l, b j ) can be done by a duration strictly greater than b i − b j .The other cases are illustrated on Figure 3. Now consider the case of symbolic states (l, (b i ; b i+1 )).The structure of α over such an interval is always decreasing: indeed either the best strategy for P n consists in performing a distribution from the current interval, in which case it is always better to delay until the last point (b − i+1 ) of the interval, or the best strategy consists in delaying until a future state or interval.We can see that the value of the rightmost position inside the interval will be always of the form "> k": indeed it depends either on the value in b i+1 (if the strategy goes through this point) or on the value in some (l ′ , b 0 ) if there is transition with a reset of clock x.Assume that this value is "ǫ k" and consider a point (l, v) with v ∈ (b i ; b i+1 ).Then any duration in (0; b i+1 − v) is sufficient to reach Φ 2 in more than k time units in case of an optimal strategy: note that this fact does not depend on ǫ.Given a value "> k" for the rightmost position of (b i ; b i+1 ), we can deduce the function α for any position v in the interval: it is b i+1 − v + k.
Therefore (1) the optimal strategies use only the singular points and the rightmost positions b − i+1 in the intervals, and (2) the function α over an interval can be derived from the value in the rightmost position.Thus we will restrict the computation of coefficients α to these points.
Thus the algorithm consists in computing the function α by using values of the form "< k", "= k" or "> k".This is slightly more technical than the basic case.
Finally similar techniques can be used also for the other functions (β, γ and δ).
s ′ describes the continuous passage of time, and thus a path ω = s 0 d 0 ,ν 0 − −− → s 1 d 1 ,ν 1 − −− → • • • describes implicitly an infinite set of visited states.In the sequel, we use continuous TMDPs to give the semantics of probabilistic timed automata.
Formally we let C = {0} ∪ Cst(P) ∪ i∈{1,2} l∈L Cst(Sat[l, Φ i ]), where, as in the proof of Proposition 4.1, Cst(P) is the set of constants occurring in the clock constraints of P, and where Cst(Sat[l, Φ i ]) is the set of constants occurring as endpoints of the intervals in Sat[l, Φ i ].Moreover for any right-open interval [b; ∞) occurring in some Sat[l, ] we add the constant b + c + 1 to C. We enumerate C as b 0 , b 1 , ..., b M with b 0 = 0 and b i < b i+1 for i < |C|.Note that |C| is bounded by 4 • |Ψ| • |prob|.State space of T r : We consider first the definition of S r , the state space of T r .Considering the discrete TMDP corresponding to T[P] restricted to states (l, b i ), with b i ∈ C, is sufficient to compute the values of functions α, β, γ and δ in any state (l, b i ).However, this does not allow us to deduce the value for any intermediate states in (b i ; b i+1 ): indeed some probabilistic edges enabled from b i may be disabled throughout the interval (b i ; b i+1 ).Therefore, in T r , we have to consider also (l, b + i ) and (l, b − i+1 ) corresponding respectively to the leftmost and rightmost points in (b i ; b i+1 ) (when i < M ).Then S r is defined as the set including the pairs (l, b i ) with b i ∈ C and b i |= inv (l), and (l, b + i ) and (l, b − i+1 ) with b i ∈ C, i < M and (b i ; b i+1 ) ⊆ [[inv (l)]].Note that the truth value of any invariant is constant over such intervals (b i ; b i+1 ).Moreover note that all T[P] states of the form (l, v) with v ∈ (b i ; b i+1 ) satisfy the same boolean combinations of Φ 1 and Φ 2 , and enable the same probabilistic edges.For any (l, g, p) ∈ prob, we write b + i |= g (and b − i+1 |= g) when (b i ; b i+1 ) ⊆ [[g]].Similarly, we write b + i |= inv (l) (and b − i+1 |= inv (l)) when (b i ; b i+1 ) ⊆ [[inv (l)]].For an interval I ⊆ R ≥0 , we write b + i ∈ I and b − i+1 ∈ I when (b i ; b i+1 ) ⊆ I.We also consider the ordering b 0 otherwise.Labelling function of T r : To define lab r , for a state (l, b i ), we let a Φ j ∈ lab r (l, b i ) if and only if b i ∈ Sat[l, Φ j ], for j ∈ {1, 2}.The states (l, b + i ) and (l, b − i+1 ) are labeled depending on the truth value of the Φ j 's in the interval (b 2, over the state space of T[P].Consider location l ∈ L. For b ∈ C, we have α(l, b) = α(l, b), β(l, b) = β(l, b), γ(l, b) = γ(l, b) and δ(l, b) = δ(l, b).

Figure 3 :
Figure 3: Example of optimal value for α

Table 1 :
Complexity results for model checking probabilistic timed automata Similarly, it is common to observe complex real-time behaviour in systems.
• Discrete TMDPs are TMDPs in which (1) the state space S is finite, and (2) the transition relation → is finite and of the form → ⊆ S ×N×Dist(S).In discrete TMDPs, the delays are interpreted as discrete jumps, with no notion of a continuously changing state as time elapses.The size |T| of a discrete TMDP T is |S| + | → |, where | → | includes the size of the encoding of the timing constants and probabilities used in → : the timing constants are written in binary, and, for any s, s ′ ∈ S and (s, d, ν), the probability ν(s ′ ) is expressed as a ratio between two natural numbers, each written in binary.We let T u be the untimed Markov decision process (MDP) corresponding to the discrete TMDP T, in which each transition (s, d, ν) ∈ → is represented by a transition (s, ν).A discrete TMDP T is structurally non-Zeno when any finite path of T of the form s