Extensional and Intensional Semantics of Bounded and Unbounded Nondeterminism

We give extensional and intensional characterizations of functional programs with nondeterminism: as structure preserving functions between biorders, and as nondeterministic sequential algorithms on ordered concrete data structures which compute them. A fundamental result establishes that these extensional and intensional representations are equivalent, by showing how to construct the unique sequential algorithm which computes a given monotone and stable function, and describing the conditions on sequential algorithms which correspond to continuity with respect to each order. We illustrate by defining may-testing and must-testing denotational semantics for sequential functional languages with bounded and unbounded choice operators. We prove that these are computationally adequate, despite the non-continuity of the must-testing semantics of unbounded nondeterminism. In the bounded case, we prove that our continuous models are fully abstract with respect to may-testing and must-testing by identifying a simple universal type, which may also form the basis for models of the untyped {\lambda}-calculus. In the unbounded case we observe that our model contains computable functions which are not denoted by terms, by identifying a further"weak continuity"property of the definable elements, and use this to establish that it is not fully abstract.


Introduction
This paper describes denotational models of sequential higher-order functional computation with nondeterminism, which may be given as an explicit choice operation or arise via an under-specified operational semantics or abstract interpretation of a richer language. Interpreting nondeterminism in a sequential setting presents some novel challenges for denotational semantics -for example, accurately representing the branching points within a program at which choices are made. Moreover unbounded nondeterminism allows programs to be written which will always return a value but may take an unbounded number of steps to do so (corresponding to the notion of fairness [Dji76]). Any sound denotational interpretation of such programs will not be a continuous function, meaning that many standard semantic techniques are not available.
We take an approach which relates extensional and intensional representations of nondeterministic functional programs, by developing the theory of biorders (sets with two partial orders) and applying it to nondeterminism, and defining a new notion of ordered concrete data structure -a game-like structure representing the interaction between a program and its environment at a given type. Our key results are based on an isomorphism between the monotone stable functions between the biorders of states of ordered concrete data structures, and the sequential algorithms which compute them. We show that these equivalent representations may be used to give may-and-must testing models of an "observably sequential" functional programming language with nondeterminism, Λ + (N ) . These models are fully abstract when restricted to bounded choice, as is shown by identifying a simple universal type, of which every other computational type is a retract. However, the model of must testing for unbounded non-determinism contains a class of elements which are not definable as terms, leading to a failure of full abstraction and suggesting that it could be further refined by restricting to "weakly continuous" functions.
1.1. Related Work: Extensional Semantics for Non-determinism. There is a body of research describing the appropriate order-theoretic structures for representing nondeterminism denotationally, and establishing their relationship to notions of testing and equivalence -e.g. [HA80,Plo82]. The principal difficulty which arises when interpreting countable nondeterminism -the non-continuous nature of such a semantics -may be resolved by weakening the continuity properties demanded (e.g. to ω 1 -continuity [AP86]). However, this admits many undefinable functions and leaves fewer principles with which to reason about program behaviour (not least, proving computational adequacy).
In this work we focus on the relationship between sequentiality of (higher-order) functions and non-determinism, axiomatizing the structure required to interpret choice in this setting. More specifically, our approach is based on two orders -an extensional order corresponding to the observable input/output behaviour of functions, and a stable order, capturing, in effect, the minimal computation required to produce a given result. This is foreshadowed by Roscoe's work on the semantics of CSP (in which the stable order is called the "strong order") [Ros93a,Ros93b] and Berry's introduction of biorders [Ber78], which combine both orders in constructing models of functional (but deterministic) languages. In previous work, the author has observed that stable and continuous functions on biorders with a (extensionally) greatest element are sequential in the sense of Milner and Vuillemin, and used them to construct fully abstract models of sequential languages such as the lazy λ-calculus [Lai03]. However, although these models technically carry information about the way higher-order programs evaluate their arguments, this must be recovered from the graphs or traces of the functions they denote, suggesting that an alternative representation would be valuable for program analysis.
One approach is suggested by the work of Curien, Plotkin and Winskel [CWP97], which describes a fully faithful functor into the category of bidomains from a Cartesian closed category constructed from a model of linear logic based on bistructures. This is a more general representation of bidomains than that given by ordered concrete data structures (i.e. the biorders which are finite-branching states of the latter are strictly included in those which correspond to cliques of the former). Thus it includes stable and extensionally continuous functions which are not sequential, such as Gustav's function [Ber78]. As a continuous semantics, it does not capture programs with unbounded nondeterminism. extending these models with non-determinism by relaxing the determinacy constraint on strategies is a natural step. This was taken by Harmer and McCusker [HM98], who gave a game semantics of Idealized Algol with bounded non-determinism, fully abstract with respect to may-and-must testing equivalence. This bears clear parallels with our intensional model, e.g. in the explicit recording of divergent traces. Representing the branching behaviour of functional programs without state is more challenging; witness the difficulty of combining innocence with nondeterminism [Har99], resolved in work by Tsukada and Ong [TO15] based on a sheaved model, as well as in concurrent games models [RW11] This succeeds in characterizing the definable elements in a model of the λ-calculus with choice, although it does not capture their observational equivalence in a direct or effective manner. Intensional models of unbounded nondeterminism encounter challenges related to non-continuity -as noted in [HM98], representations of strategies as collections of finite sequences of moves are insufficient to capture the distinction between infinite interactions and finite, but unbounded ones, and Levy [Lev08] describes a semantics of recursively defined nondeterministic programs in terms of their infinite traces. Our model similarly contains representations of infinitary interaction, although these are positions which may be reached by (potentially, many different) ordinal chains of events.
The intensional semantics for nondeterminism developed here is based on concrete data structures, which were introduced by Kahn and Plotkin [KP93], as part of a definition of sequentiality for (deterministic) higher-order functionals (see further discussion in Section 6.2), although they offer an appealing model of computation in their own right, via the notion of sequential algorithm [BC82]. On the one hand, concrete data structures correspond to a positional form of games, and sequential algorithms to positional strategies (see e.g. [HS02]). On the other, sequential algorithms may be related to purely extensional models: in the deterministic case, Cartwright, Curien and Felleisen [CCF94] have established that they compute, and are equivalent to "observably sequential" functions; the author has given a more abstract representation of the latter as bistable functions on bistable biorders [Lai07a,Lai05].
Berry and Curien's deterministic sequential algorithms are strongly sequential -in each position the data required to compute further is unique (each cell is filled with a unique value). By contrast, we describe a weakly sequential representation (cells may be filled with multiple values) -which may be required either because evaluation is explicitly non-deterministic, or because evaluation order is not observable -by abandoning this consistency condition. However, this also requires an ordering on cells and values (corresponding to game positions), to reflect the fact that (for example) any program which may diverge in response to a given argument may still diverge in response to an argument with a wider range of behaviours. This notion of an ordered concrete data structure was first introduced in [Lai09], in which stable and continuous functions were shown to correspond to finite-branching sequential algorithms. Extension to unbounded nondeterminism [Lai15] requires a new notion of transfinite (ordinal-indexed) interaction, to distinguish computations which are infinite from those which are finite but unbounded.
1.2. Outline of the Paper. Section 2 recalls and brings together some earlier work by using it to give a biorder semantics of a minimal functional language with non-determinism, Λ + . This illustrates some features of sequential nondeterministic higher-order functions, and motivates the rest of the paper which expands and elaborates on this model. Section 3 extends it with infinitary data (natural numbers) and fixed points, allowing the expression of equivalent when applied to any closed term. So, in particular, there are only finitely many equivalence classes at each type.
• In this finitary setting, may testing and must testing (and thus the corresponding notions of approximation and equivalence) are dual, in the sense that t ⇓ may if and only if t[Ω/H, H/Ω] ⇓ must . So one model will suffice for both. • The operational semantics for must testing is the same as that given for Unary PCF in e.g. [Loa98]. Unary PCF is the simply-typed λ-calculus over a ground type containing a single value, and a single binary operator on this type which tests both of its arguments for convergence to that value. The latter is usually regarded as a sequential composition, but since the order of evaluation is not observable, the arguments may be evaluated in either order, or in parallel, or non-deterministically.
By relating non-determinism to Unary PCF we may use some important facts about the latter -including that its observational equivalence is decidable [Loa98], and that it has a universal model in the category of biorders and monotone stable functions [Lai03]. Conversely, our sequential algorithms model may be seen as a solution to the problem of giving a corresponding intensional semantics of Unary PCF. Unary PCF (and thus Λ + ) is a sequential language in the sense of Milner-Vuillemin: for any term t : T 1 → . . . T k → o which is strict (i.e. there is some sequence of arguments on which it diverges) there is a sequentiality index i ∈ {1, . . . , k} such that t s 1 . . . s k ⇓ must implies s i must λ x.Ω. In fact, it is weakly sequential -there are strict but non-divergent terms with more than one sequentiality index -e.g. λxy.x + y. As in this instance, weak sequentiality may arise because evaluation is non-deterministic, or because the order of evaluation is underspecified or unobservable. Other examples of weakly sequential languages include PCF itself (for which the fully abstract model is known to not be effectively presentable [Loa00]), and the call-by-value and lazy λ-calculi [Abr90] -e.g. in the absence of side-effects such as mutable state, a call-by-value application t s may be evaluated by first evaluating t to a value, then s, or vice-versa (or, indeed, non-deterministically or in parallel) -which do have effectively presentable fully abstract models based on biorders [Lai03].
Intensional (game) semantics are typically based on explicitly sequential representations of interaction such as sequences of moves: one of the aims of the current work is to investigate a model of higher-order computation which is intrinsically weakly sequential. The following example illustrates why this is needed to give fully abstract semantics of non-deterministic functional programs.
Example 2.1. Define the following terms x 0 : bool, x 1 : bool t ij : bool • t 0,0 If x 0 then (If x 1 then tt else tt) else (If x 1 then ff else ff), • t 1,1 If yx 1 then (If x 0 then tt else tt) else (If x 0 then ff else ff).
Each term evaluates both arguments and returns one of them -t ij evaluates argument i first and returns the value of argument j. Thus they are pairwise (may or must) observationally distinguishable -t 0,1 and t 0,0 may be distinguished from t 1,0 and t 1,1 by application to Ω and H, and t 1,0 and t 0,0 may be distinguished from t 0,1 and t 1,1 by application to tt and ff. However the terms t 0,1 + t 1,0 and t 0,0 + t 1,1 are may-and-must equivalent -both can test their arguments in either order, and return either one of them. Note that they are observationally distinguishable using mutable state (e.g. in erratic Idealized Algol), by using an imperative variable to record the order in which the arguments actually were tested.
A second (well-known) example illustrates another difficulty of giving an intensional semantics of non-deterministic functions -in addition to capturing the order in which they explore their arguments, it is necessary to represent the branching points where different evaluation paths may be chosen.
Example 2.2. The following approximations are strict: λf.(f tt) + (f ff) may λf.f (tt + ff) and λf.f (tt + ff) must λf.(f tt) + (f ff) Indeed, these terms are separated by application to λx.If x then Ω else (If x then H else Ω) (may-testing) and λx.If x then H else (If x then Ω else H) (must-testing).
To capture the difference between these functions at the level of the strategies which compute them, it is necessary to record whether different calls by f to its argument may be supplied with different values. For strategies which are innocent (i.e. not stateful) this requires a representation of interaction which captures branching points, such as the sheaved model of [TO15] 2.1. Biorders and Stable Functions. Turning to denotational semantics, we recall, redefine and develop the notion of biorder [Ber78,CWP97]. At its most basic a biorder is a set with two partial orders, related as follows: Definition 2.3. A biorder (|D|, E , ≤ S ) is a set with two partial orders (the extensional and stable orders) such that any non-empty set X ⊆ |D| which is bounded above in the stable order (for which we may write ↑ X) has a greatest lower bound X in that order, which is also a least upper bound X for X in the extensional order.
Remark 2.4. Note that it is a consequence of this definition that the stable order is contained in the reverse of the extensional order, unlike the original definition [Ber78], where it appears as a refinement of that order. This is a presentational choice -it is possible to reframe the following definitions in the original style -which has some advantages in the context of interpreting non-deterministic computation: • May-and-must duality -as we have noted, the may and must orders (which correspond to the extensional order and its dual) are dual to each other in the finite setting of Λ +one is a meet semilattice, the other is a join semilattice. Our definition allows a single notion of stability, rather than (in the may testing case) using its dual, co-stability. • Continuity -introducing fixed points breaks the duality between may and must (and opens the space between finite and unbounded nondeterminism in the must case). Our definition of biorder accommodates this by allowing the definition of stably continuous and extensionally monotone functions (must testing) versus extensionally continuous and stable functions (may testing) in the same setting. • In our intensional model, which essentially extends sequential algorithms with explicit divergences, it is natural to interpret the extensional order as a containment between states, but the stable order as a refinement which removes divergences. • We identify a new property of programs with unbounded nondeterminism -that they preserve suprema of extensional chains, but only up to stable approximation -which is most naturally presented as a weak continuity property. We shall say that a biorder is extensionally pointed if it has an extensionally least element, ⊥ E , stably pointed if it has a stably least element ⊥ S (note that the latter element is greatest in the extensional order), and just pointed if it has both.
Key constructions on biorders include: • For any family of biorders {D i | i ∈ I}, the (set-theoretic) product Π i∈I D i and disjoint sum i∈I D i with both orders defined pointwise. • If D is a biorder, its stable lifting D ↑ is given by adding a new element to D which is least with respect to the stable order (and thus greatest with respect to the extensional order). From these constructions we obtain the empty biorder, the one-element biorder 1 and its stable lifting -the (pointed) biorder Σ, with two elements {⊥ S , ⊥ E } such that ⊥ E E ⊥ S and ⊥ S ≤ S ⊥ E . For any set X, we write X for the disjoint sum x∈X 1 (in which the extensional and stable orders are both the discrete order).
To define a category of biorders, we recall Berry's notion of stable function.
Definition 2.5. A function f : D → E between partial orders is stable if it is monotone and for any x ∈ D and y ≤ f (x) there exists m ≤ x such that y ≤ f (m) and for all z ≤ x, y ≤ f (z) implies m ≤ z.
The relationship between stability and bounded infima may be summed up as follows: Lemma 2.6. If partial orders D and E have bounded infima then f : D → E is stable if and only if it preserves all bounded infima.
Proof. From left-to-right, suppose Z is bounded above by x, then taking y = z∈Z f (z) there exists m such that y ≤ f (m) and for all z ∈ Z, m ≤ z and thus z∈Z f (z) = f ( Z) as required.
From right-to left, given y ≤ x, take m = {a ≤ x | y ≤ f (a)}.
We define a category B in which objects are biorders, and morphisms from D to E are functions from |D| to |E| which are monotone stable -i.e. monotone with respect to the extensional order and stable with respect to the stable order. This concrete category is Cartesian closed -the product of biorders is a cartesian product and the internal hom [D, E] is the biorder consisting of the set of monotone stable functions from D to E with the extensional (Scott) and stable (Berry) orders on functions -i.e.
The infimum of a stably bounded set of monotone stable functions is given pointwise -i.e. ( F )(x) = f ∈F f (x), which is clearly stable and monotone, and a E -supremum for F . It is well known (and an interesting exercise to prove) that stability of f : D × E → F implies stability of its currying Λ(f ) : D → [E, F ] and vice-versa. Thus B(D × E, F ) and B(D, [E, F ]) are naturally isomorphic and so B is Cartesian closed, as is its subcategory of pointed objects. So, in particular, it contains may and must testing denotational semantics of Λ + in which the ground type o is interpreted as the biorder Σ, the choice operator as the extensional join operator : Σ × Σ → Σ (which is monotone stable), and the constants H and Ω as ⊥ S and ⊥ E (respectively) in the may testing model and ⊥ E and ⊥ S in the must testing model.  Proof is simple based on strong normalization of the λ-calculus and soundness of the reduction rules. We will give a proof of adequacy for a conservative extension of this model (Proposition 3.23).
2.2. Sequentiality and Universality. Sequentiality for higher-order functions may be defined in a variety of ways, which will be discussed further in later sections: via correspondence with an explicitly sequential representation such as games or sequential algorithms (see Section 5) via Kahn-Plotkin sequentiality indices [KP93] (see Section 6), or via universality and full abstraction results for an interpretation of a sequential language (here, for Λ + and in Section 7 for a more expressive extension). First, we recall from [Lai03] a simple result establishing that conditionally multiplicative functions on biorders are sequential in the "first-order" sense of Milner and Vuillemin.
Proposition 2.9 (Milner-Vuillemin Sequentiality). Any strict morphism f : Then {⊥ E [⊥ S ] i | i ∈ I} is stably bounded above by ⊥ E , and has stable greatest lower bound Note that a function may be i-strict for several values of i (for example, the function from Σ × Σ to Σ sending x, y to x y). In other words our model contains weakly sequential functions, as expected.
It was shown in [Lai03] that any (Milner-Vuillemin) sequential and order-extensional model of Unary PCF in a CCC which interprets the ground type as the two-point order, is universal (i.e. every element is denoted by a term) and thus fully abstract. We sketch a simplified version of the proof here, which is the basis for a proof of universality for an infinitary extension of Λ + in Section 7.
Proof. By duality, we may give the proof for the must testing model only. For any types S and T , and n ∈ N, write S ¢ T n if there is a definable retraction from S to T n -that is, there are terms inj 1 : S → T, . . . , inj n : S → T and proj : The following facts are evident: • If S ¢ T n then universality at type T implies universality at type S.   Proof. This follows from adequacy, universality and extensionality of the model following standard arguments as in [Plo77].
Thus, we have given a fully abstract semantics of a simple, sequential non-deterministic functional language using biorders. In the remainder of the paper, we will extend it to a a fully expressive model of computation, and give an intensional characterization of its types and programs.
3. Λ + (N ) and its semantics Λ + is too limited to describe generalized higher-order non-deterministic computation because it lacks infinitary datatypes. In particular, this means that there is no real distinction between may and must testing (witness the formal duality between them) and there is no possibility of describing unbounded nondeterminism and comparing it with the binary choice operation. So we will define a new prototypical language by extending Λ + with a type N of natural number values -i.e. terms of type N do not have computational effects (nondeterminism or non-termination). Maintaining this distinction between computation and value types (as in call-by-push-value [Lev04]) simplifies programming with non-determinism and allows construction of fully-abstract semantics of richer languages by CPS interpretation.
Types of Λ + (N ) are either pointed (computation) types P or possibly unpointed (value) types V , given by the grammars: which its argument is applied. Λ + (N ) may be regarded as a target language for continuation-passing-style translation of languages with bounded and unbounded nondeterminism, with o as the return or answer type. Most simply, by defining the type nat of natural number computations to be the type (N → o) → o, we may express the language SPCF (observably sequential PCF [CF92]) extended with bounded (binary) or unbounded (natural number) choice. In particular, note that we may define pred : SPCF is itself an extension of PCF with a simple non-local control operator (e.g. we may define callcc : ((nat → nat) → nat) → nat λf.λk.(f λxy.x k) k) and some error terms which immediately abort computation (in this case, a single such term, H). SPCF provides a complete syntactic representation of the (strongly) observably sequential functions (and the corresponding deterministic sequential algorithms) [CCF94,Lai07a] as demonstrated by full abstraction and universality results -this paper may be seen as an extension of these results to weakly observably sequential functions and non-deterministic sequential algorithms.
3.1. Operational Semantics. We give may and must testing operational semantics for programs (closed terms of type o) by extending the rules for Λ + (Table 1) with the additional rules in Table 2. The definitions of may, must and may-and-must approximation and equivalence extend directly from Λ + . Also as in Λ + , the presence of the error element makes the language extensional -any terms of function type are observationally equivalent if (and only if) applying to the same argument returns equivalent results. (This is essentially Milner's Context Lemma [Mil77] and may be proved using the same techniques.) In particular, we will use the result for must equivalence in Section 7.
Lemma 3.1 (Function Extensionality). For any closed terms s, t : S → T , if s r must t r for all r : S then s must t.
3.2. Unbounded Nondeterminism. Λ + (N ) gives us a setting in which we may express programs with unbounded non-determinism -they may evaluate to (countably) infinitely many different values without diverging. Evidently, we may express bounded choice up to may-and-must equivalence using unbounded choice -e.g. s + t may must ? N (λk.(Eq(k, 0) s) t). Conversely: Proposition 3.2. ? N is macro-expressible up to may equivalence in bounded Λ + (N ) .
Proof. The terms ? N and λk.(Y nat λf.λu.(k u) + (f (suc x))) 0 are may-equivalent. (They denote the same elements in our fully abstract model of may testing.) Note that this equivalence fails with respect to must equivalence (evaluation may always take the right hand branch of the binary choice and so diverge), as will any such attempt to define countable choice using bounded choice (a term which reduces to infinitely many different values will have an infinite reduction path by König's lemma and may therefore diverge). In other words: Proposition 3.3. ? N is not macro-expressible up to must equivalence using binary choice.
Formal proof of this claim is straightforward using our fully abstract model (Section 7).
Remark 3.4 (Fairness). One reason for studying unbounded nondeterminism is its close connection with the notion of fairness, according to which an event will eventually occur, even if it is not possible to bound the number of steps taken before this happens. For example, suppose we wish to implement a type coin with constants new coin : (coin → o) → o (which supplies a coin to its argument) and toss : coin which tosses its first argument and chooses its second or third argument depending on whether the result is heads or tails, and passes the tossed coin to it. A necessary condition for an implementation of coin to be fair is that tossing a new coin will eventually come up heads, but may require an unbounded number of tosses to do so. In other words, a program which repeatedly tosses a coin and increments a counter until it gets a head, and then returns the contents of the counter may return any number value (but not diverge) -i.e.
is must equivalent to ? N . So a fair coin may be used to express unbounded choice and is therefore not definable in bounded Λ + (N ) by Proposition 3.3. Conversely, in unbounded Λ + (N ) we may implement a coin which passes the fairness test, e.g. by defining the macros coin N , new coin ? N and toss λu.λf.λg.(Eq(u, 0) (? N f )) (pred(u) g).
Remark 3.5 (Abstract Interpretation). Another setting in which unbounded nondeterminism may arise is in the "abstract interpretation" of a domain with infinitely many states as a finite domain -the "concretization" of such an abstract interpretation may then represent a single abstract state as an unbounded choice of concrete states. Observe that in Λ + (N ) , countable nondeterminism may be used to define an embedding-projection pair from the x and λy.e (p y) must λy.y. In other words, there is a Galois insertion between N → o and o, giving an abstract interpretation of the former, infinitary type inside the latter, finitary one, which we may use to define, inductively, a Galois connection between each pointed type of Λ + (N ) and the corresponding type of Λ + obtained by erasing all instances of N .
3.3. Continuity. A challenge encountered in defining and reasoning about an adequate denotational semantics of must testing for programs with unbounded non-determinism is that it cannot be continuous with respect to the must ordering (on must equivalence classes of terms).
For instance, define t n : N → o Y N →o λf.λu.(Eq(n, u) H) (f suc(u)), so that t n i ⇓ must iff i ≤ n. It is easy to see (and prove using our fully abstract model) that t n must t n+1 for each n ∈ N, and that the must -least upper bound for this chain of terms (up to must equivalence) is the term λf.H. However, whereas ? N t n may diverge for all n, ? N (λf.H) must converge -i.e. ? N cannot denote a function which is continuous with respect to the must ordering.
Another example shows that function application is not continuous in the function component either -or in other words, that least upper bounds of chains of functions are not given pointwise, in general. For instance, let s k : which nondeterministically supplies to its argument a numeral greater than or equal to k. It is evident that s k must s k+1 for each k ∈ N, and that the must -least upper bound for this chain of terms (up to must equivalence) is the term λf.H. However, s k λu.Ω diverges for all k, but (λf.H) λu.Ω must converge.
This suggests that we cannot define well-behaved fixed points for functions by taking least fixed points of chains of approximants with respect to the must approximation ordering. This problem will be resolved semantically by defining least fixed points with respect to the stable order rather than the extensional (observational) order. Application is stably-continuous with respect to chains of functions, although not their arguments. (The t i defined above denote a stable chain in our model, but the s k do not.) Failure of continuity also entails that there are must inequivalent terms which cannot be distinguished by finitary tests (i.e. by applying them to arguments which are compact in the must ordering). We give an example at the end of this section, after developing the necessary interpretation of fixed points.
3.4. Complete Biorders. To extend our model of Λ + to Λ + (N ) we require biorders which are directed complete with respect to at least one of their orders, in order to define least fixed points with respect to that order. We now define these notions of complete biorder.
e. every such family has a least upper bound, and for any E -directed I-indexed families x and y, It is not hard to show that the product, disjoint sum, lifting and function-space preserve extensional completeness.
The requirements for a biorder to be stably complete are slightly more involved. They include a "bounded distributivity condition" requiring certain bounded meets to distribute over directed joins, which will be used to define CCCs of stably complete biorders.
Definition 3.7. A stable upper bound for a set X of stably directed, I-indexed families over a biorder D is an I-indexed, stably directed family y such that for all x ∈ X and i ∈ I, x i ≤ S y i , and for all i, j ∈ I, D is stably complete if the stable order is directed complete and for any stably bounded set X of stably directed I-indexed families of elements, x∈X i∈I x i = i∈I x∈X x i .
A biorder is bicomplete if it is both extensionally and stably complete.
It is easy to see that the coproduct and product of families of stably complete domains are stably complete. To show that lifting preserves stable completeness (so in particular Σ is stably complete) is a little harder. Note that the above distributivity property does not hold in general for unbounded sets of stably directed families over Σ (i.e. it is not a completely distributive lattice) -take, for example the set of families {{n i | i ∈ ω} | n ∈ N} with n i = S if i > n and n i = ⊥ S otherwise, which satisfies i∈ω n∈N n i = ⊥ S and n∈N i∈ω n i = S . Lemma 3.8. If D is stably complete then D ↑ is stably complete.
Proof. We check that the bounded distributivity property holds. Suppose the set X of I-indexed directed families is bounded above by {y i | i ∈ I}. By definition, i∈I x∈X x i ≤ S x∈X i∈I x i . If i∈I x∈X x i = ⊥ S then defining J = {i ∈ I | x∈X x i = ⊥ S , we have a set of J-indexed directed sets of elements of D, bounded above by {y j | j ∈ J} and so i∈I x∈X x i = j∈J x∈X x j = x∈X j∈J x j = x∈X i∈I x i by stable completeness of D.
So we need to consider the case in which (a) i∈I x∈X x i = ⊥ S and (b) x∈X i∈I x i = ⊥ S . We claim that this implies x i = ⊥ S for all i ∈ I and x ∈ X and so in fact x∈X i∈I x i = ⊥ S as required. To prove the claim, choose any i ∈ I. There exists z ∈ X such that z i = ⊥ S by (a) and j ∈ I such that z j = ⊥ S by (b). Because z i ∧ y j = y i ∧ z j , we have y i = ⊥ S , and hence x i = ⊥ S for all x ∈ X as required. Proof. Joins of stably directed sets are defined pointwise (i.e. function application is stably continuous with respect to functions). To show that these are stable, we need to show that they commute appropriately with bounded meets using the bounded distributivity property. Suppose F is a stably directed set of monotone stable functions from D to E, and X is a set of elements stably bounded above by y.
So, in particular, the full subcategory of B consisting of bicomplete biorders is Cartesian closed. We define the following subcategories of B in which morphisms are continuous: • B E -objects are extensionally complete biorders and morphisms from D to E are functions from D to E which are continuously stable -i.e. they are monotone stable and preserve suprema of extensionally directed sets. • B S -objects are stably complete biorders and morphisms from D to E are functions from D to E which are stably continuous -i.e. they are monotone stable and preserve suprema of stably directed sets. Each of these categories is Cartesian closed, with products given pointwise, and internal homs given the by relevant biorders of functions -i.e. [D, E] B E is the restriction of [D, E] to continuously stable functions and [D, E] B S is the restriction of [D, E] to stably continuous functions. We need to check that in each case this defines an object of our categoryi.e. is (respectively) extensionally complete or stably complete. We give the second (more interesting) case. Proof. The proof follows that of Lemma 3.9. We also need to show that if F is a set of continuously stable functions, stably bounded above by a (continuous) function g, then F is continuous: if X is a stably directed set then Finally, we need to check that the isomorphism of biorders Λ : restricts to each of our categories -i.e. Λ(f ) is respectively continuously stable or stably continuous if and only if f is continuously stable/stably continuous. This is straightforward, based on these properties for the Scott and Berry orders. Thus we have shown: Proposition 3.11. The categories B E and B S are Cartesian closed.
As an example we show that for any set X, the function space corresponds to a powerdomain biorder on X, justifying the identification of the Λ + (N ) type (N → o) → o with the type nat of natural number computations.
Definition 3.12. Given a set X, let P(X) be the biorder consisting of subsets of X, with the extensional order being inclusion and the stable order discrete. There is an evident strict morphism φ : P(X) ↑ → [Σ X , Σ] sending Y ⊆ X to y∈Y π y . To show that this is an isomorphism by giving an inverse, we define: Definition 3.13. Given monotone stable f : Π i∈I A i → Σ let si(f ) be the set of i ∈ I such that f is i-strict (the sequentiality indices of f ). By Proposition 2.9 this is non-empty if and only if f is strict. Proof. By Lemma 3.14, f (x) = ⊥ S if and only if π i (x) = ⊥ E for some i ∈ si(f ), and thus we may define an inverse to φ : but not stably continuous (cf the non-continuity of unbounded choice with respect to must testing).
So defining P fin (X) to be the restriction of P(X) to finite subsets of X, we have: 3.5. Denotational Semantics of Λ + (N ) . To interpret recursive higher-order programs, we require fixed points of endomorphisms f : D → D, where D is extensionally or stably complete and pointed. These may be obtained in standard fashion, as (respectively, extensional or stable) suprema of the chain of approximants ⊥, f (⊥), . . . , f i (⊥), . . .. In the categories B E and B S , which are cpo-enriched, the fixed point is reached at f ω , whereas in the category of biorders and monotone stable functions it may be necessary to continue the chain beyond ω, but it will still reach a stationary point [AP86].
Proposition 3.17. If D is stably (resp. extensionally) complete and pointed then every monotone stable function f : D → D has ≤ S -least (resp. E -least) fixed points.
Proof. Define the chain of stable approximants f λ ∈ D for each ordinal λ by: By Hartog's Lemma this has a stationary point, which is a ≤ S -least fixed point for fevidently if f is λ-continuous (preserves suprema of λ-chains) then this stationary point is f λ .
We now have the basis for may testing and must testing semantics of Λ + (N ) in our categories of biorders -more precisely: • May testing and must testing semantics of unbounded Λ + (N ) in the category of bicomplete biorders and monotone stable functions. (As we have noted, unbounded choice is expressible up to may-testing in bounded Λ + (N ) , so the bounded/unbounded distinction is moot in this case and we focus on must testing for unbounded nondeterminism). 3.6. Computational Adequacy. Soundness of the reduction rules with respect to the denotational semantics follows from the structure identified in our categories, yielding: We establish the converse for terms of type o (computational adequacy) by defining "approximation relations" in the style of Plotkin [Plo85]. The models in which this proof is not completely standard are those which are not ω-cpo-enriched -in particular, the case of unbounded nondeterminism with must testing, which we give here. Plotkin's proof still adapts to this case because fixed point combinators denote least upper bounds of stable chains, which are defined pointwise at function type.
For each type T we define a relation ¡ T between elements of [[T ]] and closed terms of type T : We want to show that every program is approximated by its denotation, for which we use the following two lemmas, proved by straightforward induction on type structure.
Lemma 3.19. If t ⇓ must C implies s ⇓ must C then e ¡ T t implies e ¡ T s.
Lemma 3.20. If e α | α < λ is a stable chain of functions such that e α ¡ T t for all α < λ then α<λ e α ¡ T t.
The key lemma for our proof establishes that the fixed point combinator is approximated by its denotation. Proof. We show by induction on λ that   Example 3.25. Consider the terms:  ((f i) H)  We will study the completeness (full abstraction) properties of our models in Section 7. First, we develop a setting in which we can give intensional characterizations of them.

Ordered Concrete Data Structures
We will now define the notion of ordered concrete data structure and use it to give an intensional representation of monotone stable functions. It is based on the original notion of concrete data structure, introduced by Kahn and Plotkin [KP93] and further developed by Berry and Curien [BC82].
A concrete data structure consists of sets of cells, values and events (which are pairs of cells and values), and an enabling relation between events and cells. The idea is that each step of a sequential computation is represented as an event (the filling of a cell with a value), which may be dependent on some combination of previous events having occurred (as specified by the enabling relation). Programs correspond to states which specify values for filling enabled cells -sets of events which satisfy two conditions: consistency -every cell must be filled with a unique value -and safety -for every filled cell there is a finite chain of enablings of filled cells within the state, back to an "initial cell" which does not depend on any prior events. In order to model nondeterministic computation we adapt this setting in the following ways: • Removing the consistency condition, so that a cell may be filled with multiple different values. • Placing an ordering on cells and values, (and thus events) and requiring states to be upwards closed under this ordering. This reflects the fact that (for example) the responses of a program to an input which is a nondeterministic choice of a and b must include all of its responses to both a and b. • Including a distinct element • -representing failure by divergence or error, depending on may or must testing interpretation -with which any cell may be filled (cf the representation of divergence in the game semantics of must testing in [HM98]). • Extending the safety condition to allow transfinite ordinal chains of enabling events.
(Capturing the distinction between infinite, and finite but unbounded interaction.) Remark 4.1. We may think of a concrete data structure as a kind of two-player game in which one player chooses a cell for the second player to fill with a value of their choice, enabling a choice of cells for the first player, and so on. In the deterministic setting, this analogy can be made into a precise correspondence between the category of concrete data structures and sequential algorithms, and a category of graph games and strategies [HS02]. However, this correspondence depends on determinacy to decompose the CCC of sequential algorithms into a model of linear logic from which the graph games model is constructed: we leave the description of a more explicitly games-like characterization of ordered concrete data structures as future work.
Note that: • Every event in a proof in A except (perhaps) the last one is in E(A).

Definition 4.4. A state of an ocds A is a set of events x ⊆ E(A) • which satisfies:
Upwards Closure: If e ∈ x and e ≤ e then e ∈ x. Safety: If e ∈ x then there exists e ∈ x such that e ≤ e and x * e .
We write D(A) for the set of states of A. Some further conditions on states will be useful. Note that any total state which satisfies infinite livelock also satisfies finite safety.
Example 4.6. For any set X, the ocds X = ({c}, X, {( * , c)}, {c} × X), has a single initial cell c, which may be filled with any value in X (ordered discretely). Clearly, every state of X trivially satisfies finite safety and infinite livelock • The total states of X are in one-to-one correspondence with the subsets of X, • The total and finite-branching states are in one-to-one correspondence with the finite subsets of X. • There is a single non-total (and finite-branching) state containing (c, •) and thus every event of X.
A second key example is the following "universal" ocds, which will play an important role in our full abstraction result. Notwithstanding the simplicity of U, for any countable ocds A, each state x ∈ A may be uniquely represented as a state of U: let {e i | i ∈ ω} be an enumeration of E(A) • and define inj(x) ∈ D(U): The following construction illustrates the role of the partial order on the cells of an ocds. There is a subclass of ocds for which this order is given by the transitive closure of the relation: c c if there exists v such that (c, v) c . In fact, this forms a sub-CCC of our category of ocds. However, it is not closed under the following lifting operation, which adds an unfillable cell which can therefore not enable any other cells. We will also write x + (c, V ) for v∈V (x + (c, v)).
Say that a state x is complete when every cell with a finite proof in x is filled. Proofs are ascending chains of events (c 0 , suc), (c 1 , suc), . . ., and thus: • The total, complete and finitely safe states are in one-to-one correspondence with the non-empty subsets of N. • The total and complete finitely branching states are in one-to-one correspondence with the non-empty finite subsets of N (since any state in which infinitely many cells are filled must contain a proof of (c ω , •) by the infinite livelock condition). The states which are either non-total or incomplete correspond to a subset of N bounded above by i ≤ ω, together with a proof of c i , which is either filled with • or (for i < ω) unfilled. (Filling c ω with • corresponds to presence of the recursively defined element n := suc(n).)

Ordered Concrete Data Structures as Biorders.
For each ocds, we define a biorder in which the set of elements is the set D(A) of states of A. The extensional order is set-inclusion: since any union of states satisfies the safety and upwards-closure conditions, (D(A), ⊆) is a complete (join) lattice, with least element the empty state and greatest element D(A).
Definition 4.10. The stable order on states is defined as follows: x ≤ S y if y ⊆ x and for all (c, v) ∈ x, either (c, v) ∈ y or (c, •) ∈ x.
(D(A), ⊆, ≤ S ) is a biorder -if X is a non-empty set of states bounded above by y, then X is their greatest lower bound in the stable order: if (c, v) ∈ X then (c, v) ∈ x for some x ∈ X and either (c, •) ∈ x ⊆ X or else (c, v) ∈ y and so for all x ∈ X, (c, v) ∈ x . If z ≤ S x for all x ∈ X then X ⊆ z and if (c, v) ∈ z either ∀x.(c, v) ∈ x ∈⊆ X or (c, •) ∈ z. (D(A), ⊆, ≤ S ) is pointed -its stably least element is D(A).
It is easy to see that D(A) is extensionally complete, but what of stable completeness? In fact, there are pathological examples of ocds in which this property does not hold, but we will show that the ocds for which D(A) is stably complete form a Cartesian closed category. Note that in any case, if ∆ is a non-empty set of states with a stable upper bound then its stable l.u.b. is ∆.
Evidently, the sub-biorder D * (A) consisting of states which are finitely safe is also an (extensionally complete) biorder, as this property is preserved by arbitrary unions of states. The sub-biorder D * (A) consisting of finite branching states is a biorder, although not, in general extensionally complete, since the infinite union of finite-branching states may not be finite-branching. However, it is stably complete. Otherwise there is an event (c , v) in the proof such that (c, v) ∈ y, and hence x ∈ X such that (c , •) ∈ x ⊆ X such that (c , •) ∈ x and so (c, •) ∈ X by upwards closure.
We shall use the following alternative characterizations of the stable order and stable boundedness. Lemma 4.13. y ≤ S x iff y = x C for some C ⊆ En(x).

Proof.
Evidently, x C ≤ S x, so it suffices to show that every element y ≤ S x has this form. Let C = {c ∈ En(x) | (c, •) ∈ y}, so that x C ⊆ y. We claim that y ⊆ x C : suppose (c, a) ∈ y, but (c, a) ∈ x. Let (c α , v α ) | α < κ be a proof of (c, a) in y. Then either (c, a) ∈ x or there exists a least value α such that (c α , v α ) ∈ x. So c α ∈ E(x), and (c, •) ∈ y by stability and so c α ∈ C. Since c α E c, (c, a) ∈ x C as required.
Lemma 4.14. A non-empty set X of states is stably bounded above if (and only if ): Proof. From left-to-right, this follows from the fact that X is bounded above by X. From right-to-left, we need to show that if X satisfies ( †) then X satisfies the safety condition, since it is then a stable upper bound. Suppose (c, v) ∈ X -choose x ∈ X and let P be a shortest proof of (c, a) in x ∈ X (i.e. there is no shorter proof). Then P ⊆ X, since for every preceding element (c , v ) in P we have (c , •) ∈ x (or else a shorter proof would exist) and so (c , v ) ∈ X.
Returning to our examples, we may observe that: • For any set X, D( X) ∼ = P(X) ↑ and D * ( X) ∼ = P fin (X) ↑ . In particular, D( ∅) is the two-element space Σ.
The full, identity-on-morphisms functor D : OC → B sending each concrete data structure to its set of states (which restricts to functors D * : OC E → B E and D * : OC S → B S ) preserves products, and so to establish Cartesian closure in each case, it remains to define an internal hom ocds for each pair of objects and show that its biorder of states is isomorphic to the internal hom (function space) in the corresponding category of biorders. This is a key result, since it establishes that every stable and monotone function between the biorder of states of an ocds is computed by a unique state of this exponential ocds or sequential algorithm.
Focussing first on the general case of monotone stable functions, for each pair of ocds A, B we define an ordered concrete data structure A ⇒ B (cf. the analogous definition of unordered concrete data structure [BC82] , c) In the following, we will assume that the sets of cells of A and values of B are disjoint, and so omit explicit tagging.  ((y, c), b) ∈ σ with y ⊆ x, and a proof of (y, c) in σ which therefore restricts to a proof of c in f (y).
Say that a function f : . We now show that any such function is monotone stable.
Proof. If (c, a) ∈ fun(σ)( X) then there exists an event e ∈ σ such that either e = ((w, c), a), where w ⊆ X or e = ((w, c), c ), where w + (c , •) ⊆ X. If w ⊆ x for every x ∈ X then in the first case (c, a) ∈ fun(σ)(x) for every x ∈ X, and in the second case there exists x ∈ X with w + (c , •) ⊆ x and so (c, •) ∈ fun(σ)(x ).
So suppose w ⊆ x for some x ∈ X. Fixing a proof of e in σ, let e = ((w , c ), a ) be the first element in this proof such that w ⊆ x for some x ∈ X. Then there is an immediately preceding event ((w , c ), c ) such that w = w + (c , V ) for some set of values V , including a value u such that (c , u) ∈ x. Because w ⊆ X, there exists x ∈ X with (c , u) ∈ x . Since x ↑ x , therefore (c , •) ∈ x , and hence (c , •) ∈ fun(σ)(y). Since c ≤ c, we have (c, •) ∈ fun(σ)(x ) as required.
Proposition 5.3. For any sequential algorithm σ, fun(σ) is a monotone stable function.

Examples.
We now describe the sequential algorithms which compute some of the monotone stable functions described in the examples of Sections 2 and 3.
Example 5.4. For any set X, consider the ocds ∅ X ⇒ ∅ -i.e. ∅ X is the X-indexed product of copies of ∅, which has (initial) cells {c x | x ∈ X} and no values, so ∅ X ⇒ ∅ has a single initial cell ({ }, c), which may be filled by any of the cells c x for x ∈ X. In other words, it is the same ocds as X, up to renaming of cells and values, and so By considering the algorithms computing the functions of Example 2.1, we may see that unlike a non-deterministic strategy in e.g. the games model of [HM98], a non-deterministic sequential algorithm may have different decompositions into a non-deterministic choice between two (complete and total) algorithms. So a sequential algorithm in B × B ⇒ B may investigate (both, either or neither) of its arguments in any order, and return a Boolean value. The functions denoted by the terms t i,j of Example 2.1 are computed by sequential algorithms σ i,j as follows: Observe that the cell ({(c 0 , v 0 ), (c 1 , v 1 )}, c) does not depend on i or j, and thus σ 0,0 ∪ σ 1,1 = σ 0,1 ∪ σ 1,0 .
In fact, we can present sequential algorithms from A into ∅ (which might be considered as continuations on A) more simply by ignoring the unfillable cell of ∅ -i.e. cells correspond to states of A ordered by inclusion, values to cells of A, and an event to a pair (x, c) of a state of A and a cell which is accessible from it, which enables the states (new cells) of the form x + (c, V ). The sequential algorithms computing the functions of Example 2.2 may be presented in this way: Example 5.6. Consider the concrete data structure ( B ⇒ ∅) ⇒ ∅. Ignoring the unfillable cell of ∅, its initial cell is the empty state, which can be filled (only) by the initial cell of  Figure 1 for the extensional hierarchy of sequential algorithms at this type.) The above examples relate to finitary (or, at least, finitely safe) sequential algorithms. Recasting Example 3.25 furnishes an instance of an event with an infinite proof which is required to separate the sequential algorithms computing the terms s and t defined there.
Example 5.7. Consider the ocds U ⇒ ∅: again, ignoring the unfillable cell on the right this has as cells total states of U (which correspond to subsets of N), and as values, cells of U (which correspond to elements of N) -an event is thus a pair (X, x) such that x ∈ X, A sequential algorithm on U ⇒ ∅ thus corresponds to a set of finite non-repeating sequences over N, together with a set of divergences -finite and infinite sequences over N ending with a cell filled with • -such that any sequence which extends a divergence appears in both sets. Compare with Harner and McCusker's game semantics of non-determinism [HM98]. The sequential algorithms on U ⇒ ∅ which compute the functions denoted by s and t defined in Example 3.25 contain the same finite sequences of events (corresponding to finite prefixes of 0123 . . .). However, the algorithm computing s contains a divergence corresponding to the infinite sequence 0123 . . ., whereas the algorithm computing t does not.
To be more precise, for each j ≤ ω, let P j be be the state {(c.i, ) | i < j} in D(U), and for each k ≤ ω let Q k be the (upwards closure of) {((P j , c), c j ) | j < k} in D(U ⇒ ∅). Then [[s]] is computed by Q ω ∪ {((P ω , c), •)} and [ [t]] is computed by Q ω -i.e. they are differentiated only by the event ((P ω , c), •), which is the conclusion of the infinite proof ((P j , c), c j ) j<ω in Q ω .
One might wonder whether the only infinite proofs which are needed to distinguish terms of Λ(N ) + are the ones that lead to a divergence in this way. However, at the next order of types (third order) we may give examples of terms denoting total sequential algorithms which are distinguished only by events with infinite proofs. Define p : (U → o) → o λg.g λu.λx.x, and r : (U → o) → o λu.λx.? N λv.(leq(u, v) x) H, so that [[p]] is computed by applying its argument to the infinite state P ω ∈ D(U) and [[r]] by non-deterministically applying its argument to some finite state P j in U. Writing ([p]) and ([r]) for these sequential algorithms: • every event in ([p]) with a finite proof (i.e. the finite states P j for j < ω) also occurs in ([r]). • p and r are distinguished by application to t -p t ⇓ must and r t ⇓ must , since ([p]) contains the event ((Q ω , c), P ω ) -which has the proof ((Q k , c), P k ) k≤ω in ([p]) -but ([r]) does not.
We may extend these examples to give terms which are distinguished only by events with proofs of length ω + l, for any l < ω. Defining U 0 U and U l+1 (U → o) → o, we give terms p l : U l and t l , s l : U l → o by p 0 λu.λx.x and p l+1 λf.f p l , t 0 t and t l+1 λf.f t l and s 0 s and s k+1 λf.f s k . Then t l p l ⇓ must for each l, but s l p l ⇓ must .
Denotationally, for each l ∈ ω, ( we conjecture that any inequivalent terms at a type of order at most 2l may be distinguished by an event with a proof no longer than ω + l.

Stable Functions and Sequentiality
We now return to the task of showing that the functions between the biorders of states of ocds which are computed by sequential algorithms are precisely the monotone and stable ones. Recall that concrete data structures were originally introduced by Kahn and Plotkin in order to give a description of sequentiality for higher-order deterministic functionals [KP93]. Essentially, in this setting a function between the states of concrete data structures A and B is Kahn-Plotkin sequential if any argument (state of A) x, and cell c of B which is filled in f (y) for some y which extends x, can be associated with a cell, accessible from x, which must be filled in any state z (which extends x) such that c is filled in f (z). However, in this original setting, divergence is represented implicitly, by not filling an enabled cell (rather than as an explicit divergence by filling a cell with •), and inclusion of states corresponds to the stable order. Thus we translate this original definition of Kahn-Plotkin sequentiality to the current setting by (essentially) replacing the role of "accessible cell" with that of "cell filled with •", and "filled cell" with "enabled cell not filled with •". We define a partial order ( ) on total states (which plays the role of the stable order in the original definition of Kahn-Plotkin sequentiality): x y if x ⊆ y and if c ∈ F (x) then (c, v) ∈ y implies (c, v) ∈ x. We will now show that all monotone stable functions are explicitly sequential.
Lemma 6.2. If x y then y ⊆ x A(x)∩F (y) .
Proof. Suppose (c, v) ∈ y but (c, v) ∈ x. Let P be a proof of c in y, and (c , v ) the least element of P which is not in x. Then c is accessible (enabled but not filled) in x, as x y - By conditional multiplicativity, f (x A(x)∩F (y) ) = c ∈A(x)∩F (y) f (x + (c , •)) and so there is a cell c ∈ A(x) ∩ F (y) such that (c, •) ∈ f (x + (c , •)) as required.
We establish the converse -that every explicitly sequential function is monotone stable -by showing below that every explicitly sequential function from D(A) to D(B) is computed by a sequential algorithm.  D(A ⇒ B). Define this to be the set of events: and so ((x , c ), a ) ∈ strat(f ). If a, a ∈ C(A), so that a ≤ a and hence x + (a, •) ⊆ x + (a , •), then (c, •) ∈ f (x + (a , •)) and so ((x , c ), a ) ∈ strat(f ) as required.
Proof. We construct a proof of each event ((x, c), a) in strat(f ) using the explicit sequentiality property for f . Suppose a ∈ V(B) • : let (c β , v β ) | β ≤ λ be a proof of (c, a) in f (x).
Hence we have shown that:  To complete the proof that fun is a (bi)order-isomorphism from D(A ⇒B) to [D(A),D(B)] we need to show that it is a biorder-embedding -i.e. that fun and strat are monotone in both orders (from which stability follows). Monotonicity with respect to the extensional order is straightforward, leaving the stable order.
Proof. By induction on n. Suppose we have { ((y 1 , c), c 1 ), . . . , ((y n , c), c n )} ⊆ σ satisfying the above conditions. If any of the c i is filled with • in ∆, then (c, •) ∈ f ( ∆), so suppose that for each i ∈ {1, . . . , n}, (c i , •) ∈ ∆ and so there exists x i ∈ ∆ such that (c i , •) ∈ x i . By directedness of ∆, there exists x ∈ ∆ such that none of the c i are filled with • in x. Since (c, •) ∈ fun * (σ)(x), there exists an event ((y, c), a) ∈ σ such that y ⊆ x, and either a = • or a = c , where y + (c , •) ⊆ x. If y ⊆ ∆, then in the first case (c, •) ∈ fun * σ)( ∆), and in the second case we may set y n+1 = y and c n+1 = c , satisfying the induction hypothesis.
If y ⊆ ∆, let P be a proof of ((y, c), a) in σ, and let ((z, d), a) be the least element of this proof such that z ⊆ ∆. Then ((z, d), a) is immediately preceded in P by an event ((z , d), d ) such that z ⊆ ∆ and z = z + (d , V ), for some set of values V . So there exists v ∈ V such that (d , v) ∈ ∆ and hence (d , d ), d) ∈ σ and so we may set y n+1 = z and c n+1 = d , satisfying the induction hypothesis.
We can now prove that fun * (σ) is ≤ S -continuous.
Proof. By Lemma 6.14, if (c, •) ∈ x∈∆ fun * (σ)(x), either (c, •) ∈ fun * σ)( ∆) or there is an infinite set of events {((y i , c), c i ) ∈ σ | i ∈ ω} such that each y i ⊆ ∆ and c i = c j implies i = j. In the former case we are done, so assume the latter and consider the tree of proofs of these events in σ. If this is infinite branching, then either there is some ((y i , c), c i ) which enables infinitely many cells (y j , c) -in which case c i contains infinitely many values in ∆ and so (c i , •) ∈ ∆ and (c, •) ∈ f ( ∆) as required -or else there is some y ⊆ ∆ such that y i = y for infinitely many i -i.e. (y, c) contains infinitely many c i in σ, and so ((y, c), •) ∈ σ and (c, •) ∈ f ( ∆) as required.
If the tree of proofs is finite-branching then by König's lemma it contains an infinite branch -i.e. there exists z ⊆ ∆ such that (z, c) has an infinite proof in σ, and by the infinite livelock condition ((z, c), •) ∈ σ and (c, •) ∈ f ( ∆) as required.
The inverse to fun * is the restriction of strat to stably continuous functions on finitely branching states -i.e. for stably continuous f : Proof that this satisfies the safety condition and furnishes an inverse to fun * follows the proof of Lemma 6.8. We need to show in addition that if f : D * (A) → D * (B) is stably continuous then strat * (f ) is finite-branching.
Otherwise, the proof contains an infinite substring ((x α+i , c α+i ), a α+i ) i<ω , where c α+i = c α and a α+i is a cell in A enabled by x α+i for each i < ω. Let The y i form a stable ω-chain such that (c α , •) ∈ f (y i ) for each i ∈ ω and hence by stable continuity (c, and so ((x, c), •) ∈ strat(f ) as required.
Proof. Suppose (x, c) is an enabled cell in A ⇒ * B such that there are infinitely many values a such that ((x, c), a) ∈ strat * (f ). If there are infinitely many values of B such that ((x, c), v) ∈ strat * (f ) then (c, •) ∈ f (x) by the finite branching condition on f (x), and so ((x, c), •) ∈ strat * (f ) as required.
Otherwise there are infinitely many cells c of A such that ((x, c), c ) ∈ strat(f ). Let Y be the set of states y which extend x by filling all but a finite number of its accessible cells with • -i.e. such that y ≤ S x and A(x) − F (y) is finite. Y is ≤ S -directed -if y, y ∈ Y then y ∩ y ∈ Y -and has supremum x.
Finally, we observe that A ⇒ * B is also the internal hom in the category of ocds and continuously stable functions, based on the following observations:  ((x , c), a) ∈ σ ∨ ∃c .(c , •) ∈ x ∧ ((x , c), c ) ∈ σ} for which stability follows as for fun, and continuity follows from proposition 6.18. Its inverse, is similarly defined by restriction of strat to events ((x, c), a) such that x is finitely generated, which will have a finite proof in strat * (f ) provided (c, a) has a finite proof in f (x). The composition of strat * with fun * in each direction is the identity on all compact elements, and is therefore the identity on all elements. Finally, we may show (by induction on proof-length) that: Lemma 6.19. ((x, c), a) ∈ E(A ⇒ * B) has a finite proof if and only if x ∈ K(A) and c has a finite proof in B.
Hence the ocds A ⇒ * B and A ⇒ * B are isomorphic in OC E .

Full Abstraction and Universality
We now return to the may and must testing semantics of bounded and unbounded Λ + (N ) . In each case we now have an intensional representation of each program of pointed type t : P as a state ( [t]) of an ordered concrete data structure ([P ]), since the biorder denoted by a type of the form N → T is isomorphic to [[T ]] ω . (Formally, Λ + (N ) may be interpreted by applying the families construction (biproduct completion) to our categories of ordered concrete data struction -see [Abr98].) First, our aim is to show that our models of bounded Λ + (N ) are fully abstract with respect to may testing and must testing, respectively, and characterize their Λ + (N ) -definable elements (a form of universality result). Full abstraction for models in which terms denote continuous morphisms may be established by proving that each type-object has a basis of elements which are definable (i.e. denotations of terms). In the case of bounded Λ + (N ) one may derive such a basis from the fact -already remarked (Proposition 2.10) -that all elements of finite types are definable in Λ + , since every pointed Λ + (N ) type T is a limit of a chain of finite products of such types (see [Lai09] or [McC96] for details of such a proof). However, this rather cheap reduction to universality for Λ + is not very informative about the definability properties of infinite elements -such as the difference between the may and must testing models. We shall prove a stronger result extending Proposition 2.10 -that the type U N → o → o is universal in our models of bounded Λ + (N ) -i.e. T ¢ U for all pointed Λ + (N ) types T . In particular U → U ¢ U -i.e. U denotes a reflexive object in the categories of concrete data structures/complete biorders and continuous functions.
Remark 7.1. This is similar to results for the sequential algorithms model of deterministic SPCF [Lai07a], which are refined in [Lai07b] to show that all retractions are definable in a linearly typed version of SPCF without nesting of function calls (and fixed point combinators replaced with iteration). The retractions given here can be defined within a similar typing system for non-deterministic SPCF Remark 7.2. We have already observed that [[U ]] ∼ = D * (U), where U is the ocds with a countable set of initial cells {c.i | i ∈ ω} which may each be filled by a single value, and that this is a universal object in the categories of countable ocds and stably continuous/continuously stable functions. Thus every pointed type of Λ + (N ) denotes a retract of U. It remains to show that each such a retraction is definable.
The proofs that there are Λ + (N ) -definable retractions from pointed Λ + (N ) types into U are similar for the may testing and must testing models -we give the former (slightly simpler) case first.
Observe that N → T denotes a N-indexed product (or infinite list) of copies of [[T ]]. Thus we may define the following list operations on terms: Given t 1 : T and t 2 : N → T , let t 1 ::t 2 : N → T λu.(Eq(u, 0) t 1 ) (pred(u) t 2 ) and given t : N → T , define hd So in particular, N → U ¢ U . Recall that the denotation of the type nat is isomorphic to the biorder P(N) ↑ .
The key retraction reducing the order of types (depth to which the arrow is nested on the left) is based on the observation that we may represent a continuously stable function f  Proof. If g(i) = ⊥ S = E for some sequentiality index i ∈ si(f ) then f (g) = ⊥ S = g(i) f (g[I] i ) = i∈si(f ) (π i (x) f (x[I] i )). So suppose that g(i) ∈ {⊥ E , I} for all i ∈ si(f ). If g(i) = ⊥ E then g(i) f (g[I] i ) = ⊥ E and if g(i) = I then g(i) f (g[I] i ) = I f (g) = f (g). So if g(i) = ⊥ E for every sequentiality index i, then i∈si(f ) (g(i) f (x[I] i )) = ⊥ E = f (g) by 7.2. Unbounded Nondeterminism. The situation in the non-continuous models is different. Although finitary elements of these models are still definable, in the absence of continuity this does not imply full abstraction, which in fact does not hold (contradicting [Lai06]). We focus on the model of must testing (since this requires non-continuity). We have shown by example that some events with infinite proofs are required to distinguish inequivalent terms in this model, but hinted that there are many more which may distinguish denotations of terms which cannot be separated operationally -we now give an example of a stable and monotone function which can be used to distinguish two must equivalent terms, establishing that the model is not fully abstract. Consider the function h : is finite but non-empty, • h(x) = I, otherwise (si(x) is infinite). This is monotone and stable, since si(x) ⊆ si(y) and si(y) finite implies si(x) finite, and We may prove that h is not definable in Λ + (N ) by showing that all definable functions have the following property: Definition 7.14. A monotone stable function of extensionally complete biorders f : D → E is weakly continuous if for any E -directed set X, f ( X) ≤ S f (X).
The function h is not weakly continuous: let X be the E -directed set { i∈J π i : Σ ω → Σ | J ⊆ fin N}. Then si( X) = N and so h( X) = I ≤ S ⊥ E = x∈X h(x). However, all Λ + (N ) terms denote weakly continuous functions. We prove this using a logical relationfor each type T Proposition 7.15. Every term of Λ + (N ) denotes a hereditarily weakly continuous function.

Proof.
A straightforward structural induction. For the fixed point operator, we note that if F is a ≤ S -directed set of weakly continuous functions from D to (bicomplete) E, then F is weakly continuous, since for any E -directed set X, Hence we have established that: Proposition 7.16. There is no term t : nat → o → o which denotes the function h.
Moreover, the weak continuity of definable functions may be used to prove that the model is not fully abstract: we now give a Λ + (N ) -definable test which distinguishes h from all weakly continuous functions, and therefore from all definable elements of the model. Define ? : N → nat λu.λf.? N λv.(leq(u, v) (f v)) H so that ?(n) returns a bounded choice over all values less than n. Clearly, ?(n) n∈ω is a E -chain with n∈ω ?(n) =? N .