Randomness extraction and asymptotic Hamming distance

We obtain a non-implication result in the Medvedev degrees by studying sequences that are close to Martin-L\"of random in asymptotic Hamming distance. Our result is that the class of stochastically bi-immune sets is not Medvedev reducible to the class of sets having complex packing dimension 1.


Introduction
We are interested in the extent to which an infinite binary sequence X, or equivalently a set X ⊆ ω, that is algorithmically random (Martin-Löf random) remains useful as a randomness source after modifying some of the bits.Usefulness here means that some algorithm (extractor) can produce a Martin-Löf random sequence from the result Y of modifying X.For further motivation see Subsection 1.2 and Section 3.
A set that lies within a small Hamming distance of a random set may be viewed as produced by an adaptive adversary corrupting or fixing some bits after looking at the original random set.Similar problems in the finite setting have been studied going back to Ben-Or and Linial [1].
If A is a finite set and σ, τ ∈ {0, 1} A , then the Hamming distance d(σ, τ ) is given by Let the collection of all infinite computable subsets of ω be denoted by C. Let p : ω → ω.For X, Y ∈ 2 ω and N ⊆ ω we define a notion of proximity, or similarity, by We will study the effective dimension of sequences that are ∼ p,N to certain algorithmically random reals for suitably slow-growing functions p.
We use the following notation for a kind of neighborhood around X.
Moreover, for a collection A of subsets of ω, Turing functionals as random variables.Since a random variable must be defined for all elements of the sample space, we consider a Turing functional Φ to be a map into Setting the domain of Φ to also be Ω allows for composing maps.Let Thus Λ : 2 ω → 2 ω is the identity Turing functional.We define a probability measure λ on Ω called Lebesgue (fair-coin) measure, whose σ-algebra of λ-measurable sets is F = {S ⊆ Ω : S ∩ 2 ω is Lebesgue measurable}, by letting λ(S) equal the fair-coin measure of S ∩ 2 ω .Thus λ(2 ω ) = 1 and λ(2 <ω ) = 0, that is, λ is concentrated on the functions that are actually total.
The distribution of Φ is the measure S → λ{X : Φ X ∈ S}, defined on F. Thus the distribution of Λ is λ.
If X ∈ 2 ω then X is called a real, a set, or a sequence depending on context.If I ⊆ ω then X ↾ I denotes X, viewed as a function, restricted to the set I. We denote the cardinality of a finite set A by |A|.Regarding X, Y as subsets of ω and letting + denote sum mod two, note that (X + Y ) ∩ n = {k < n : X(k) = Y (k)} and generally for a set For an introduction to algorithmic randomness the reader may consult the recent books by Nies [10] and Downey and Hirschfeldt [4].Let MLR denote the set of Martin-Löf random elements of 2 ω .For a binary relation R we use a set-theoretic notation for image, Let the use ϕ X (n) be the largest number used in the computation of Φ X (n).We write Φ X (n) ↓ @s if Φ X (n) halts by stage s, with use at most s; if this statement is false, we write Φ X (n) ↑ @s.We may assume that the running time of a Turing reduction is the same as the use, because any X-computable upper bound on the use is a reasonable notion of use.
For a set A ⊆ 2 ω , let As a kind of effective big-O notation, p n = ω * (q n ) means lim * n→∞ q n /p n = 0, i.e., q n /p n goes to zero effectively.Central Limit Theorem.Let N be the cumulative distribution function for a standard normal random variable; so Let P denote fair-coin probability on Ω.We may write We will make use of the following quantitative version of the central limit theorem.
Theorem 1.3 (Berry-Esséen1 ).Let {X n } n≥1 be independent and identically distributed real-valued random variables with the expectations E(X n ) = 0, E(X 2 n ) = σ 2 , and E(|X n | 3 ) = ρ < ∞.Then there is a constant d (with .41≤ d ≤ .71)such that for all x and n, We are mostly interested in the case 1.1.New Medvedev degrees.Let ≤ s denote Medvedev (strong) reducibility and let ≤ w denote Muchnik (weak) reducibility.A recent survey of the theory behind these reducibilities is Hinman [8].
Definition 1.4 (see, e.g., [9] The class of stochastically dominated sequences is denoted SD = SD p .If ω \ X ∈ SD p then we write X ∈ SD p and say that X is stochastically dominating.
Let IM denote the set of immune sets, CIM the set of co-immune sets, and W3R the set of weakly 3-random sets.Let K denote prefix-free Kolmogorov complexity.Definition 1.7 (see, e.g., [4,Ch. 13

]). The effective Hausdorff dimension
Proof.The inequality dim H (A) ≤ dim cp (A) uses the fact that each cofinite set N ⊆ ω is in C. The inequality dim cp (A) ≤ dim p (A) uses the fact that each N ∈ C is an infinite subset of ω.
By examining the complex packing dimension of reals that are ∼ p,N to a Martin-Löf random real for p growing more slowly than n/(log n), we will derive our main result, which states the existence, for each Turing reduction Φ, of a set Y of complex packing dimension 1 for which Φ Y is not stochastically bi-immune.
1.2.Relation of our results to other recent results.Jockusch and Lewis [9] prove that the class of bi-immune sets is Medvedev reducible to the class of almost diagonally noncomputable functions DNC * , i.e., functions f such that f (x) = ϕ x (x) for at most finitely many x.Downey, Greenberg, Jockusch, and Milans [3] show that DNC 3 (the class of DNC functions taking values in {0, 1, 2}) and hence also its superset DNC * , is not Medvedev above the class of Kurtz random sets.We do not know whether the class of stochastically bi-immune sets is Medvedev reducible to the class of DNC * functions.We show in Theorem 4.3 below that from a set of complex packing dimension 1 one cannot uniformly compute a stochastically bi-immune set; on the other hand, to compute a DNC * function from a set of complex packing dimension 1 one would apparently also need to know the witnessing set N ∈ C. Definition 1.9 (see, e.g., [10,Def. 7.6.4]).A sequence X ∈ 2 ω is Mises-Wald-Church (MWC) stochastic if no partial computable monotonic selection rule can select a biased subsequence of X, i.e., a subsequence where the relative frequencies of 0s and 1s do not converge to 1/2.Definition 1.10.A sequence X ∈ 2 ω is BI 2 (bi-immune for sets of size two) if there is no computable collection of disjoint finite sets of size 2 on which the set omits a certain pattern such as 01.More precisely, X is BI 2 if for each computable disjoint collection {T n : n ∈ ω} where each T n has cardinality two, say T n = {s n , t n } where s n < t n , and each P ⊆ {0, 1}, there is an n such X(s n ) = P (0) and X(t n ) = P (1).
Each von Mises-Wald-Church stochastic (MWC-stochastic) set is stochastically biimmune.Our main theorem implies that a set of complex packing dimension 1 does not necessarily uniformly compute a MWC-stochastic set.This consequence is not really new with the present paper, however, because the fact that DNC 3 is not Medvedev above BI 2 is implicit in Downey, Greenberg, Jockusch, and Milans [3] as pointed out to us by Joe Miller.The situation is diagrammatically illustrated in Figure 1, with notation defined in Figures 2 and 3.In the future we could hope to replace complex packing dimension by effective Hausdorff dimension in Theorem 4. complex packing dimension 1 1.7 Figure 3: Abbreviations used in Figure 1.

Hamming space
The Hamming distance between a point and a set of points is defined by d(y, A) := min a∈A d(y, a).The r-neighborhood of a set In particular, Γ r ({c}) = {y ∈ {0, 1} n : d(y, c) ≤ r}, and Γ r (A) = a∈A Γ r ({a}).
A Hamming-sphere2 with center c ∈ {0, 1} n is a set S ⊆ {0, 1} n such that for some k, Theorem 2.1 (Harper [7]; see also Frankl and Füredi [6]).For each n, r ≥ 1 and each set Moreover, for each m 0 ∈ ω and computable q ∈ (p, 1) there is a modulus of effective convergence in (2.3) that works for all sets {E m } m∈ω such that for all m ≥ m 0 , P(E m ) ≤ q.
Proof.Let X ∈ 2 ω be a random variable with X = d Λ, and Let and let (2.5) 3 Indeed, let Yi = Xi − E(Xi) where E(Xi) = 1 2 is the expected value of Xi, so E(Yi) = 0.By the Berry-Esséen Theorem 1.3, for all x where ρ = 1/8 = E(|Yi| 3 ), and σ = 1/2 is the standard deviation of Xi (and Yi).Let m 0 be such that for all m ≥ m 0 , Since by assumption lim * n→∞ χ(n)/ √ n = ∞, we have that b m is the sum of a term that goes effectively to −∞, and a term that after m 0 never goes above N −1 (p) + 1 again.Thus It is this rate of convergence that is transformed in the rest of the proof.Now Let H be the complement of H.If the Hamming sphere H is centered at c ∈ {0, 1} n then clearly H is a Hamming sphere centered at c, where c(k we have H ⊂ Γ r+1 ({c}).So we have: Since we showed that lim * m→∞ f m (b m ) = 0, and since by assumption lim

Turing reductions that preserve randomness
The way we will obtain our main result Theorem 4.3 is by proving essentially that for any "randomness extractor" Turing reduction, and any random input oracle, a small number of changes to the oracle will cause the extractor to fail to produce a random output.This would be much easier if we restricted attention to Turing reductions having disjoint uses on distinct inputs, since we would be working with independent random variables.Indeed, one can give an easy proof in that case, which we do not include here.The main technical achievement of the present paper is to be able to work with overlapping use sets; key in that respect is Lemma 3.3 below.The number of changes to the random oracle that we need to make is small enough that the modified oracle has complex packing dimension 1.We were not able to set up the construction so as to guarantee effective Hausdorff dimension 1 (or even greater than 0); this may be an avenue for future work.
Proof.By Fubini's theorem, For a real X and a string σ of length n, Thinking of σ and X as functions we may write and thinking in terms of concatenation we may write σ ⌢ X = σ X. Lemma 3.3.Let Φ be a Turing reduction such that and let Φ X σ = Φ σցX .Then for any finite set Proof.First note that for all σ ∈ 2 <ω , λ(Φ −1 σ SD p = 1 as well.Suppose otherwise, and fix ε, i 0 and Σ such that By density of the rationals in the reals we may assume ε is rational and hence computable.Since there are infinitely many i but only finitely many σ, it follows that there is some σ such that and in fact lim sup |{k < n : P(Φ σ (k) = 1) > p + ε}| /n > 0. Fix such a σ and let Ψ = Φ σ .Let {ℓ n } n∈ω be infinitely many values of k in (3.4) listed in increasing order; note that L = {ℓ n } n∈ω may be chosen as a computable sequence.
For an as yet unspecified subsequence We obtain then also projections . By (3.4) we have for all n ∈ ω, The fraction of events E n that occur in N = {0, . . ., N − 1} for X is denoted Thus there is an M and a K (using that C is countable) such that Let Ω 1 be the unit interval  Since Φ is total for almost all oracles, it is clear that i ′ is a computable function f (k, n) of ε = 1/k and n.Let g : ω → ω be the computable function with lim n→∞ g(n) = ∞ given by g(s) = 2s.Let n 0 = 0 and i 0 = 0. Assuming s ≥ 0 and n s and i s have been defined, let and let n s+1 be large enough that Let X ∈ MLR.We aim to define Y ∼ p X such that Φ Y ∈ MLR.We will in fact make Y ≤ T X, so we define a reduction Ξ and let Y = Ξ X .Since we are defining Y by modifying bits of X, the use of Ξ will be the identity function: ξ X (n) = n.
Since n 0 = 0, Y ↾ n 0 is the empty string.Suppose s ≥ 0 and Y ↾ns has already been defined.The set of "good" strings now is Define the "cost" of τ to be the additional Hamming distance to X, i.e., Case 1: G = ∅.Then let Y ↾n s+1 be any τ 0 ∈ G of length n s+1 and of minimal cost, i.e., such that d(τ 0 ) = min{d(τ ) | τ ∈ G}.That is, let Case 2: Otherwise.Then make no further changes to X up to length n s+1 , i.e., let This completes the definition of Ξ and hence of Y .It remains to show that Φ Y ∈ MLR.For any string σ of length n s let Since (4.2) and (4.4) hold for all strings of length n s , in particular they hold for σ = Ξ X ↾ n s , so we can apply Lemma 2.2 and there is h(s) with lim * s→∞ h(s) = 0 and P(U X↾ns s ) ≤ h(s) that only depends on an upper bound for an s 0 such that for all s ≥ s 0 , P(E s+1 ) ≤ q (where p < q < 1 and q is just some fixed computable number).Since by (4.5) such an upper bound can be given that works for all X, actually h(s) may be chosen to not depend on X.
To find the probability of V s we note that for each of the 2 ns possible beginnings of Z, there are at most (h(s) • 2 n s+1 −ns ) continuations of Z on [n s , n s+1 ) that make Z ∈ V s ; so we compute ≤ 2 ns (h(s) • 2 n s+1 −ns )2 −n s+1 = h(s) so since lim * s→∞ h(s) = 0, {V s } s∈ω is a Kurtz randomness test.Let {m s } s∈ω be a computable sequence such that s≥t h(m s ) ≤ 2 −t .Let W t = s≥t V ms .Then P(W t ) ≤ 2 −t and W t is uniformly Σ 0 1 and hence it is a Martin-Löf randomness test.Since X ∈ MLR, X ∈ W t for some t and hence X ∈ V ms for all but finitely many s.So Φ Y (m s ) = 0 for all but finitely many s, hence Φ Y ∈ CIM.By construction, we have Proof.Suppose there are at most p(n) many bits changed to go from X ↾ n to Y ↾ n, in positions a 1 , . . ., a p(n) .(In case there are fewer than p(n) changed bits, we can repeat a i representing the bit 0 which we may assume is changed.)Let (Y ↾ n) * be a shortest description of Y ↾ n.From the code we can effectively recover X ↾ n.Thus Theorem 4.3.For each Turing reduction procedure Φ there is a set Y with dim cp (Y ) = 1 such that Φ Y is not stochastically bi-immune.
Proof.Let p(n) = n 2/3 , so that p(n) = o(n/ log n) and p(n) = ω * ( √ n).By the proof of Theorem 4.1 and since the sequence of numbers n s is computable, for each weakly 3-random set X there is a set Y ∼ p,N X (for some N ∈ C) such that Φ Y is not both co-immune and in SD 1/2 , in particular Φ Y ∈ SBI.By Lemma 4.2, each such Y has complex packing dimension 1. DNC DNC 3 u u u 5 u 5 u 5 u 5 u 5 u 5 u 5 u 5 u 5 u 5 u 5 u 5

lim * m→∞ N
(b m ) = 0. Hence by (2.5), lim * m→∞ f m (b m ) = 0. Let us write B t (X) := B Im t (X ↾ I m ), considering X ↾ I m as a string of length n.By Harper's Theorem 2.1, we have a Hamming sphere H with |H| = |¬E m | and Γ
Definition 1.1 (Effective convergence).Let {a n } n∈ω be a sequence of real numbers.• {a n } n∈ω converges to ∞ effectively if there is a computable function N such that for all k and all n ≥ N (k), a n ≥ k. • {a n } n∈ω converges to 0 effectively if the sequence {a −1 n } n∈ω converges to ∞ effectively.Definition 1.2.For a sequence of real numbers {a n } n∈ω , lim * n→∞ a n is the real number to which a n converges effectively, if any; and is undefined if no such number exists.
where = * denotes almost equality for all but finitely many inputs.It is easy to see that Interior p,N (MLR) = ∅ whenever N ⊆ ω and p is unbounded. 3.