Computability, Unsolvability, Randomness

Stephen G. Simpson Department of Mathematics Pennsylvania State University

http://www.math.psu.edu/simpson/

February 5, 2009

Please send corrections to the author at ⟨simpson@math.psu.edu⟩.

Preface

This book originated as a set of lecture notes for a junior-senior-level course which I taught at the Pennsylvania State University in Fall Semester (August 27 through December 14), 2007. The note-taker was my teaching assistant in the course, Jonas Kibelbek, a Mathematics Ph.D. student at Penn State. The students in the course were outstanding undergraduate mathematics majors from colleges and universities around the United States. These students were partipating in our Mathematics Advanced Study Semester program, also known as MASS. The MASS program is sponsored by the United States National Science Foundation.

The purpose of my Fall 2007 course and of this book is twofold. First, I exposit Turing’s 1936 theory of computability and unsolvability, as subsequently developed by Kleene and Post. This theory is of the essence in theoretical computer science and in the study of unsolvable mathematical problems. Second, I provide an introductory account of a research area which is currently very active: algorithmic randomness and Kolmogorov complexity.

Contents

Title 1

Preface 2

1 Computability 6 1.1 Computable functions . . . . . . . . . . . . . . . . . . . . . . . . 6 1.2 Composing computable functions . . . . . . . . . . . . . . . . . . 9 1.3 Computable predicates . . . . . . . . . . . . . . . . . . . . . . . . 12 1.4 Primitive recursion . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.5 Prime power coding . . . . . . . . . . . . . . . . . . . . . . . . . 20 1.6 Computable real numbers . . . . . . . . . . . . . . . . . . . . . . 22

2 Partial recursive functions 24 2.1 Partial recursive functions . . . . . . . . . . . . . . . . . . . . . . 24 2.2 Gödel numbers and indices . . . . . . . . . . . . . . . . . . . . . 26 2.3 The Enumeration Theorem . . . . . . . . . . . . . . . . . . . . . 28 2.4 The Parametrization Theorem . . . . . . . . . . . . . . . . . . . 30 2.5 Universal partial recursive functions . . . . . . . . . . . . . . . . 32

3 Unsolvability 33 3.1 Noncomputable sets and functions . . . . . . . . . . . . . . . . . 33 3.2 An unsolvable decision problem . . . . . . . . . . . . . . . . . . . 34 3.3 Almost all problems are unsolvable . . . . . . . . . . . . . . . . . 35 3.4 Unsolvable problems in core mathematics . . . . . . . . . . . . . 36 3.5 A remark on Hilbert’s 10th Problem . . . . . . . . . . . . . . . . 39 3.6 Many-one reducibility . . . . . . . . . . . . . . . . . . . . . . . . 40

4 The arithmetical hierarchy 43 4.1 Definition of the arithmetical hierarchy . . . . . . . . . . . . . . . 43 4.2 Closure properties . . . . . . . . . . . . . . . . . . . . . . . . . . 44 4.3 Distinguishing levels of the hierarchy . . . . . . . . . . . . . . . . 46 4.4 Many-one completeness . . . . . . . . . . . . . . . . . . . . . . . 48 4.5 Recursively enumerable sets . . . . . . . . . . . . . . . . . . . . . 49 4.6 Homework #3, due September 17, 2007 . . . . . . . . . . . . . . 51 4.7 Recursively enumerable sets . . . . . . . . . . . . . . . . . . . . . 51 4.8 Parametrization and uniformity . . . . . . . . . . . . . . . . . . . 53

5 Oracles and relativization 56 5.1 Oracle computation . . . . . . . . . . . . . . . . . . . . . . . . . 56 5.2 Relativization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 5.3 Turing degrees . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 5.4 The Turing jump operator . . . . . . . . . . . . . . . . . . . . . . 60 5.5 Finite approximation . . . . . . . . . . . . . . . . . . . . . . . . . 62 5.6 Structure of the Turing degrees . . . . . . . . . . . . . . . . . . . 66 5.7 Homework #5, due October 1, 2007 . . . . . . . . . . . . . . . . 68 5.8 Comments on Homework #4, continued . . . . . . . . . . . . . . 70 5.9 Review for the upcoming Midterm Exam . . . . . . . . . . . . . 73 5.10 Solutions of some homework and midterm problems . . . . . . . 79

6 Kolmogorov complexity 80 6.1 Kolmogorov complexity . . . . . . . . . . . . . . . . . . . . . . . 80 6.2 Prefix-free complexity . . . . . . . . . . . . . . . . . . . . . . . . 83 6.3 The Kraft/Chaitin Theorem . . . . . . . . . . . . . . . . . . . . . 85

7 The Cantor space 90 7.1 Partial recursive functionals, etc. . . . . . . . . . . . . . . . . . . 90 7.2 Homework #8, due October 22, 2007 . . . . . . . . . . . . . . . . 92 7.3 Σ_{1}^{0} and Π_{1}^{0} sets in 2^{ℕ} and ℕ^{ℕ} . . . . . . . . . . . . . . . . . . . . 94 7.4 Compactness of 2^{ℕ} . . . . . . . . . . . . . . . . . . . . . . . . . . 99 7.5 Σ_{1}^{0} and Π_{1}^{0} predicates in 2^{ℕ} . . . . . . . . . . . . . . . . . . . . . 101

8 Randomness 104 8.1 Foundations of mathematics . . . . . . . . . . . . . . . . . . . . . 104 8.2 Definitions of randomness . . . . . . . . . . . . . . . . . . . . . . 105 8.3 Homework #9, due October 29, 2007 . . . . . . . . . . . . . . . . 108 8.4 Properties of Martin-Löf randomness . . . . . . . . . . . . . . . . 109 8.5 Comments on Homework #8 . . . . . . . . . . . . . . . . . . . . 115 8.6 Homework #10, due November 5, 2007 . . . . . . . . . . . . . . . 117 8.7 Initial segment complexity . . . . . . . . . . . . . . . . . . . . . . 119 8.8 Solutions for Homework #9 . . . . . . . . . . . . . . . . . . . . . 121 8.9 Homework #11, due November 12, 2007 . . . . . . . . . . . . . . 123

9 Some advanced topics 125 9.1 Turing degrees of random sequences . . . . . . . . . . . . . . . . 125 9.2 The Low Basis Theorem . . . . . . . . . . . . . . . . . . . . . . . 126 9.3 Randomness relative to an oracle . . . . . . . . . . . . . . . . . . 129 9.4 Comments on Homework #11 . . . . . . . . . . . . . . . . . . . . 130 9.5 Homework #12, due November 26, 2007 . . . . . . . . . . . . . . 132 9.6 The Kučera/Gács Theorem . . . . . . . . . . . . . . . . . . . . . 132 9.7 Some solutions for Homework #10 . . . . . . . . . . . . . . . . . 136 9.8 Some solutions for Homework #11 . . . . . . . . . . . . . . . . . 138 9.9 Comments on Homework #12 . . . . . . . . . . . . . . . . . . . . 141 9.10 Homework #13, due December 3, 2007 . . . . . . . . . . . . . . . 141 9.11 LR- and LK-reducibility . . . . . . . . . . . . . . . . . . . . . . . 142

10 Solutions to all of the exercises 151

Chapter 1

Computability

In this chapter we study Turing’s concept of what it means for a function to be computable. Computable functions are also known as recursive functions.

1.1 Computable functions

We denote the natural numbers by ℕ = {0, 1, 2, …}. We shall deal with 1-place number-theoretic functions f : ℕ → ℕ.

Example 1.1.1. An example of a 1-place number-theoretic function is f(x) = 2^{x}. Thus f(0) = 1, f(1) = 2, f(2) = 4, f(3) = 8, etc. Note that the variable x ranges over ℕ, i.e., x takes on integer values only. Later we shall see that the number-theoretic function f(x) = 2^{x} is, in a precisely defined sense, “computable.”

There are a great many number-theoretic functions, and many of them behave in a very unpredictable manner. We wish to define a subclass consisting of the number-theoretic functions which are “well behaved” or “predictable.” We shall call these functions “computable.” In order to define this concept rigorously, we now introduce register machines and register machine programs.

Definition 1.1.2. A register machine consists of a finite set of registers, denoted R_{1}, …, R_{s}. Each register R_{i} is a container which can hold an arbitrary natural number z_{i} ∈ ℕ. Here 1 ≤ i ≤ s. We think of each register R_{i} as a box containing z_{i} marbles. The basic register machine operations are: adding a marble to a box, and removing a marble from a box. The register R_{i} is said to be empty if z_{i} = 0, i.e., there are no marbles in R_{i}. A register machine program is a finite flow diagram consisting of the following types of instructions.

1. The increment instruction, denoted R_{i}^{+}. This instruction replaces z_{i} by z_{i} + 1 (i.e., adds a marble to the box R_{i}) and then goes to the next instruction, indicated by an arrow.

2. The decrement instruction, denoted R_{i}^{−}. This is a branching instruction. If z_{i} > 0, it replaces z_{i} by z_{i} − 1 (i.e., removes a marble from the box R_{i}) and then goes to the next instruction, indicated by an unlabeled arrow. If z_{i} = 0, it leaves R_{i} empty and goes to the next instruction, indicated by an arrow labeled e.

3. Start and stop instructions. Each program has exactly one start instruction. A stop instruction indicates that we are to stop, i.e., no more instructions are to be executed. We sometimes use the word “halt” instead of “stop.”

Example 1.1.3. Consider the following register machine program.

[figure: start; R_{1}^{−}; e; stop; R_{2}^{+}; R_{2}^{+}]

Note that this program, if started with x in R_{1} and y in R_{2}, will eventually halt (after a finite number of steps) with 0 in R_{1} and 2x + y in R_{2}.

Definition 1.1.4 (computable functions). A 1-place number-theoretic function f(x) is said to be computable if there exists a program, call it P, with the following property. For any x ∈ ℕ, if we start P with x in R_{1} and all other registers empty (in other words, we start with 0 in R_{2}, …, R_{s}), then P will eventually halt with f(x) in R_{2}.

Example 1.1.5. The program of Example 1.1.3 computes the function f(x) = 2x. Thus, f(x) = 2x is a computable function.

Exercise 1.1.6. Exhibit a register machine program showing that the function f(x) = 2^{x} is computable.

Solution. The following program, if started with x in R_{1} and 0 in R_{2} and R_{3}, will eventually halt with 2^{x} in R_{2}. This shows that the function 2^{x} is computable.

[figure: start; R_{2}^{+}; R_{1}^{−}; e; R_{2}^{−}; e; R_{3}^{−}; e; stop; R_{3}^{+}; R_{2}^{+}; R_{3}^{+}]

We now generalize Definition 1.1.4 to functions of several variables.

Definition 1.1.7 (computable functions). A k-place number-theoretic function, f : ℕ^{k} → ℕ, is said to be computable if there exists a program, call it P, with the following property. For any x_{1}, …, x_{k} ∈ ℕ^{k}, if we start P with x_{1} in R_{1}, …, x_{k} in R_{k}, and all other registers empty (i.e., 0 in R_{k+1}, …, R_{s}), then P will eventually halt with f(x_{1}, …, x_{k}) in R_{k+1}.

Example 1.1.8. The following program, if started with x in R_{1}, y in R_{2}, and 0 in R_{3}, will eventually halt with x + y in R_{3}.

[figure: start; R_{1}^{−}; e; R_{2}^{−}; e; stop; R_{3}^{+}; R_{3}^{+}]

Thus we see that the 2-place number-theoretic function f(x, y) = x + y is computable.

The following piece of notation will be useful.

Notation 1.1.9. We write P(x_{1}, …, x_{k}) to denote the run of the program P when it is started with x_{1} in R_{1}, …, x_{k} in R_{k}, and all other registers empty. In this context R_{1}, …, R_{k} are called input registers, R_{k+1} is called the output register, and R_{k+2}, …, R_{s} are called auxiliary registers.

Example 1.1.10. Exhibit a register machine program showing that the 2-place number-theoretic function f(x, y) = xy is computable.

Solution. The following program, if started with x in R_{1}, y in R_{2}, and 0 in R_{3} and R_{4}, will eventually halt with xy in R_{3}.

[figure: start; R_{1}^{−}; e; R_{2}^{−}; e; R_{4}^{−}; e; stop; R_{3}^{+}; R_{2}^{+}; R_{4}^{+}]

Note that R_{4} serves as an auxiliary register.

Exercise 1.1.11. Exhibit a register machine program showing that the 2-place number-theoretic function

x · − y = \{ \begin{matrix}x − y & if x > y , \\ 0 & if x ≤ y \end{matrix}

is computable.

Solution. The following program, if started with x in R_{1}, y in R_{2}, and 0 in R_{3}, will eventually halt with x · − y in R_{3}.

[figure: start; R_{1}^{−}; e; R_{2}^{−}; e; R_{3}^{+}; R_{1}^{−}; e; stop; stop; R_{3}^{+}]

Exercise 1.1.12. Exhibit a register machine program which computes the exponential function, exp(x, y) = x^{y}. Remember that the variables x and y range over ℕ, the set of natural numbers. Note that x^{0} = 1 for all x, even for x = 0.

In general, given a program P and inputs x_{1}, …, x_{k}, the run P(x_{1}, …, x_{k}) may or may not eventually halt, depending on x_{1}, …, x_{k}. Later we shall prove the following important theorem.

Theorem 1.1.13 (Turing, 1936). We can construct a particular program, call it P, with the following property. The 1-place number-theoretic function

g(x) = \{ \begin{matrix}1 & if P(x) eventually halts, \\ 0 & if P(x) never halts, \end{matrix}

is not computable. (Here we are using Notation 1.1.9.)

We shall describe this situation by saying that the “Halting Problem” for the program P is “unsolvable.”

1.2 Composing computable functions

The purpose of this chapter is to convince the reader that a great many familiar number-theoretic functions and predicates are computable. We are going to develop some simple yet powerful tools for proving that particular functions are computable, without actually having to write programs which compute them.

We begin with the following simple theorem.

Theorem 1.2.1 (composition). Given 1-place number-theoretic functions f and g, there is a unique 1-place number-theoretic function h = g ∘ f defined by h(x) = g(f(x)) for all x. Moreover, if f and g are computable, then h is computable.

Proof. The existence and uniqueness of h = g ∘ f are obvious.

Suppose now that f and g are computable. Let F and G be programs which compute f and g respectively. We may safely assume that F and G use the same set of registers R_{1}, …, R_{s} where s ≥ 2. Let H be the program

[figure: start; R_{1}^{+}; stop; F; R_{1}^{−}; e; R_{2}^{−}; R_{3}^{−}; e; ⋯; R_{s}^{−}; e; G]

It is easily verified that H computes h.

Below we shall generalize the above theorem from 1-place functions to kplace functions. But first, in order to simplify constructions of this kind, let us make the following definition.

Definition 1.2.2 (clean computation). Let f : ℕ^{k} → ℕ be a k-place numbertheoretic function, and let P be a program which computes f in the sense of Definition 1.1.7. We say that P cleanly computes f if, whenever P halts, all registers except the output register R_{k+1} are empty.

For example, the program of Example 1.1.8 cleanly computes the function f(x, y) = x + y.

Remark 1.2.3. Given a program P which computes a k-place number-theoretic function f, we can easily modify P to obtain a program which cleanly computes f. Namely, replace the stop instruction of P by

[figure: R_{1}^{−}; e; ⋯; e; R_{k}^{−}; e; R_{k+2}^{−}; e; ⋯; e; R_{s}^{−}; e; stop]

Here we have k + 1 ≤ s where R_{1}, …, R_{s} are the registers of P and R_{k+1} is the output register. Obviously this modified program has the desired property.

Using Definition 1.2.2 and Remark 1.2.3, we may simplify the proof of Theorem 1.2.1 as follows.

Simplified Proof of Theorem 1.2.1. Let F and G be programs on R_{1}, …, R_{s} which compute f and g. By Remark 1.2.3 we may assume that F and G cleanly compute f and g. Letting H be the program

[figure: start; R_{1}^{+}; stop; F; R_{2}^{−}; e; G]

we see that H computes g ∘ f. (In fact, H cleanly computes g ∘ f, but this is not essential for our purposes. We are only interested in knowing that g ∘ f is computable.)

We now generalize Theorem 1.2.1 as follows.

Theorem 1.2.4 (generalized composition). Let g be an m-place number-theoretic function, and let f_{1}, …, f_{m} be k-place number-theoretic functions. Then, there is a unique k-place number-theoretic function h given by

h(x_{1}, …, x_{k}) = g(f_{1}(x_{1}, …, x_{k}), …, f_{m}(x_{1}, …, x_{k}))

for all x_{1}, …, x_{k} ∈ ℕ. Moreover, if g and f_{1}, …, f_{m} are computable then h is computable.

Proof. The special case m = k = 1 has already been proved as Theorem 1.2.1. Let us consider the more representative special case m = 2, k = 1. Thus we have h(x) = g(f_{1}(x), f_{2}(x)) for all x. Let G, F_{1}, F_{2} be programs on R_{1}, …, R_{s} which cleanly compute g, f_{1}, f_{2} respectively. Introduce new auxiliary registers X = R_{s+1}, Y = R_{s+2}, W = R_{s+3}. Let H be the program

[figure: start; R_{1}^{−}; e; W^{−}; e; F_{1}; R_{2}^{−}; e; X^{−}; e; F_{2}; X^{+}; R_{1}^{+}; Y^{+}; R_{1}^{+}; W^{+}; stop; R_{3}^{−}; e; G; Y^{−}; e; R_{2}^{+}; R_{1}^{+}]

It is straightforward to verify that H computes h. (In fact, H cleanly computes h, but this is not essential for our purposes.)

Examples 1.2.5. We know from Section 1.1 that the functions x+y, xy, and x^{y} are computable. Moreover, it is easy to see that the functions f(x_{1}, …, x_{k}) = x_{i}, 1 ≤ i ≤ k are computable, and for each c ∈ ℕ the constant function f(x) = c is computable. But then, by applying generalized composition, we see that many more functions such as

x + 2y, x^{5} + 7xy^{2}, x^{y^{z}}, x^{y+3z} · y^{x}, etc.,

are computable.

1.3 Computable predicates

Definition 1.3.1 (computable predicates). A k-place number-theoretic predicate is a set P ⊆ ℕ^{k}. We view P as a proposition with k variables:

P (x_{1}, …, x_{k}) ≡ “the k-tuple ⟨x_{1}, …, x_{k}⟩ is an element of the set P.”

If P is a k-place predicate, the characteristic function of P is the k-place numbertheoretic function χ_{P} defined by

χ_{P} (x_{1}, …, x_{k}) = \{ \begin{matrix}1 & if P (x_{1}, …, x_{k}) is true, \\ 0 & if P (x_{1}, …, x_{k}) is false \end{matrix}

for all x_{1}, …, x_{k} ∈ ℕ. We shall often identify P with χ_{P}. In particular, a predicate is said to be computable if and only if its characteristic function is computable.

Example 1.3.2. Consider the “less than” predicate, denoted by x < y or simply <. We identify this 2-place predicate with the set L ⊆ ℕ^{2} given by

\begin{aligned}L & = & \{⟨x, y⟩ ∈ ℕ^{2} | x < y\} \\ & = & \{⟨0, 1⟩, ⟨0, 2⟩, ⟨1, 2⟩, ⟨0, 3⟩, …\} . \end{aligned}

The characteristic function of < is the characteristic function of L, i.e., the 2-place function χ_{L} given by

χ_{<}(x, y) = χ_{L}(x, y) = \{ \begin{matrix}1 & if x < y , \\ 0 & if x ≥ y . \end{matrix}

The function χ_{<} is computable, for instance by the program

[figure: start; R_{1}^{−}; e; R_{2}^{−}; e; R_{3}^{+}; R_{2}^{−}; e; stop]

Therefore, the predicate < is computable.

Example 1.3.3. Consider the 2-place predicate D(x, y) ≡ “x is a divisor of y.” Thus D(3, 5) is false, while D(3, 6) is true. Viewing this as a set D ⊆ ℕ^{2}, we have

D = {⟨x, y⟩ | x is a divisor of y}
= {⟨1, 1⟩, ⟨1, 2⟩, ⟨2, 2⟩, ⟨1, 3⟩, ⟨3, 3⟩, ⟨1, 4⟩, ⟨2, 4⟩, …} .

Thus χ_{D}(x, y) = 1 if x is a divisor of y, χ_{D}(x, y) = 0 otherwise. It can be shown that the 2-place function χ_{D} is computable. (See Exercise 1.3.15 below.) Therefore, the predicate D(x, y) is computable.

We now develop some useful techniques for proving that various predicates and functions are computable.

Theorem 1.3.4 (Boolean operations). Assume that P and Q are computable k-place predicates. Then, the following Boolean combinations are computable:

1. P ∧ Q ≡ “P and Q”, in other words, P ∩ Q.

2. P ∨ Q ≡ “P or Q”, in other words, P ∪ Q.

3. ¬ P ≡ “not P ”, in other words, ℕ^{k} \ P, the complement of P.

Proof. Let us use − as an abbreviation for the list of number-theoretic variables x_{1}, …, x_{k}. Thus P (−) is abbreviates P (x_{1}, …, x_{k}). First, note that P ∧ Q is computable because χ_{P ∧Q}(−) = χ_{P} (−) · χ_{Q}(−). Next, ¬ P is computable because χ_{¬ P} (−) = α(χ_{P} (−)) where α(1) = 0, α(0) = 1. It is easy to see that α(x) is a computable function, for example it is computed by the program

[figure: start; R_{1}^{−}; e; R_{2}^{+}; stop]

Finally, P ∨ Q is computable because χ_{P ∨Q}(−) = α(α(χ_{P} (−)) · α(χ_{Q}(−))).

Example 1.3.5. Here is an example illustrating the usefulness of Theorem 1.3.4. By Example 1.3.2 we know that the predicate x < y is computable. It follows by Theorem 1.3.4 that the predicate x = y is computable. This is because

x = y if and only if ¬ (x < y ∨ y < x),

or in other words, χ_{=}(x, y) = α(χ_{<}(x, y)) · α(χ_{<}(y, x)). Alternatively, we could have proved directly that χ_{=} is computable by writing a program for it, for instance

[figure: start; R_{1}^{−}; e; R_{2}^{−}; e; R_{3}^{+}; R_{2}^{−}; e; stop]

However, the easier proof is the one via Theorem 1.3.4.

Theorem 1.3.6 (definition by cases). Let P be a k-place number-theoretic predicate, and let f_{1} and f_{2} be k-place number-theoretic functions. Then, there is a unique k-place number-theoretic function f defined by

f(−) = \{ \begin{matrix}f_{1}(−) & if & P (−) holds, \\ f_{2}(−) & if & ¬ P (−) holds. \end{matrix}

Moreover, if P and f_{1} and f_{2} are computable, then f is computable.

Proof. Clearly f(−) = χ_{P} (−) · f_{1}(−) + α(χ_{P} (−)) · f_{2}(−). From this it follows easily that if P, f_{1}, f_{2} are computable then f is computable.

Remark 1.3.7. In Theorem 1.3.6 we have considered only two cases. The generalization to more than two cases is straightforward.

Example 1.3.8. Using definition by cases, we easily see that the function

f(x, y) = \{ \begin{matrix}0 & if & x = y, \\ 1 & if & x < y, \\ 2 & if & x > y \end{matrix}

is computable.

Theorem 1.3.9 (the μ-operator). Let P be a k + 1-place number-theoretic predicate. Assume that for all k-tuples ⟨x_{1}, …, x_{k}⟩ ∈ ℕ^{k} there exists at least one y ∈ ℕ such that P (x_{1}, …, x_{k}, y) holds. Then, there is a unique k-place number-theoretic function f defined by

f(x_{1}, …, x_{k}) = the least y such that P (x_{1}, …, x_{k}, y) holds
= μy P (x_{1}, …, x_{k}, y) .

Furthermore, if P is computable then f is computable.

Proof. The existence and uniqueness of f(x_{1}, …, x_{k}) follows from the leastnumber principle for ℕ. Namely, our assumption is that, for each k-tuple x_{1}, …, x_{k} ∈ ℕ, the set

{y ∈ ℕ | P (x_{1}, …, x_{k}, y)}

is nonempty. Since this set is nonempty, it has a least element. The least element of this set is denoted μy P (x_{1}, …, x_{k}, y).

Assume now that the predicate P is computable. We wish to prove that the function f is computable. For simplicity we consider only the case k = 1. Thus P (x, y) is a 2-place predicate, f(x) is a 1-place function, and f(x) = μy P (x, y) for all x. Let P be a program which cleanly computes χ_{P} using the registers R_{1}, …, R_{s}. Referring to Definitions 1.1.7 and 1.2.2, we know that for all x and y the run P(x, y) starting with x, y, 0, …, 0 in R_{1}, R_{2}, R_{3}, …, R_{s} will eventually halt with 1 in R_{3} if P (x, y) holds, 0 in R_{3} otherwise, and with 0 in R_{1}, R_{2}, R_{4}, …, R_{s}. Introduce additional registers X = R_{s+1}, Y = R_{s+2}, and W = R_{s+3}. Letting Q be the program

[figure: start; Y^{+}; W^{−}; e; R_{1}^{−}; e; X^{+}; W^{+}; W^{+}; R_{2}^{+}; Y^{−}; e; W^{−}; e; R_{1}^{+}; R_{1}^{+}; X^{−}; e; P; R_{2}^{+}; Y^{+}; R_{3}^{−}; e; Y^{−}; e; stop]

we claim that Q computes f. The idea is that, when we run Q(x), we are successively running P(x, 0), P(x, 1), P(x, 2), …, P(x, y), …, until we find a y such that P(x, y) halts with R_{3} nonempty. In order to conserve our knowledge of x and y while running P(x, y), we store copies of x and y in the registers X and Y respectively.

Example 1.3.10. By Theorem 1.3.9 and Exercise 1.1.12, the function

f(x) = the number of decimal digits in x
= μy (x < 10^{y})

is computable.

Theorem 1.3.11 (bounded quantification). Assume that R(x, y, −) is a k + 2place predicate.^{1} Consider the k + 1-place predicates

P (y, −) ≡ (∀x < y) R(x, y, −)

and

Q(y, −) ≡ (∃x < y) R(x, y, −) .

If R is computable, then P and Q are computable.

Proof. Note that P (y, −) holds if and only if R(x, y, −) holds for all x < y. Thus we have

P (y, −) ≡ y = μx (x = y ∨ ¬ R(x, y, −)) .

Similarly, Q(y, −) holds if and only if R(x, y, −) holds for some x < y, and thus we have

Q(y, −) ≡ y > μx (x = y ∨ R(x, y, −)) .

By Theorems 1.3.4 and 1.3.9, the proof follows.

Remark 1.3.12. The content of Theorems 1.3.4 and 1.3.11 is that the class of computable predicates is closed under Boolean operations and bounded quantification. This provides an easy way to prove that many familiar predicates are computable. The following example is particularly important.

Example 1.3.13. Consider the 1-place predicate

Prime(x) ≡ “x is a prime number.”

This predicate can be written using Boolean operations and bounded quantifiers as follows:

Prime(x) ≡ x > 1 ∧ ¬ (∃u < x) (∃v < x) [ x = u · v] .

Therefore, by Theorems 1.3.4 and 1.3.11, Prime(x) is a computable predicate. Note that we proved this without actually having to write a program to compute the characteristic function of the predicate Prime(x). Of course, we know in principle how to write such a program if needed.

Example 1.3.14. Using Theorem 1.3.9 and Example 1.3.13 we see that the function

f(x) = the least prime number greater than x
= μy (y is prime ∧ y > x)

is computable.

Exercise 1.3.15. Use Theorems 1.3.9 and 1.3.11 to prove that the following functions and predicates are computable.

1Here we are using − as an abbreviation for the list of variables z_{1}, …, z_{k}.

Rem(y, x) = the remainder of y on division by x.

Quot(y, x) = the quotient of y on division by x.

For example, Rem(17,5)=2 and Quot(17,5)=3.

GCD(x, y) = the greatest common divisor of x and y.

LCM(x, y) = the least common multiple of x and y.

D(x, y) ≡ x is a divisor of y.

LD(x, y) ≡ x is the largest prime divisor of y.

1.4 Primitive recursion

An important tool for proving that many familiar functions are computable is primitive recursion. This is essentially just the well-known method of “definition by induction.”

Example 1.4.1. We can define the factorial function f(n) = n! by

\begin{aligned}0! & = & 1 \\ (n + 1)! & = & n! · (n + 1). \end{aligned}

Note that there is one and only one (unique) number-theoretic function f satisfying these equations.

Example 1.4.2. Similarly, we can define multiplication by

\begin{aligned}f(x, 0) & = & 0 \\ f(x, y + 1) & = & f(x, y) + x \end{aligned}

because the only number-theoretic function satisfying these equations is f(x, y) = xy. In these terms the equations look like this:

\begin{aligned}x · 0 & = & 0 \\ x · (y + 1) & = & x · y + x. \end{aligned}

Theorem 1.4.3 (primitive recursion). Let f(x_{1}, …, x_{k}) be a k-place numbertheoretic function, and let g(x_{1}, …, x_{k}, y, z) be a k + 2-place number-theoretic function. Then, there is one and only one k +1-place number-theoretic function h(x_{1}, …, x_{k}, y) satisfying the primitive recursion equations

\begin{aligned}h(x_{1}, …, x_{k}, y) & = & f(x_{1}, …, x_{k}), \\ h(x_{1}, …, x_{k}, y + 1) & = & g(x_{1}, …, x_{k}, y, h(x_{1}, …, x_{k}, y)) \end{aligned}

for all x_{1}, …, x_{k}, y. Moreover, if f and g are computable, then h is computable.

Remark 1.4.4. Before proving Theorem 1.4.3, let us sketch the proof. As usual, we use − as an abbreviation for x_{1}, …, x_{k}. The existence and uniqueness of h(−, y) for all − is easily proved by induction on y. The hard part of the proof is to show that if f and g are computable then h is computable. The program for h will include the programs for g and h and work as follows. Given the inputs −, y, we wish to compute h(−, y). To guide the computation, we maintain a counter i which will run from 0 to y. We also maintain a quantity called z. At certain stages of the computation, z will take on successive values z_{i} = h(−, i) where i = 0, 1, …, y. We compute these values as follows. Initially i = 0 and we use our program for f to compute z_{0} = f(−). Then, for each i < y successively, having already computed z_{i}, we increment the counter i and use our program for g to compute z_{i+1} = g(−, i, z_{i}). Finally, when i = y, we output z = z_{y}. The details of the proof are below.

Proof of Theorem 1.4.3. For simplicity, assume k = 1. Thus we have

\begin{aligned}h(x, 0) & = & f(x) \\ h(x, y + 1) & = & g(x, y, h(x, y)) \end{aligned}

for all x, y ∈ ℕ. Let F and G be programs which cleanly compute f and g respectively, using registers R_{1}, …, R_{s}. The output registers used by F and G are R_{2} and R_{4} respectively. Introduce new registers X = R_{s+1}, Y = R_{s+2}, W = R_{s+3}, and I = R_{s+4}. Let H be the program

[figure: start; R_{1}^{−}; e; W^{−}; e; R_{2}^{−}; e; F; X^{+}; R_{1}^{+}; Y^{+}; R_{2}^{−}; e; R_{3}^{+}; W^{+}; Y^{−}; e; stop; R_{3}^{+}; R_{4}^{−}; e; I^{+}; G; W^{−}; e; I^{−}; e; W^{−}; e; X^{−}; e; I^{+}; R_{2}^{+}; X^{+}; R_{1}^{+}; W^{+}; W^{+}]

We claim that H computes h. The idea is that we run F once to compute h(x, 0) = f(x) and then we run G y times to successively compute h(x, i + 1) = g(x, i, h(x, i)) for i = 0, 1, …, y − 1. The successive values of i and h(x, i) for i = 0, 1, …, y are stored in the registers I and R_{3} respectively. We use the register Y as a counter which is initially set to y and decremented each time we run G. When Y becomes empty, we halt with h(x, y) in R_{3}.

Example 1.4.5. An example illustrating Theorem 1.4.3 and its proof is our program for multiplication using repeated addition. See Examples 1.1.10 and 1.4.2.

The following example of primitive recursion will be important in Section 1.5.

Example 1.4.6. Recall from Example 1.3.13 that the predicate

Prime(x) ≡ “x is a prime number” is computable. We can use the μ-operator plus primitive recursion to show that the function

p_{n} = the nth prime number

is computable as a function of n. Note that p_{0} = 2, p_{1} = 3, p_{2} = 5, p_{3} = 7, p_{4} = 11, …. The basic idea is to use primitive recursion to define p_{0} = 2, p_{n+1} = the least prime > p_{n}. By Theorem 1.3.9 and Example 1.3.13 (see also Example 1.3.14), we see that the function

g(x) = μw (w > x ∧ w is prime)

is computable. We then have the recursion p_{0} = 2, p_{n+1} = g(p_{n}) and this shows that p_{n} is computable as a function of n.

1.5 Prime power coding

Given a k-tuple ⟨a_{1}, …, a_{k}⟩ ∈ ℕ^{k}, we can “encode” the k-tuple as a single integer, z ∈ ℕ. Our preferred method of doing so is to designate

z = ∏_{i=1}^{k} p_{i}^{a_{i}}

as the “code” of the k-tuple ⟨a_{1}, …, a_{k}⟩. For example, the “code” of the 3-tuple ⟨8, 9, 10⟩ is the number z = 3^{8}5^{9}7^{10}.

Moreover, the Fundamental Theorem of Arithmetic tells us that every positive integer can be factored uniquely into prime powers. Thus, z can be “decoded” to recover the k-tuple ⟨a_{1}, …, a_{k}⟩.

For us, the point is that these coding and decoding methods are computable! For decoding, consider the 2-place function g(z, i) = the exponent of p_{i} in the prime power decomposition of z. This is computable, because

g(z, i) = (μx < z) (Rem(z, p_{i}^{x+1}) ≠ 0).

Here Rem(u, v) = the remainder of u on division by v, which is computable by Exercise 1.3.15.

Notation 1.5.1. We write

(z)_{i} = the exponent of p_{i} in the prime power decompsition of z.

We have seen that (z)_{i} is computable as a function of two variables, z and i.

The following example illustrates how to use prime power coding to prove that various functions are computable.

Example 1.5.2 (the Fibonacci sequence). Consider the sequence

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, …

defined by

\begin{aligned}F_{0} & = & 0 \\ F_{1} & = & 1 \\ F_{n} & = & F_{n−1} + F_{n−2} for n ≥ 2 . \end{aligned}

Here F_{n} is the nth Fibonacci number. We would like to prove that F_{n} is computable as a function of n. Unfortunately, the above definition of F_{n} does not quite fit into our scheme of primitive recursion,

\begin{aligned}h(−, 0) & = & f(−) \\ h(−, y + 1) & = & g(−, y, f(−, y)) \end{aligned}

(see Theorem 1.4.3). The difficulty is that F_{n} is defined by a 2-step recursion, while primitive recursion is a 1-step recursion.

In order to get around this difficulty, let G_{n} = 3^{F_{n}}5^{F_{n+1}} = the “code” of the ordered pair ⟨F_{n}, F_{n+1}⟩ via prime power coding. Thus (G_{n})_{1} = F_{n} and (G_{n})_{2} = F_{n+1}. We then have

\begin{aligned}G_{0} & = & 5 \\ G_{n+1} & = & 3^{(G_{n})_{2}}5^{(G_{n})_{1}+(G_{n})_{2}} \end{aligned}

and this is a 1-step recursion. Thus, by Theorem 1.4.3, G_{n} is computable as a function of n. It follows that F_{n} = (G_{n})_{1} is computable as a function of n.

As another application of prime power coding, we now introduce a technique known as course-of-values recursion.

Course-of-values recursion is a powerful variant of primitive recursion, in which h(−, y) is defined by recursion on y in terms of the entire sequence of previous values, h(−, 0), h(−, 1), …, h(−, y−1). We can use prime power coding to convert this to an ordinary 1-step recursion. Namely, consider the course-of-values function h̃ defined by

h̃(−, y) = ∏_{i=0}^{y−1} p_{i}^{h(−,i)} .

We then have the following lemma.

Lemma 1.5.3 (course-of-values recursion). Let g(−, y, z) be a k + 2-place number-theoretic function. Then, there is a unique k+1-place number-theoretic function h(−, y) defined by

h(−, y) = g(−, y, h̃(−, y))

for all −, y. Moreover, if g is computable then h is computable.

Proof. The course-of-values function h̃ can be defined from g by primitive recursion, namely

\begin{aligned}h̃(−, 0) & = & 1 , \\ h̃(−, y + 1) & = & h̃(−, y) · p_{y}^{g(−,y,h̃(−,y))} . \end{aligned}

Since g is computable, it follows by Theorem 1.4.3 that h̃ is computable. Hence

h(−, y) = (h̃(−, y + 1))_{y}

is computable. Example 1.5.4. The function

h(x, y) = x + ∑_{i=0}^{y−1} h(x, i)^{x}

is computable, because it is defined by course-of-values recursion. Remark 1.5.5. Summarizing our discussion so far, we have introduced several tools for proving that various familiar number-theoretic functions and numbertheoretic predicates are computable. Among our tools are:

1. generalized composition

2. Boolean operations ∧, ∨, ¬

3. bounded quantifiers (∀x < y), (∃x < y)

4. the μ-operator

5. primitive recursion

6. prime power coding

7. course-of-values recursion.

1.6 Computable real numbers

We end this chapter with some exercises dealing with Turing’s concept of a computable real number. These exercises are not essential for the rest of the book, but they are interesting nonetheless. Exercise 1.6.1 (approximating the square root of 2). Consider successive rational approximations of \sqrt{2} given by Newton’s method:

x_{0} = 1, x_{n+1} = x_{n} − \frac{f(x_{n})}{f′(x_{n})}

where f(x) = x^{2}−2. The first few values are x_{0} = 1, x_{1} = 3/2, x_{2} = 17/12, x_{3} = 577/408. Let a(n) and b(n) respectively be the numerator and denominator of x_{n}. Thus a(n) and b(n) are 1-place number-theoretic functions. The first few values of these functions are a_{0} = b_{0} = 1, a_{1} = 3, b_{1} = 2, a_{2} = 17, b_{2} = 12, a_{3} = 577, b_{3} = 408. Prove that the functions a(n) and b(n) are computable.

Hint: First prove that a(n) and b(n) can be defined by the simultaneous recursion

\begin{aligned}a(0) & = & 1 , \\ b(0) & = & 1 , \\ a(n + 1) & = & a(n)^{2} + 2b(n)^{2} , \\ b(n + 1) & = & 2a(n)b(n) . \end{aligned}

Then use the method of Example 1.5.2 to prove that these functions are computable.

Exercise 1.6.2 (computable real numbers). A positive real number r is said to be computable if there exist computable sequences of positive integers a_{n}, b_{n}, n = 0, 1, 2, …, such that

r = lim_{n→∞} \frac{a_{n}}{b_{n}}

and in addition

|r − \frac{a_{n}}{b_{n}} | < \frac{1}{2^{n}}

for all n ∈ ℕ.

Use primitive recursion to give a convincing argument that all of the standard examples of positive real numbers including \sqrt{2} = 1.41421 ⋯, e = 2.71828 ⋯, π = 3.14159 ⋯, etc., are computable.

Remark 1.6.3. Warning: In our definition of a computable real number given above, the condition |r − a_{n}/b_{n}| < 1/2^{n} is essential. Without this condition, Exercise 1.6.4 would not be correct.

Exercise 1.6.4. Let r be a positive real number. Prove that r is computable if and only if the number-theoretic function

f(n) = the nth decimal digit of r

is computable.

Exercise 1.6.5. Prove that the sum, product, and quotient of two computable positive real numbers are computable.

Partial recursive functions

2.1 Partial recursive functions

Consider a program P and inputs x_{1}, …, x_{k}. The run P(x_{1}, …, x_{k}) (see Notation 1.1.9) may or may not eventually halt. If it does not halt, then obviously the k-place computable function described by P is undefined at the arguments x_{1}, …, x_{k}. Thus it becomes necessary to consider computable functions which may or may not be defined for all arguments.

The purpose of this section is to introduce an appropriate terminology in order to deal with this kind of situation.

Definition 2.1.1 (partial functions). A (k-place number-theoretic) partial function

ψ : ⊆ ℕ^{k} → ℕ

is a function ψ(x_{1}, …, x_{k}) where x_{1}, …, x_{k} ∈ ℕ. The value y = ψ(x_{1}, …, x_{k}) also belongs to ℕ, or it may be undefined. Thus, the domain of ψ is a subset of ℕ^{k}, namely

dom(ψ) = {⟨x_{1}, …, x_{l}⟩ | ψ(x_{1}, …, x_{k}) is defined}.

Example 2.1.2. The partial function ψ(x, y) = x/y is undefined if y ≠ 0 or if Rem(x, y) ≠ 0. Thus, the domain of ψ is

dom(ψ) = {⟨x, y⟩ | y ≠ 0 and Rem(x, y) = 0}.

Definition 2.1.3 (total functions). Let ψ : ⊆ ℕ^{k} → ℕ be a k-place partial function. We say that ψ is total if the domain of ψ is all of ℕ^{k}. Note that a k-place total function is just what we have previously called a k-place numbertheoretic function (see for instance Definition 1.1.7).

Definition 2.1.4. A partial function ψ(x_{1}, …, x_{k}) is said to be partial recursive if it is “computable” in the sense that there exists a program P with the foloowing property. For all x_{1}, …, x_{k} ∈ ℕ, ψ(x_{1}, …, x_{k}) is defined if and only if P(x_{1}, …, x_{k}) eventually halts, in which case the value y = ψ(x_{1}, …, x_{k}) appears in register R_{k+1}.

In the previous definition, note that the expression ψ(x_{1}, …, x_{k}) may or may not be defined. In order to deal with such expressions, we introduce the following notation and terminology.

Notation 2.1.5. Let E be an expression which may or may not be defined. We write E ↓ to mean that E is defined. We write E ↑ to mean that E is undefined. If E_{1} and E_{2} are two such expressions, we write E_{1} ≃ E_{2} to mean that E_{1} and E_{2} are both defined and have the same value, or both are undefined. The binary relation ≃ is known as strong equality.

Remark 2.1.6. Given a program P and a positive integer k ≥ 1, there is obviously a unique k-place partial recursive function ψ : ⊆ ℕ^{k} → ℕ which is computed by P. Namely,

ψ(x_{1}, …, x_{k}) ≃ \{ \begin{matrix}the content of R_{k+1} if and when \\ P(x_{1}, …, x_{k}) halts, \\ undefined otherwise. \end{matrix}

Remark 2.1.7. The k-place partial recursive functions which happen to be total are exactly what we have previously called the computable k-place functions. See Definition 1.1.7.

Remark 2.1.8. One way that partial recursive functions arise naturally is from the μ-operator, as shown by the following lemma.

Theorem 2.1.9 (the μ-operator, revisited). Given a k + 1-place predicate P (x_{1}, …, x_{k}, y), there is a unique k-place partial function ψ(x_{1}, …, x_{k}) given by

ψ(x_{1}, …, x_{k}) ≃ μy P (x_{1}, …, x_{k}, y)

≃ \{ \begin{matrix}the least y such that P (x_{1}, …, x_{k}, y) holds, \\ if such a y exists, \\ undefined, if no such y exists. \end{matrix}

Moreover, if P is computable then ψ is partial recursive.

Proof. The proof of this theorem is essentially the same as that of our earlier theorem on the μ-operator, Theorem 1.3.9. Again we assume k = 1 for simplicity. The same program Q is used. The only new point is to note that if μy R(x, y) ↑ then there is no y such that R(x, y) holds, hence Q(x) goes into an infinite loop and never halts.

Exercise 2.1.10. Let ψ(x_{1}, …, x_{k}) be a k-place partial function. Show that ψ is partial recursive if and only if the 1-place partial function

ψ^{∗}(w) ≃ ψ((w)_{1}, …, (w)_{k})

is partial recursive. Thus, for many purposes, the study of k-place partial recursive functions is equivalent to the study of 1-place partial recursive functions.

Exercise 2.1.11. If f is a computable permutation of ℕ, prove that the inverse permutation f^{−1} is also computable.

(Here f^{−1}(y) = x if and only if f(x) = y. By a computable permutation of ℕ we mean a computable 1-place function f : ℕ → ℕ which maps ℕ one-to-one onto ℕ.)

Exercise 2.1.12. Generalize the previous exercise as follows. Prove that if ψ is a 1-place partial recursive function which is one-to-one, then the inverse function ψ^{−1} is again partial recursive.

Warning: This exercise is more difficult than the previous exercise.

2.2 Gödel numbers and indices

In this section we introduce what amounts to a computable method of indexing the partial recursive functions. The existence of such an indexing method will be of crucial importance in what follows.

Definition 2.2.1 (Gödel numbers). To each register machine program P we shall assign a unique number #(P), the Gödel number of P. Thus we shall have a one-to-one function

# : \{programs\} \underrightarrow{1-1} ℕ.

The idea is that #(P) will be a number which completely describes the program P. In order to define #(P), suppose P consists of l instructions of the form R_{i}^{+} or R_{i}^{−} which are labeled I_{1}, …, I_{l}. This list of instructions does not include the start and stop instructions. We always assume that I_{1} is the instruction pointed to by the start instruction. We take I_{0} to be the stop instruction. Using prime power coding, we define

#(P) = ∏_{m=1}^{l} p_{m}^{#(I_{m})}

where #(I_{m}), m = 1, …, l are numbers which completely describe the instructions I_{m}, m = 1, …, l. These numbers are defined as follows. If I_{m} is an increment instruction R_{i}^{+} pointing to I_{n}, let #(I_{m}) = 3^{i} · 5^{n}. If I_{m} is a decrement instruction R_{i}^{−} pointing to I_{n_{0}} if R_{i} is empty, I_{n_{1}} otherwise, let #(I_{m}) = 2 · 3^{i} · 5^{n_{0}} · 7^{n_{1}}. This completes the definition of #(P). Note that #(I_{1}), …, #(I_{l}) ∈ ℕ and #(P) ∈ ℕ.

Example 2.2.2. Here is a program P with three instructions labeled I_{1}, I_{2}, I_{3} not including the start and stop instructions.

[figure: I_{1}; I_{3}; start; R_{1}^{−}; e; R_{2}^{+}; stop; R_{2}^{+}; I_{2}]

Note that the program P computes the function f(x) = x + 1. The Gödel number of P is found as follows:

#(I_{1}) = 2 · 3^{1} · 5^{3} · 7^{2} = 6 · 125 · 49 = 36750
#(I_{2}) = 3^{2} · 5^{1} = 45
#(I_{3}) = 3^{2} · 5^{0} = 9
#(P) = p_{1}^{#(I_{1})}p_{2}^{#(I_{2})}p_{3}^{#(I_{3})} = 3^{36750} · 5^{45} · 7^{9}

Note that this number encodes the complete structure of the program P, via prime power coding.

Notation 2.2.3. For each e, k ∈ ℕ we have a k-place partial recursive function φ_{e}^{(k)} defined as follows. If e = #(P) for some program P, then φ_{e}^{(k)} is the kplace partial recursive function computed by P. If e is not the Gödel number of a program, then φ_{e}^{(k)} is the empty or totally undefined function.

In other words, for all nonnegative integers e, k, x_{1}, …, x_{k}, the expression

φ_{e}^{(k)}(x_{1}, …, x_{k})

is defined as the content of R_{k+1} if and when P(x_{1}, …, x_{k}) halts, provided e is the Gödel number of a program P, and provided the run P(x_{1}, …, x_{k}) eventually halts. Otherwise, φ_{e}^{(k)}(x_{1}, …, x_{k}) is undefined.

Example 2.2.4. Let e = #(P) = 3^{36750} · 5^{45} · 7^{9} as in Example 2.2.2. Note that P(x) simply adds 1 to x and outputs x + 1 in register R_{2}. Therefore, for this e and for all x, φ_{e}^{(1)}(x) is defined and equal to x + 1.

Definition 2.2.5. An index of a partial recursive function is the Gödel number of a program which computes the function. In the previous example, the integer e = 3^{36750} · 5^{45} · 7^{9} is an index of the 1-place function x + 1.

Remark 2.2.6. Clearly each partial recursive function has an index. Indeed, each partial recursive function has infinitely many indices, because there are infinitely many distinct programs which compute it.

Exercise 2.2.7. Consider the 2-place computable number-theoretic function f(x, y) = x + y. Exhibit three different indices of f. Show how to construct infinitely many distinct indices of f.

Definition 2.2.8 (recursively inseparable sets). Two sets A, B ⊆ ℕ are said to be recursively separable if there exists a recursive function f : ℕ → {0, 1} such that f(n) = 1 for all n ∈ A, and f(n) = 0 for all n ∈ B. Otherwise, A and B are said to be recursively inseparable.

Exercise 2.2.9. Consider the sets

B_{n} = {x ∈ ℕ | φ_{x}^{(1)}(x) ≃ n}

where n = 0, 1, 2, . . .. Note that the sets B_{n} for n = 0, 1, 2, …, are pairwise disjoint. Show that the sets B_{0} and B_{1} are recursively inseparable. More generally, show that B_{m} and B_{n} are recursively inseparable for all m, n such that m ≠ n.

2.3 The Enumeration Theorem

We now present an important theorem concerning indices. This theorem is basic for everything that follows.

Theorem 2.3.1 (the Enumeration Theorem). For each k ≥ 1, the k + 1-place function

φ_{e}^{(k)}(x_{1}, …, x_{k})

is partial recursive. Note that the arguments of this function are e, x_{1}, …, x_{k}.

Proof. The proof will be explained in terms of a (k + 2)-place function

State(e, x_{1}, …, x_{k}, n) = p_{0}^{m}p_{1}^{z_{1}} ⋯ p_{s}^{z_{s}} = z

which uses prime power coding to represent the state of P(x_{1}, …, x_{k}) after n steps of computation. We write e = #(P) where P is a program with registers R_{1}, …, R_{s}. Here I_{m} is the next instruction to be executed, and z_{i} is the content of register R_{i}. Note that (z)_{0} = m and (z)_{i} = z_{i} for each i = 1, …, s. Note also that this data together with P completely determines the run of P(x_{1}, …, x_{k}) from step n onward.

The main point of the proof is that State(e, x_{1}, …, x_{k}, n) is a computable (k + 2)-place function. This is proved by primitive recursion on n. We begin with

State(e, x_{1}, …, x_{k}, 0) = p_{0}^{1}p_{1}^{x_{1}} ⋯ p_{k}^{x_{k}}

which means that at step 0 we are about to execute instruction I_{1} and the inputs x_{1}, …, x_{k} are in the input registers R_{1}, …, R_{k}. This is the initial state of the run P(x_{1}, …, x_{k}). In general, for n ≥ 0 we have

State(e, x_{1}, …, x_{k}, n + 1) = NextState(e, State(e, x_{1}, …, x_{k}, n))

where NextState is a function to be described. This means that the state of P(x_{1}, …, x_{k}) after n + 1 steps of computation is to be specified in terms of the program P and the state of P(x_{1}, …, x_{k}) after n steps of computation. It remains to show that an appropriate 2-place function NextState(e, z) is computable. The details of this are below.

Note that z = State(e, x_{1}, …, x_{k}, n) is a halting state if and only if I_{0} is the next instruction to be executed, i.e., (z)_{0} = 0, in which case the output is (z)_{k+1}. Thus by Theorem 2.1.9 we have a partial recursive function

StopTime(e, x_{1}, …, x_{k}) ≃ μn (State(e, x_{1}, …, x_{k}, n))_{0} = 0)

which gives the number of steps needed for P(x_{1}, …, x_{k}) to halt, and then

φ_{e}^{(k)}(x_{1}, …, x_{k}) ≃ (State(e, x_{1}, …, x_{k}, StopTime(e, x_{1}, …, x_{k}))_{k+1}

which completes the proof.

Proof of Theorem 2.3.1, additional details. The heart of the proof is the function State(e, x_{1}, …, x_{k}, n). Recall that e = #(P) and P(x_{1}, …, x_{k}) is the run of P with inputs x_{1}, …, x_{k} in registers R_{1}, …, R_{k} and all other registers empty. The first instruction to be executed is I_{1}. After n steps of computation, we have z_{1}, …, z_{s} in registers R_{1}, …, R_{s} and we are about to execute instruction I_{m} for some m in the range 1 ≤ m ≤ l. Then

State(e, x_{1}, …, x_{k}, n) = z = p_{0}^{m}p_{1}^{z_{1}} ⋯ p_{s}^{z_{s}}.

We use primitive recursion on n to show that the (k + 2)-place function State is computable. Namely

State(e, x_{1}, …, x_{k}, 0)  = p_{0}^{1}p_{1}^{x_{1}} ⋯ p_{k}^{x_{k}}
State(e, x_{1}, …, x_{k}, n + 1) = NextState(e, State(e, x_{1}, …, x_{k}, n))

where the 2-place function NextState(e, z) does the following. It decodes from z all of the information about the state of P(x_{1}, …, x_{k}) after n steps. It then decodes the Gödel number e to find the next instruction, and executes that instruction. It then recodes the state of P(x_{1}, …, x_{k}) after n + 1 steps, using prime power coding. In detail we have

NextState(e, z) = \{ \begin{matrix}z · p_{i} · p_{0}^{−m+n_{0}} & if ((e)_{m})_{0} = 0 (increment) \\ z · p_{0}^{−m+n_{0}} & if ((e)_{m})_{0} = 1 and (z)_{i} = 0 (decrement with R_{i} empty) \\ z · p_{i}^{−1} · p_{0}^{−m+n_{1}} & if ((e)_{m})_{0} = 1 and (z)_{i} > 0 (decrement with R_{i} nonempty) \\ z & otherwise \end{matrix}

where m = (z)_{0} and i = ((e)_{m})_{1} and n_{0} = ((e)_{m})_{2} and n_{1} = ((e)_{m})_{3}. Here we are using the method of definition by cases (see Theorem 1.3.6).

It is now clear that State(e, x_{1}, …, x_{k}, n) is computable as a (k + 2)-place function of e, x_{1}, …, x_{k}, n.

We shall also need the following easy lemma.

Lemma 2.3.2. The predicate

Program(e) ≡ “e is the Gödel number of a program”

is computable.

Proof. This is straightforward, using bounded quantification. Roughly speaking, Program(e) ≡ (∃l < e) (e = ∏_{m=1}^{l} p_{m}^{(e)_{m}} ∧ (∀m < e) (if 1 ≤ m ≤ l then ∃i, j, k < e) ((e)_{m} = 3^{i} · 5^{j} ∨ (e)_{m} = 2 · 3^{i} · 5^{j} · 7^{k})), etc.

We have seen that State(e, x_{1}, …, x_{k}, n) = z is a computable (k + 2)-place function. Note that (z)_{0} = 0 means m = 0, i.e., we are in a halting state, because I_{0} is the stop instruction. Therefore, the function

StopTime(e, x_{1}, …, x_{k}) ≃ μn ((State(e, x_{1}, …, x_{k}, n))_{0} = 0 ∧ Program(e))

tells us how many steps it takes for P(x_{1}, …, x_{k}) to halt. This is undefined if P(x_{1}, …, x_{k}) never halts, or if e is not the Gödel number of a program. By Theorem 2.1.9 StopTime(e, x_{1}, …, x_{k}) is a (k + 1)-place partial recursive function. Furthermore (z)_{k+1} = the content of R_{k+1} after n steps. Therefore, we have

φ_{e}^{(k)}(x_{1}, …, x_{k}) ≃ (State(e, x_{1}, …, x_{k}, StopTime(e, x_{1}, …, x_{k})))_{k+1}

which shows that φ_{e}^{(k)}(x_{1}, …, x_{k}) is partial recursive as a function of e, x_{1}, …, x_{k}. This completes the proof of the Enumeration Theorem.

2.4 The Parametrization Theorem

The Enumeration Theorem for 1-place partial recursive functions says:

φ_{e}^{(1)}(x) is a 2-place partial recursive function (as a function of e, x).

An important supplement to the Enumeration Theorem is the Parametrization Theorem. The Parametrization Theorem for 1-place partial recursive functions says:

Given a 2-place partial recursive function ψ(w, x), we can find a 1-place total recursive function h(w) such that

φ_{h(w)}^{(1)}(x) ≃ ψ(w, x)

for all w, x.

Remark 2.4.1. The Parametization Theorem will be useful in solving Exercise 2.5.2, and it will also be used many times in later chapters of this book. See also the discussion of parametrization and uniformity in Section 4.8 below. We shall see there that the Uniformity Principle is of fundamental importance.

Proof of the Parametrization Theorem. Let P be a program which computes the 2-place partial recursive function ψ(w, x). For each fixed w, consider the program

[figure: start; R_{1}^{−}; e; R_{1}^{+}; ⋯; R_{1}^{+}; P; R_{2}^{+}; stop; R_{3}^{−}; e; R_{2}^{−}; e; R_{2}^{+}]

where the number of R_{1}^{+} instructions is w. Call this program Q_{w}.

To explain the operation of Q_{w}, suppose we start Q_{w} with an arbitrary x ∈ ℕ in R_{1} and all other registers empty. We begin by transferring x to R_{2}. We then put w into R_{1} by simply incrementing R_{1} w times. We are now ready to run P(w, x). After running P(w, x), the output ψ(w, x) is in R_{3}. We then clear R_{2}, transfer the output from R_{3} to R_{2}, and halt.

Thus, starting with x in R_{1}, Q_{w}(x) eventually halts with ψ(w, x) in R_{2} provided ψ(w, x) is defined. If ψ(w, x) is undefined, Q_{w}(x) does not halt. In other words, Q_{w} is a program which computes ψ(w, x) as a function of x. Defining h(w) = #(Q_{w}), we see that h(w) has the desired property. It remains to show that h(w) is computable as a function of w, but this is straightforward.

We now state a more general version of the Parametrization Theorem, with k arguments instead of one.

Theorem 2.4.2 (Parametrization Theorem). Given a k +1-place partial recursive function ψ(w, x_{1}, …, x_{k}), we can find a total recursive function h(w) such that

φ_{h(w)}^{(k)}(x_{1}, …, x_{k}) ≃ ψ(w, x_{1}, …, x_{k})

for all w, x_{1}, …, x_{k}.

Proof. The proof is a straightforward generalization of the special case k = 1, which was proved above.

2.5 Universal partial recursive functions

The Enumeration Theorem, due to Turing in 1936, embodies the idea of a “universal machine” which is capable of computing any recursive function. Historically, this concept of a “universal machine” led to the development of the stored-program digital computer, an invention which is of great importance in modern life, to say the least.

In this section we present some exercises which elaborate upon Turing’s idea of universality. The results of this section will not be used in the rest of the book, but we discuss them anyway because they are interesting and elegant.

Definition 2.5.1 (universal partial recursive functions). Let ψ(x) and θ(x) be 1-place partial recursive functions. We say that ψ is reducible to θ if there exists a 1-place total recursive function h(x) such that ψ(x) ≃ θ(h(x)) for all x ∈ ℕ. We refer to h(x) as a reduction function, and we say that h reduces ψ to θ. We say that θ is universal if all 1-place partial recursive functions are reducible to θ.

Exercise 2.5.2.

1. Prove that the 1-place function φ_{(x)_{1}}^{(1)}((x)_{2}) is a universal partial recursive function.

Hint: Use the Enumeration Theorem.

2. Prove that the 1-place function φ_{x}^{(1)}(x) is a universal partial recursive function.

Hint: Use the Enumeration and Parametrization Theorems.

3. Give some additional examples of universal partial recursive functions.

Exercise 2.5.3. Construct a 1-place partial recursive function θ which is universal via linear reduction functions.

(This means that each 1-place partial recursive function is reducible to θ by means of a reduction function which is linear. We say that h(x) is linear if there exist constants a and b such that h(x) = ax + b for all x.)

Hint: Proceed as in part 1 of Exercise 2.5.2. However, instead of the pairing function 3^{x}5^{y}, use a pairing function which is linear in one of its arguments.

Exercise 2.5.4. Prove that if θ is a universal partial recursive function, then the domain of θ is not recursive.

(The domain of θ is defined to be the set dom(θ) = {x | θ(x) ↓}.)

Exercise 2.5.5. Prove that any two universal partial recursive functions θ_{1} and θ_{2} are recursively isomorphic. This means that there exists a computable permutation of ℕ, call it f, such that

θ_{1}(x) ≃ y if and only if θ_{2}(f(x)) ≃ f(y)

for all x and y.

Unsolvability

The purpose of this chapter is to study a certain class of mathematical problems known as decision problems. We show that, in a certain precise sense, almost all decision problems are unsolvable. We present some specific examples of unsolvable decision problems. We indicate a method of classifying unsolvable decision problems.

3.1 Noncomputable sets and functions

Remark 3.1.1. The Enumeration Theorem gives us a 2-place partial recursive function, namely φ_{e}^{(1)}(x), which enumerates all 1-place partial recursive functions. Using this, we can exhibit a specific, total, 1-place function which is not recursive. This is accomplished by means of a trick known as diagonalization, as in the proof of the following theorem and corollary.

Theorem 3.1.2. We can exhibit a 1-place partial recursive function which cannot be extended to a 1-place total recursive function.

Proof. It follows from the Enumeration Theorem 2.3.1 that the 1-place partial function

ψ(x) ≃ φ_{x}^{(1)}(x) + 1

is partial recursive. We claim that there is no extension of ψ(x) to a total 1-place computable function f : ℕ → ℕ. To see this, let f(x) be a total 1-place function which extends ψ(x), i.e., f(x) = ψ(x) whenever ψ(x) is defined. We claim that f(x) is not computable. If f(x) were computable, let e be an index of f(x), i.e., e = #(P) where P is a program which computes f(x). In other words, f(x) = φ_{e}^{(1)}(x) for all x. In particular φ_{e}^{(1)}(e) = f(e) ≠ f(e) + 1 = φ_{e}^{(1)}(e) + 1, a contradiction. This completes the proof.

Corollary 3.1.3. We can exhibit a specific 1-place number-theoretic function which is not computable.

Proof. For example, the function

f(x) = \{ \begin{matrix}φ_{x}^{(1)}(x) + 1 & if φ_{x}^{(1)}(x) is defined \\ 0 & otherwise \end{matrix}

is noncomputable.

Remark 3.1.4. In set theory, a set S is defined to be countable if and only if S can be placed in one-to-one correspondence with a subset of ℕ. From this point of view, it is obvious that the set of computable 1-place functions is countable. (This is because there are only countably many programs to compute such functions.) On the other hand, it can be shown that the set of all 1-place functions is uncountable. (See for instance Section 3.3 below.)

From these set-theoretic cardinality considerations, it is obvious that there exist noncomputable 1-place functions. However, this kind of argument doesn’t give us any specific examples of noncomputable functions. Specific examples are most easily obtained as in Theorem 3.1.2 and 3.1.3 above, by means of the Enumeration Theorem.

3.2 An unsolvable decision problem

Definition 3.2.1. Let A ⊆ ℕ. In other words, A is a subset of ℕ. We associate to A what is called a decision problem, namely:

Given n, to “decide” whether n ∈ A or not.

This decision problem is said to be solvable if A is computable, i.e., if χ_{A}(n) is a recursive 1-place function. Otherwise, this decision problem is said to be unsolvable.

Example 3.2.2. Consider the set K defined by

K = {x ∈ ℕ | φ_{x}^{(1)}(x) ↓}.

We claim that K is noncomputable, i.e., the decision problem for K is unsolvable. To see this, suppose K were recursive, i.e., computable. Consider the function

f(x) = \{ \begin{matrix}φ_{x}^{(1)}(x) + 1 & if x ∈ K , \\ 0 & if x ∉ K . \end{matrix}

If K were computable, f(x) would be computable, using definition by cases plus the Enumeration Theorem. But we already know (by diagonalization, see Corollary 3.1.3) that this particular function f(x) is noncomputable.

Remark 3.2.3. Note also that K = ⋃_{n=0}^{∞} B_{n} where B_{n} is as in Exercise 2.2.9.

Moreover, Exercise 2.2.9 implies that each of the sets B_{n}, n = 0, 1, 2, … is, like K, noncomputable. In other words, each of the decision problems associated to B_{n}, n = 0, 1, 2, … is, like the decision problem for K, unsolvable.

Remark 3.2.4. The decision problem for K is our first main example of an unsolvable problem. This specific decision problem is known as the Diagonal Halting Problem. Namely, it is the problem of deciding for a given x ∈ ℕ whether or not the diagonalized computation φ_{x}^{(1)}(x) eventually halts.

One can also consider the Halting Problem, i.e., the decision problem for the set

H = {x | φ_{x}^{(1)}(0) ↓} .

It is called the Halting Problem, because it is the problem of deciding whether or not a given program P eventually halts, if started with all registers empty. Note that P eventually halts if and only if φ_{x}^{(1)}(0) ↓, where x = #(P).

Later we shall see that the Halting Problem is unsolvable. Moreover, we shall see that H and K have the same “degree of unsolvability.”

Later we shall also see various additional examples of unsolvable decision problems, from various branches of mathematics. Moreover, we shall classify the “degree of unsolvability” of these problems.

3.3 Almost all problems are unsolvable

In this section we argue that, in a certain precise sense, almost all decision problems are unsolvable.

Uncountable sets

By definition, the set ℕ is countable, and any set that can be indexed by ℕ is countable. For example, the set {φ_{e}^{(1)} | e ∈ ℕ} is countable. In particular, there are only countable many computable number-theoretic functions.

On the other hand, a diagonal argument shows that various sets of numbertheoretic functions are uncountable:

1. ℕ^{ℕ} = {f : ℕ → ℕ}

2. {0, 1}^{ℕ} = {f : ℕ → {0, 1}}

3. S_{∞} = {permutations of ℕ}

Namely, given a countable sequence of functions f_{n} : ℕ → ℕ, n = 0, 1, 2, …, construct g ∉ {f_{n} | n = 0, 1, 2, …} by letting g(n) = f_{n}(n) + 1 for all n. This shows that ℕ^{ℕ} is uncountable. To show that {0, 1}^{ℕ} is uncountable, use a 0, 1-valued variant, namely

g(n) = \{ \begin{matrix}1 & if f_{n}(n) = 0 \\ 0 & if f_{n}(n) ≠ 0 . \end{matrix}

To show that S_{∞} is uncoutable, define a permutation g by letting g(2n) = 2n+1 and g(2n + 1) = 2n if f_{n}(n) = 2n, otherwise g(2n) = 2n and g(2n + 1) = 2n + 1.

In particular, we see that the group of recursive permutations of ℕ is a countable subgroup of an uncountable group, namely the group of all permutations of ℕ. In other words, S_{∞}(recursive) is a countable subgroup of S_{∞}. See also Exercise 2.1.11.

Note also that, if g ∈ {0, 1}^{ℕ} is noncomputable, then the real number g(0).g(1)g(2)g(3) ⋯ is noncomputable. Here g(n) is the nth decimal digit of the number. See also Exercise 1.6.4.

The above arguments based on cardinality show that the mentioned sets of functions are uncountable. It follows that they contain noncomputable functions. In fact, the same argument shows that “almost all” functions are uncountable, where “almost all” means “all except a countable set.” The point is that, set-theoretically speaking, any uncountable set is much, much larger than any countable set.

Diagonal nonrecursiveness

Definition 3.3.1. A 1-place total number-theoretic function g : ℕ → ℕ is said to be diagonally non-recursive (abbreviated DNR) if

g(x) ≄ φ_{x}^{(1)}(x)

for all x.

Remark 3.3.2. Instead of φ_{x}^{(1)}(x) we could use any universal partial recursive function, as explained in Section 2.5.

Obviously, any DNR function is nonrecursive. The class of DNR functions and its subclasses have been the subject of much recent study.

3.4 Unsolvable problems in core mathematics

We have defined a decision problem to be a set A ⊆ ℕ. The problem A is said to be solvable if A is recursive, i.e., computable. As an example of an unsolvable problem, we have mentioned the Diagonal Halting Problem,

K = {x ∈ ℕ | φ_{x}^{(1)}(x) ↓} .

We shall now discuss two examples of unsolvable problems which are interesting from the viewpoint of core mathematics:

1. Hilbert’s 10th Problem.

2. The Word Problem for Groups.

These examples will play only a small role in the rest of this book. Nevertheless, these examples are important because they provide relevant connections and applications to core mathematics.

Remark 3.4.1. In the Pennsylvania State University Department of Mathematics, there are several experts on these topics. Professor Kirsten Eisentraeger, who joined the department recently, is an expert on generalizations of Hilbert’s 10th Problem. Professor Alexandra Shlapentokh is another expert on Hilbert’s 10th problem who will visit the department and give a MASS colloquium on November 8, 2007. Professor Alexander Nabutovsky is an expert on unsolvability of the Word Problem for Groups and its applications to geometry.

Hilbert’s 10th Problem

In a famous talk in 1900 Hilbert listed 23 problems which were intended to set the tone for 20th century mathematics. Included in the list were several problems in mathematical logic and foundations of mathematics. Problem 1 was the continuum hypothesis, while Problem 2 concerned the formal consistency of mathematics. Problem 10 on Hilbert’s list read as follows:

Given a Diophantine equation f(x_{1}, …, x_{k}) = 0, does this equation have a solution in integers x_{1}, …, x_{k}? The problem is to construct an algorithm for answering this question given the polynomial f.

(By definition, a Diophantine equation is a polynomial equation with integer coefficients.)

The following theorem due to Matiyasevich 1969 is an improvement of a result of Davis/Putnam/Robinson from the 1950s.

Theorem 3.4.2 (Matiyasevich’s Theorem). Hilbert’s 10th Problem is unsolvable.

In order to state Matiyasevich’s Theorem rigorously, we use Gödel numbering to translate Hilbert’s 10th Problem into a “decision problem” as defined above. That is, we define a function

# : {Diophantine equations} ⟶^{1−1} ℕ .

Let E be a Diophantine equation

f(x_{1}, …, x_{k}) = ∑_{e_{1},...,e_{k}} a_{e_{1}···e_{k}}x_{1}^{e_{1}} ⋯ x_{k}^{e_{k}} = 0

where e_{1}, …, e_{k} ∈ ℕ and a_{e_{1}···e_{k}} ∈ ℤ. To define #(E), we first Gödel number the integers

\begin{matrix}#(a) = 2a & for a ≥ 0 \\ #(−a) = 2a − 1 & for a > 0 \end{matrix}

and the k-tuples

#(⟨e_{1}, …, e_{k}⟩) = p_{1}^{e_{1}} ⋯ p_{n}^{e_{k}}

(prime power coding). Then we define

#(E) = ∏_{e_{1},...,e_{k}} p_{#(⟨e_{1},...,e_{k}⟩)}^{#(a_{e_{1}...e_{k}})}

and this allows us to state Matiyasevich’s Theorem precisely:

Theorem 3.4.3 (Matiyasevich 1969). The set of Gödel numbers

{#(E) | E is a Diophantine equation, E has a solution in integers}

is nonrecursive.

Proof. The proof is long and involves some non-trivial number theory. Full details are in my Spring 2005 lecture notes, available on the course web page.

The word problem for groups

Consider a finite set of symbols A = {a_{1}, …, a_{k}}. By a relation we mean a group-theoretic equation on these symbols. Such an equation may be written in the form W = 1 where W is a word, i.e., a concatenation of symbols chosen from a_{1}, …, a_{k}, a_{1}^{−1}, …, a_{k}^{−1}. Given a finite set of relations

R = {W_{1} = 1, …, W_{l} = 1}

there is a unique largest group G = ⟨A | R⟩ generated by a_{1}, …, a_{k} satisfying the given set of relations R.

Example 3.4.4. Consider the generators A = {a, b} and the relations

R = {ab = ba, a^{2} = 1, b^{3} = 1}.

Note also that the relation ab = ba can be written using a word as aba^{−1}b^{−1} = 1. We can use the relations to simplify words on this alphabet. For example

aba^{2}ba^{−1}b^{−1}a^{−1}b^{2}a^{2}ba^{−1} = a^{2}b^{4} = b.

The group G = ⟨A | R⟩ defined by these generators and relations is isomorphic to C_{2} × C_{3} with elements {1, a, b, b^{2}, ab, ab^{2}}. Here C_{n} is the cyclic group of order n.

Remark 3.4.5. Finitely presented groups arise naturally in algebraic topology and geometry. For instance, the fundamental group of a finite simplicial complex is a finitely presented group.

For each finitely presented group G = ⟨A | R⟩, the word problem for G is the following problem:

Given a word W on the finite set of generators A, does the equation W = 1 hold in G? The problem is to construct an algorithm for deciding whether W = 1 in G.

This problem was originally posed by the group theorist Dehn in the 1890s.

Note that for G as in Example 3.4.4, the word problem is solvable, because there is an algorithm to simplify each word and thus identify it with one of the six elements of G.

Theorem 3.4.6 (Boone, Novikov, 1950’s). We can construct a finitely presented group G = ⟨A | R⟩ such that the word problem for G is unsolvable.

In order to state this theorem rigorously, we define a Gödel numbering

# : {words on a_{1}, …, a_{k}} ⟶^{1−1} ℕ .

We first Gödel number the symbols a_{1}, …, a_{k}, a_{1}^{−1}, …, a_{k}^{−1} by

\begin{aligned}#(a_{i}) & = & 2i \\ #(a_{i}^{−1}) & = & 2i + 1 \end{aligned}

and then words W = a_{i_{1}}^{ϵ_{1}} ⋯ a_{i_{n}}^{ϵ_{n}} can be Gödel numbered as

#(a_{i_{1}}^{ϵ_{1}} ⋯ a_{i_{n}}^{ϵ_{n}}) = ∏_{j=1}^{n} p_{j}^{#(a_{i_{j}}^{ϵ_{j}})}

where i_{j} ∈ {1, …, k} and ϵ_{j} = ±1. We can then state:

Theorem 3.4.7 (Boone, Novikov, 1950’s). We can construct a finitely presented group G = ⟨A | R⟩ such that the set of Gödel numbers

{#(W) | W is a word on A, W = 1 in G}

is nonrecursive.

Proof. The proof is long and involves some non-trivial group theory. Full details are in my Spring 2005 lecture notes, available on the course web page.

3.5 A remark on Hilbert’s 10th Problem

Professor Kirsten Eisentraeger will talk about Hilbert’s 10th Problem at the MASS seminar tomorrow, October 30.

Recall our various characterizations of Σ_{1}^{0} subsets of ℕ. Letting A be a subset of ℕ, we know that

A is Σ_{1}^{0}

⇔ A is recursively enumerable

⇔ A is the range of a partial recursive function

⇔ A is the domain of a partial recursive function

⇔ A is finite or the range of a 1-1 total recursive function, etc. Matiyasevich showed:

A is Σ_{1}^{0}

⇔ A is Diophantine

⇔ A is range(g) ∩ ℕ where g is a polynomial with integer coefficients.

Definition 3.5.1. A k-place number-theoretic predicate P ⊆ ℕ^{k} is said to be Diophantine if we can find a polynomial equation with k + n variables and integer coefficients such that

P (x_{1}, …, x_{k}) ≡ \underbrace{∃y_{1} ⋯ ∃y_{n} (\underbrace{f(x_{1}, …, x_{k}, y_{1}, …, y_{n}) = 0}_{Diophantine equation})}_{Diophantine predicate}

for all x_{1}, …, x_{k} ∈ ℕ.

Obviously, Diophantine predicates are Σ_{1}^{0}.

Theorem 3.5.2 (Matiyasevich’s Theorem).

P is Σ_{1}^{0} if and only if P is Diophantine.

For example, the Halting Problem H ⊆ ℕ is Diophantine; hence Hilbert’s 10th Problem is unsolvable. In fact, since H is Σ_{1}^{0} complete, we have the following corollary.

Corollary 3.5.3. Hilbert’s 10th Problem ≡_{m} the Halting Problem.

3.6 Many-one reducibility

In view of the fact that almost all decision problems are unsolvable, it is desirable to develop methods for classifying unsolvable decision problems.

In this section we introduce an easy and obvious method of classifying unsolvable decision problems. Our method is embodied in the concept of many-one reducibility, which we now define.

Definition 3.6.1 (many-one reducibility). Let A, B ⊆ ℕ be decision problems. We say that A is many-one reducible to B, abbreviated A ≤_{m} B, if there exists a 1-place total recursive function f such that for all x, x ∈ A if and only if f(x) ∈ B.

Intuitively this means that, if we knew how to solve the problem B, we could then solve the problem A as follows. Given x, to decide whether x ∈ A, compute f(x) and ask whether f(x) ∈ B. If f(x) ∈ B, then x ∈ A. Conversely, if f(x) ∉ B, then x ∉ A.

Supporting this intuition, we have the following proposition.

Proposition 3.6.2. If A ≤_{m} B and B is recursive (“solvable”), then A is recursive (“solvable”).

Proof. Because A ≤_{m} B, we have χ_{A}(x) = χ_{B}(f(x)) for all x, where f is a recursive function. If in addition χ_{B} is a recursive function, then obviously χ_{A} is a recursive function.

Proposition 3.6.3. If A ≤_{m} B and A is nonrecursive, then B is nonrecursive.

Proof. This is the contrapositive of the previous proposition.

Corollary 3.6.4. If K ≤_{m} B, then B is nonrecursive.

Proof. Obvious, because K is nonrecursive.

Remark 3.6.5. The previous corollary can be used to demonstrate the unsolvability of many mathematical problems, including Hilbert’s 10th Problem and the Word Problem for Groups. One shows that the unsolvable problem K is reducible to each of these problems, hence they too must be unsolvable.

1. In the case of Hilbert’s 10th Problem, one constructs a specific Diophantine equation f(x_{0}, x_{1}, …, x_{k}) = 0 such that for all n ∈ ℕ, n ∈ K if and only if there exist integers x_{1}, …, x_{k} such that f(n, x_{1}, …, x_{k}) = 0. Thus the recursive function

n ↦ #(f(n, x_{1}, …, x_{k}) = 0)

shows that K is many-one reducible to Hilbert’s 10th Problem.

Note: By working harder, one can find a Diophantine equation

f(n, x_{1}, …, x_{k}) = 0

as above with k = 9. An open question is, what is the smallest number of variables, k, which will suffice. It is known that k = 2 will not suffice.

2. In the case of the Word Problem for groups, the rough idea is to construct a specific, finitely presented group G with generators a, b, c, d and some other generators having the following property: for all n ∈ ℕ, n ∈ K if and only if ab^{n}cb^{−n}d = 1 in G. Thus the unsolvable problem K is many-one reducible to the word problem for this particular group G.

Note: By working harder, one can construct a finitely presented group G with only two generators such that K is many-one reducible to the word problem for G.

Remark 3.6.6. Conversely, it is also true (and easy to prove, by means of the Parametrization Theorem) that Hilbert’s 10th Problem and the Word Problem for Groups are many-one reducible to K. Thus, all of these unsolvable problems are equivalent to each other, in the sense that they all have the same “degree of unsolvability.” Later we shall define and study degrees of unsolvability in more detail.

We end this section with some obvious facts and definitions.

Proposition 3.6.7. The relation ≤_{m} is reflexive and transitive. In other words, for all A, B, C ⊆ ℕ we have

1. A ≤_{m} A.

2. A ≤_{m} B and B ≤_{m} C imply A ≤_{m} C. Proof. Clearly A ≤_{m} A via the identity function, f(x) = x. If A ≤_{m} B and B ≤_{m} C via recursive functions f and g respectively, then A ≤_{m} C via the recursive function h = g ∘ f given by h(x) = g(f(x)). Definition 3.6.8 (many-one equivalence). For A, B ⊆ ℕ we say that A and B are many-one equivalent, abbreviated A ≡_{m} B, if A ≤_{m} B and B ≤_{m} A.

Intuitively, A ≡_{m} B means that the decision problems A and B are equivalent problems, in the sense that each of these problems is reducible to the other. Proposition 3.6.9. The relation ≡_{m} is an equivalence relation. In other words, it is reflexive, transitive, and symmetric:

1. A ≡_{m} A.

2. A ≡_{m} B and B ≡_{m} C imply A ≡_{m} C.

3. A ≡_{m} B if and only if B ≡_{m} A. Proof. Obvious. Definition 3.6.10. For A, B ⊆ ℕ we define

A ⊕ B = {2x | x ∈ A} ∪ {2y + 1 | y ∈ B} .

Note that we are putting a copy of A on the even numbers and a copy of B on the odd numbers.

Intuitively, if A and B are decision problems, then A⊕B is a decision problem to which both A and B are reducible. Proposition 3.6.11. A ⊕ B is the supremum (i.e., least upper bound) of A and B with respect to many-one reducibility. In other words, for all C,

A ⊕ B ≤_{m} C if and only if A ≤_{m} C and B ≤_{m} C.

Proof. Straightforward.

The arithmetical hierarchy

4.1 Definition of the arithmetical hierarchy

In Chapter 3 we proposed many-one reducibility as a method of classifying unsolvable problems. We now introduce another such method, known as the arithmetical hierarchy.

In order to motivate the arithmetical hierarchy, consider K, our favorite example of a non-recursive set. We have

x ∈ K ≡ φ_{x}^{(1)}(x) ↓
≡

∃y \underbrace{[\underbrace{(State(x, x, y))_{0} = 0}_{a halting state}]}_{a recursive predicate R(x,y)}

a Σ_{1}^{0} predicate

So, although K is not recursive, K is described by a recursive predicate plus one existential quantifier, ∃y.

We shall see that Σ_{1}^{0} is “level one” of the arithmetical hierarchy. The arithmetical hierarchy as a whole consists of level 1, level 2, …, level n, …. The arithmetical hierarchy provides a method of classifying non-recursive predicates according to the number of quantifiers needed to describe them.

Definition 4.1.1. As usual, abbreviate x_{1}, …, x_{k} as −. Recall that a k-place predicate P (−) is the same thing as a subset of ℕ^{k}.

1. For n ≥ 1, a k-place predicate P (−) is said to be Σ_{n}^{0} if P (−) can be written in the form

P (−) ≡ \underbrace{∃y_{1} ∀y_{2} ∃y_{3} ∀y_{4} ⋯ \begin{matrix}∃ \\ ∀ \end{matrix} y_{n}}_{n alternating quantifiers starting with ∃} R(−, y_{1}, y_{2}, …, y_{n})

where R is a recursive predicate.

2. A k-place predicate P (−) is said to be Π_{n}^{0} (n ≥ 1) if P (−) can be written in the form

P (−) ≡ \underbrace{∀y_{1} ∃y_{2} ∀y_{3} ∃y_{4} ⋯ \begin{matrix}∀ \\ ∃ \end{matrix} y_{n}}_{n alternating quantifiers starting with ∀} R(−, y_{1}, y_{2}, …, y_{n})

where R is a recursive predicate.

Example 4.1.2. Consider the 1-place predicate (i.e., the set) T ⊆ ℕ defined by

T = {e ∈ ℕ | φ_{e}^{(1)} is a total function}.

To classify T in the arithmetical hierarchy, we have

e ∈ T ≡ ∀x φ_{e}^{(1)}(x) ↓
≡ ∀x ∃y [ (State(e, x, y))_{0} = 0 ]
≡ Π_{2}^{0}

so T belongs to the class Π_{2}^{0}.

Question: does T belong to the class Σ_{2}^{0}? Later we shall develop a method for answering such questions. See also Example 4.4.5 below.

Exercise 4.1.3. Prove that T is not recursive.

Hint: Use the Parametrization Theorem.

4.2 Closure properties

The various levels of the arithmetical hierarchy have some useful closure properties, expressed in the next four propositions.

Proposition 4.2.1. For n ≥ 1, each of the classes Σ_{n}^{0} and Π_{n}^{0} is closed under substitution of total recursive functions.

This means that, if P (−, y) is a Σ_{n}^{0} predicate and f(−) is a total recursive function, then the predicate

Q(−) ≡ P (−, f(−))

is again Σ_{n}^{0}. And similarly for Π_{n}^{0}.

Proof. For example, suppose P is Σ_{3}^{0}, i.e.,

P (−, y) ≡ ∃u ∀v ∃w R(−, y, u, v, w)

where R(−, y, u, v, w) is recursive. Then

Q(−) ≡ ∃u ∀v ∃w R(−, f(−), u, v, w)

so Q is again Σ_{3}^{0}.

Proposition 4.2.2. The classes Σ_{n}^{0} and Π_{n}^{0} for n ≥ 1 are closed under ∧, ∨, ∀x < y, ∃x < y.

Remark 4.2.3. Caution: The classes Σ_{n}^{0} and Π_{n}^{0} for n ≥ 1 are not closed under negation. For instance, K is Σ_{1}^{0} but not Π_{1}^{0}, and ¬ K is Π_{1}^{0} but not Σ_{1}^{0}.

Proof of Proposition 4.2.2. We have

\begin{aligned}∃y P (−, y) ∧ ∃y Q(−, y) & ≡ & ∃y ∃z [ P (−, y) ∧ Q(−, z) ] \\ & ≡ & ∃w [ P (−, (w)_{1}) ∧ Q(−, (w)_{2}) ] . \end{aligned}

Thus consecutive ∃’s (and similarly, consecutive ∀’s) can be reduced to a single ∃ (or a single ∀) by means of a pairing function or prime power coding.

Bounded quantifiers can be handled by means of Theorem 1.3.11 plus the following manipulations:

(∀x < y) ∃z P (x, y, z, −) ≡ ∃w (∀x < y) (∃z < w) P (x, y, z, −)

and

(∃x < y) ∀z P (x, y, z, −) ≡ ∀w (∃x < y) (∀z < w) P (x, y, z, −) .

Proposition 4.2.4. The class Σ_{n}^{0} is closed under ∃. Similarly, the class Π_{n}^{0} is closed under ∀. Here ∃ and ∀ refer to existential and universal quantification over ℕ, respectively.

Proof. Suppose P (−, y) is Σ_{n}^{0}. We then have

Q(−) ≡ ∃y P (−, y)
≡ ∃y ∃y_{1} ∀y_{2} ⋯ y_{n} R(−, y, y_{1}, …, y_{n})
≡ ∃w ∀y_{2} ⋯ y_{n} R(−, (w)_{1}, (w)_{2}, …, y_{n})
≡ Σ_{n}^{0}

The next example illustrates how our closure properties can be useful in classifying unsolvable problems.

Example 4.2.5. Consider the set

S = {e | the domain of φ_{e}^{(1)} is infinite}.

Classifying S in the arithmetical hierarchy, we have

e ∈ S ≡ φ_{e}^{(1)}(x) ↓ for infinitely many x
≡ ∀y ∃x ( x > y ∧ φ_{e}^{(1)}(x) ↓)

≡ ∀y \underbrace{∃x \underbrace{( \underbrace{x > y}_{recursive} ∧ \underbrace{∃s \underbrace{(State(e, x, s))_{0} = 0)}_{recursive}}_{Σ_{1}^{0}} )}_{Σ_{1}^{0}}}_{Σ_{1}^{0}}

Π_{2}^{0}

Thus we see that S is Π_{2}^{0}. This kind of computation is called a Tarski/Kuratowski computation.

Proposition 4.2.6. If P (−) is Σ_{n}^{0}, then ¬ P (−) is Π_{n}^{0}. If P (−) is Π_{n}^{0}, then ¬ P (−) is Σ_{n}^{0}.

Proof. ¬ ∃ ≡ ∀¬ and ¬ ∀ ≡ ∃¬

¬ ∃y_{1} ∀y_{2} ∃y_{3} ≡ ∀y_{1} ¬ ∀y_{2} ∃y_{3} ≡ ∀y_{1} ∃y_{2} ¬ ∃y_{3} ≡ ∀y_{1} ∃y_{2} ∀y_{3} ¬

Notation 4.2.7. We shall frequently use the notation

W_{e} = domain of φ_{e}^{(1)} = {x | φ_{e}^{(1)}(x) ↓}.

Here e is called an index of the set W_{e}.

Note that some of our previous examples can be expressed concisely using this notation. In particular we have K = {e | e ∈ W_{e}}, T = {e | W_{e} = ℕ}, S = {e | W_{e} is infinite}. Another such example is:

R = {e | W_{e} is recursive}.

Exercise 4.2.8. Perform a Tarski/Kuratowski computation to classify R in the arithmetical hierarchy.

(picture of arithmetical hierarchy)

4.3 Distinguishing levels of the hierarchy

In this section we continue our discussion of the arithmetical hierarchy. We show that all of the levels of the arithmetical hierarchy are distinct.

The following theorem is a useful characterization of the lowest level of the hierarchy, Σ_{1}^{0}.

As usual, we denote x_{1}, …, x_{k} by −.

Theorem 4.3.1. P ⊆ ℕ^{k}, a k-place predicate, is Σ_{1}^{0} if and only if P is the domain of some partial recursive function.

Proof. (⇐): Assume P = dom(φ_{e}^{(k)})

Then P (−) ≡ φ_{e}^{(k)} ↓

≡ ∃n \underbrace{(State(e, −, n))_{0} = 0}_{recursive}

Σ_{1}^{0}

(⇒): P (−) is Σ_{1}^{0}. Then P (−) ≡ ∃y R(−, y) where R is recursive. Then ψ(−) ≃ μyR(−, y) is a partial recursive function, and P = dom(ψ)

In order to distinguish the levels of the arithmetical hierarchy, we first introduce universal predicates at each level.

Definition 4.3.2. A (k + 1)-place predicate U(e, −) is said to be universal Σ_{n}^{0} (for k-place predicates) if

1. U(e, −) is Σ_{n}^{0}

2. every k-place Σ_{n}^{0} predicate is ≡ U(e, −) for some fixed e.

(Universal Π_{n}^{0} predicates are defined similarly.)

Theorem 4.3.3. For each k ≥ 1 and n ≥ 1, there exist universal Σ_{n}^{0} and Π_{n}^{0} predicates.

Proof. 1. The predicate U(e, −) ≡ φ_{e}^{(k)}(−) ↓ is universal Σ_{1}^{0}.

2. If U(e, −) is universal Σ_{n}^{0}, then ¬ U(e, −) is universal Π_{n}^{0}. If U(e, −) is universal Π_{n}^{0}, then ¬ U(e, −) is universal Σ_{n}^{0}.

3. If U(e, −, y) is universal Σ_{n}^{0} for (k +1)-place predicates, then ∀y U(e, −, y) is universal Π_{n+1}^{0} for k-place predicates.

Theorem 4.3.4. For each n ≥ 1 there exist sets which are Σ_{n}^{0} and not Π_{n}^{0} (or, Π_{n}^{0} and not Σ_{n}^{0}).

Proof. Let U_{n}(e, x) be a universal Σ_{n}^{0} predicate. Consider the set

K_{n} = {e | U_{n}(e, e) holds}.

For example, K_{1} = K. Clearly K_{n} is Σ_{n}^{0}. The usual diagonal argument shows that K_{n} is not Π_{n}^{0}. By taking the complement of K_{n}, we obtain a set which is Π_{n}^{0} and not Σ_{n}^{0}.

Definition 4.3.5. Δ_{n}^{0} = Σ_{n}^{0} ∩ Π_{n}^{0}. In other words, a predicate P (−) is said to be Δ_{n}^{0} if and only if it is both Σ_{n}^{0} and Π_{n}^{0}.

Theorem 4.3.6. A predicate P (−) is Δ_{1}^{0} if and only if P (−) is recursive.

Proof. (⇐): trivial

(⇒): P (−) is Δ_{1}^{0}, so P (−) ≡ ∀y R_{1}(−, y) and P (−) ≡ ∃z R_{2}(−, z).

Exercise 4.3.7. Show that for each n ≥ 1 there exist sets which are Δ_{n+1}^{0} and neither Σ_{n}^{0} nor Π_{n}^{0}.

4.4 Many-one completeness

We continue our discussion of the arithmetical hierarchy. We develop a method for proving that decision problems at specific levels of the arithmetical hierarchy do not belong to any lower level.

Recall our discussion of many-one reducibility in Section 3.6. For A, B ⊆ ℕ, the definition of A ≤_{m} B is that we can find a total recursive function f such that x ∈ A ⇔ f(x) ∈ B for all x ∈ ℕ.

Proposition 4.4.1. If A ≤_{m} B and B is Σ_{n}^{0}, then A is Σ_{n}^{0}. And similarly for Π_{n}^{0}.

Proof. Recall that the class of Σ_{n}^{0} predicates is closed under recursive substitution. (See Proposition 4.2.1.) Suppose A ≤_{m} B via a recursive function f. Viewing A, B ⊆ ℕ as 1-place predicates, we have A(x) ≡ B(f(x)) for all x ∈ ℕ. If the predicate B(y) is Σ_{n}^{0}, it follows that the predicate A(x) is Σ_{n}^{0}, Q.E.D.

Proposition 4.4.2. If A and B are Σ_{n}^{0}, then A ⊕ B is Σ_{n}^{0}. And similarly for Π_{n}^{0}.

Proof. Straightforward.

Definition 4.4.3 (Σ_{n}^{0} completeness). A set C ⊆ ℕ is said to be Σ_{n}^{0} complete, or many-one complete within the class Σ_{n}^{0}, if it has the following two properties.

1. C is Σ_{n}^{0}.

2. Every Σ_{n}^{0} set is ≤_{m} C.

Π_{n}^{0} completeness is defined similarly.

Theorem 4.4.4. For each n ≥ 1 we have:

1. Σ_{n}^{0} complete sets exist.

2. Any such set is not Π_{n}^{0}.

Proof. 1. Let U_{n}(e, x) be a universal Σ_{n}^{0} predicate. Let C_{n} = {3^{e}5^{x} | U_{n}(e, x) holds}. Clearly C_{n} is Σ_{n}^{0} complete.

2. If C is Σ_{n}^{0} complete, we have in particular K_{n} ≤_{m} C. (See the proof of Theorem 4.3.4.) Since K_{n} is not Π_{n}^{0}, it follows that C is not Π_{n}^{0}.

Example 4.4.5. Recall from Example 4.1.2 that T = {e | φ_{e}^{(1)} is total} is Π_{2}^{0}. We claim that T is Π_{2}^{0} complete. It follows that T is not Σ_{2}^{0}. In particular, T is neither Σ_{1}^{0} nor Π_{1}^{0}.

To show that T is Π_{2}^{0} complete, let A be any Π_{2}^{0} set. We need to show that A ≤_{m} T. We have x ∈ A ≡ ∀y ∃z R(x, y, z) where R is recursive. Consider the partial recursive function ψ(x, y) ≃ μz R(x, y, z). By the Parametrization Theorem, there is a total recursive function h(x) such that φ_{h(x)}^{(1)}(y) ≃ ψ(x, y) for all x, y. Then

x ∈ A ≡ ∀y ∃z R(x, y, z)
≡ ∀y ψ(x, y) ↓
≡ ∀y φ_{h(x)}^{(1)}(y) ↓
≡ φ_{h(x)}^{(1)} is total

Example 4.4.6. Recall the sets K = {x | φ_{x}^{(1)}(x) ↓} and H = {x | φ_{x}^{(1)}(0) ↓ } = {#(P) | P halts} = the Halting Problem.

Note that K, H are Σ_{1}^{0} sets. We claim they are Σ_{1}^{0} complete. To see this, let A be any Σ_{1}^{0} set. To show A ≤_{m} K, A ≤_{m} H, note that x ∈ A ≡ ∃y R(x, y) where R(x, y) is a recursive predicate. We have a partial recursive function ψ(x, z) ≃ μy R(x, y). By the Parametrization Theorem, we get a recursive function h(x) such that φ_{h(x)}^{(1)}(z) ≃ ψ(x, z). Then

x ∈ A ≡ ∃y R(x, y)
≡ ψ(x, z) ↓
≡ φ_{h(x)}^{(1)}(z) ↓ (independent of z)
≡ φ_{h(x)}^{(1)}(0) ↓≡ x ∈ H
≡ φ_{h(x)}^{(1)}(h(x)) ↓≡ x ∈ K

Exercise 4.4.7. Show that K_{n} (in the proof of Theorem 4.3.4) is Σ_{n}^{0} complete.

Remark 4.4.8. The sets K, H, S, T, R, E are all complete at appropriate levels of the arithmetical hierarchy.

But, we shall see later that there exist Σ_{n}^{0} and Π_{n}^{0} sets which are not complete.

Remark 4.4.9. Hilbert’s 10th Problem, the Word Problem for Groups, etc. are Σ_{1}^{0} complete. Thus, they are equivalent to the Halting Problem.

4.5 Recursively enumerable sets

Theorem 4.5.1 (recursively enumerable sets). For A ⊆ ℕ, t.f.a.p.e. (the following are pairwise equivalent):

1. A is Σ_{1}^{0}

2. A = domain of a partial recursive function

3. A = W_{e} = dom(φ_{e}^{(1)}) for some e

4. A = range of a partial recursive function

5. A = ∅ or A is the range of a total recursive function

6. A is finite or A is the range of a 1-1 total recursive function

We call such sets A recursively enumerable sets, or r.e. sets for short.

Proof. 1 ⇔ 2 ⇔ 3 is obvious, and 6 ⇒ 5 ⇒ 4 is obvious.

4 ⇒ 1:

Suppose A = range of φ_{e}^{(1)} = {y | ∃x φ_{e}^{(1)}(x) ≃ y}.

Then: y ∈ A ≡ ∃x φ_{e}^{(1)}(x) ≃ y

≡ ∃x ∃n [\underbrace{(State(e, x, n))_{0} = 0 ∧ (State(e, x, n))_{2} = y}_{recursive}]

Σ_{1}^{0}
≡ Σ_{1}^{0}

We will show 1 ⇒ 6 to complete the proof, but first a lemma:

Lemma 4.5.2. Let B ⊆ ℕ be an infinite recursive set. Then the principal function

π_{B}(n) = nth element of B in increasing order

is a total recursive function.

\begin{aligned}Pf: & π_{B}(0) & = & μw (w ∈ B) \\ & π_{B}(n + 1) & = & μw (w ∈ B ∧ w > π_{B}(n)) \end{aligned}

Now, to prove 1 ⇒ 6:

Assume A is infinite and Σ_{1}^{0}, A = {x | ∃y R(x, y)} where R(x, y) is a recursive predicate. Let ψ(x) ≃ μy R(x, y) (a partial recursive function) and

B = \{3^{x}5^{ψ(x)} | x ∈ A\} = \{3^{x}5^{y} | \underbrace{R(x, y) ∧ ¬ ∃z < y R(x, z)}_{recursive}\}.

Note that B is infinite and recursive, so π_{B} is recursive. Then B = the range of π_{B}, and π_{B} is one-to-one. Define f(n) = (π_{B}(n))_{1}. Then A = the range of f; and by construction, f is one-to-one and recursive.

Remark 4.5.3. Define a set A ⊆ ℕ to be Diophantine if there exists a polynomial f(w, x_{1}, …, x_{k}) with integer coefficients such that

A = {n | ∃x_{1} ⋯ ∃x_{k} f(n, x_{1}, …, x_{k}) = 0}.

Here n ranges over ℕ and x_{1}, …, x_{k} range over ℤ. Obviously every Diophantine set is Σ_{1}^{0}. It follows by the previous theorem that every Diophantine set is recursively enumerable. A consequence of Matiyasevich’s solution of Hilbert’s 10th Problem is:

A is Diophantine if and only if A is recursively enumerable.

This is another nice characterization of r.e. sets.

4.6 Homework #3, due September 17, 2007

Remark 4.6.1. For the next exercise, recall that W_{x} = dom(φ_{x}^{(1)}). Note that W_{x}, x = 0, 1, 2, …, is the standard recursive enumeration of the recursively enumerable subsets of ℕ.

Exercise 4.6.2. Which many-one reducibility relations hold or do not hold among the following sets and their complements?

K = {x | x ∈ W_{x}}

H = {x | 0 ∈ W_{x}}

T = {x | W_{x} = ℕ}

E = {x | W_{x} = ∅}

S = {x | W_{x} is infinite}

Prove your answers.

Hint: Show that each of these sets is many-one complete within an appropriate level of the arithmetical hierarchy.

4.7 Recursively enumerable sets

Exercise 4.7.1. Prove the following:

1. Every infinite recursively enumerable set includes an infinite recursive set.

2. Every infinite recursive set includes a recursively enumerable set which is not recursive.

3. Every infinite recursive set is the union of two disjoint infinite recursive sets.

4. Every infinite recursively enumerable set is the union of two disjoint infinite recursively enumerable sets.

5. (Extra Credit) Every recursively enumerable set which is nonrecursive is the union of two disjoint recursively enumerable sets which are nonrecursive.

Solution. A simple fact is that, for any infinite set A, π_{A} (the principal function of A) is ≡_{T} A. (Recall that A = {π_{A}(0) < π_{A}(1) < π_{A}(2) < ⋯ < π_{A}(n) < ⋯}.)

In particular, if A is nonrecursive then π_{A} is nonrecursive.

For part 3, let S be infinite recursive, so π_{S} is recursive. Define S_{1} = {π_{S}(n) | n odd}, S_{2} = {π_{S}(n) | n even}. Then S_{1} and S_{2} are infinite recursive, and S = S_{1} ∪ S_{2} and S_{1} ∩ S_{2} = ∅.

Part 4 is similar. Let A be r.e. infinite. Then A = rng(f) for some 1-1 total recursive f. Let A_{1} = {f(n) | n odd} and A_{2} = {f(n) | n even}. Then A = A_{1} ∪ A_{2} and A_{1}, A_{2} are r.e. infinite and A_{1} ∩ A_{2} = ∅.

For part 1, let A and f be as above. Define a recursive increasing function g by g(0) = f(0), g(i + 1) = f(μn f(n) > g(i)). Then g is recursive and by definition g(i + 1) > g(i) for all i. Letting B = rng(g) we have g = π_{B}, B ⊆ A, B is recursive and infinite.

Here is an alternative solution, from Robin Tucker-Drob, inspired by deficiency sets. Let C = {f(n) | f(n) ≥ n}. We have m ∈ C ≡ (∃n ≤ m) (m = f(n)) so C is recursive. We can argue that C is infinite.

For part 2, let S be an infinite recursive set. Let B = {π_{S}(n) | n ∈ K}. Clearly B is an infinite r.e. subset of S. Also B is nonrecursive, because for all n we have n ∈ K ⇔ π_{S}(n) ∈ B, so K ≤_{m} B.

Part 5 is harder; the result is a published paper.

Exercise 4.7.2 (creative sets). A set P ⊆ ℕ is said to be productive if there exists a total recursive function h(x) such that for all x, if W_{x} ⊆ P then h(x) ∉ W_{x} and h(x) ∈ P. Such a function is called a productive function for P.

A creative set is a recursively enumerable set whose complement is productive.

Prove the following.

1. K is creative.

2. If A and B are recursively enumerable sets and A ≤_{m} B and A is creative, then B is creative.

3. If B is recursively enumerable and many-one complete, then B is creative.

4. (Extra Credit) If B is creative, then B is many-one complete.

5. (Extra Credit) If A and B are creative, then A and B are recursively isomorphic. This means that there exists a recursive permutation of ℕ, call it g, such that x ∈ A if and only if g(x) ∈ B, for all x.

Exercise 4.7.3 (simple sets). A set I ⊆ ℕ is said to be immune if I is infinite yet includes no infinite recursively enumerable set.

A simple set is a recursively enumerable set whose complement is immune. Prove the following.

1. If A is simple, then A is not recursive.

2. If A is simple, then A is not creative.

Exercise 4.7.4 (deficiency sets). Let f : ℕ ⟶^{1−1} ℕ be a one-to-one total recursive function such that the range of f is nonrecursive. The deficiency set of f is defined as

D_{f} = {x | ∃y (x < y ∧ f(x) > f(y))} .

Prove that D_{f} is a simple set.

Conclude that there exist recursively enumerable sets which are neither recursive nor many-one complete.

Exercise 4.7.5. (Extra Credit) Generalize Exercises 4.7.2, 4.7.3, 4.7.4 to higher levels of the arithmetical hierarchy. Conclude that for each n ≥ 1 there exist Σ_{n}^{0} sets which are neither Δ_{n}^{0} nor many-one complete within the class of Σ_{n}^{0} sets.

4.8 Parametrization and uniformity

In this section we discuss a fundamental but somewhat vague principle, known as the Uniformity Principle.

Recall the concept of indices. If e = #(P), we say that e is an index of the partial recursive function φ_{e}^{(1)}(x) which is computed by P. We also say that e is an index of the r.e. set W_{e} = dom(φ_{e}^{(1)}) = {x | φ_{e}^{(1)}(x) ↓}.

The true meaning of the Parametrization Theorem is embodied in the following vague but useful “uniformity principle”:

Many operations on partial recursive functions and r.e. sets can be described as computable operations on the indices of these functions and sets.

Example 4.8.1. Consider the union of two r.e. sets, W_{x} ∪ W_{y}. We know that this is r.e., hence there exists z such that W_{z} = W_{x} ∪ W_{y}. The Uniformity Principle tells us something more. Namely, given x and y we can \underline{compute} an index z = f(x, y) of W_{x} ∪ W_{y}. Here f(x, y) is a total computable function.

Proof. We have

u ∈ W_{x} ∪ W_{y} ≡ ∃s [ (State(x, u, s))_{0} = 0 ∨ (State(y, u, s))_{0} = 0 ]
≡ ψ(u, x, y) ↓

where ψ(u, x, y) ≃ μs [ (State(x, u, s))_{0} = 0 ∨ (State(y, u, s))_{0} = 0 ]. So far we have shown that W_{x} ∪ W_{y} is Σ_{1}^{0}, i.e., it is an r.e. set. We want to show more, that we can find its index computably from x and y.

By the Parametrization Theorem, find a recursive function g(w) such that φ_{g(w)}^{(1)}(u) ≃ ψ(u, (w)_{1}, (w)_{2}) for all u, w. Consider the recursive function f(x, y) =

g(3^{x}5^{y}). Then φ_{f(x,y)}^{(1)}(u) ≃ φ_{g(3^{x}5^{y})}^{(1)}(u) ≃ ψ(u, x, y), hence φ_{f(x,y)}^{(1)}(u) ↓ ≡ u ∈ W_{x} ∪ W_{y}. Thus W_{f(x,y)} = W_{x} ∪ W_{y} for all x, y as desired.

Example 4.8.2. Similarly, we can show that various other operations on indices are recursive. For instance, we obtain total recursive functions f, g, h, k, l with the following properties.

1. W_{f(x,y)} = W_{x} ∪ W_{y}

2. W_{g(x,y)} = W_{x} ∩ W_{y}

3. W_{h(x,y)} = W_{x} ∪ {y}

4. φ_{k(x,y)}^{(1)} = φ_{x}^{(1)} ∘ φ_{y}^{(1)}

5. W_{l(x,y)} = (φ_{x}^{(1)})^{−1} (W_{y}) = {u | φ_{x}^{(1)}(u) ↓ ∈ W_{y}}

We now apply the Uniformity Principle to solve some of the problems in Homework #3.

Solution of Homework #3, Problem 2(b)

Show that A ≤_{m} B, A creative, B r.e. imply B creative. It suffices to show that if P ≤_{m} Q and P is productive, then Q is productive.

Proof. Let h(x) be a productive function for P. This means that if W_{x} ⊆ P then h(x) ∉ W_{x} and h(x) ∈ P. Assume P ≤_{m} Q via the recursive function f, i.e., x ∈ P if and only if f(x) ∈ Q. Suppose now that W_{x} ⊆ Q. Then f^{−1}(W_{x}) is an r.e. set; further, by the uniformity principle (Parametrization Theorem), there is a recursive function g(x) such that W_{g(x)} = f^{−1}(W_{x}) for all x. We have W_{g(x)} ⊆ P since W_{x} ⊆ Q, so we apply our productive function for P to get h(g(x)) ∈ P, ∉ W_{g(x)}. Finally, apply the reduction function f to get k(x) = f(h(g(x))) ∈ Q, ∉ W_{x}. Thus k = f ∘ h ∘ g is a productive function for Q.

Solution of Homework #3, Problem 3(b)

Show that a creative set is not simple. It suffices to show that a productive set P is not immune. We need to show that P has an infinite r.e. set.

Let h(x) be a productive function for P. Start with x_{0}, an index of ∅, the empty set. Then W_{x_{0}} = ∅, so trivially W_{x_{0}} ⊆ P. Hence h(x_{0}) ∉ W_{x_{0}} (obviously) and h(x_{0}) ∈ P. Let W_{x_{1}} = {h(x_{0})} ⊆ P. Then h(x_{1}) ∉ W_{x_{1}} and h(x_{1}) ∈ P. Continuing in this fashion, we generate a sequence of distinct integers h(x_{0}), h(x_{1}), h(x_{2}), … which hopefully form an infinite r.e. subset of P.

To make this work, we need to show the construction of x_{0}, x_{1}, x_{2}, … can be done in a uniform or recursive manner, via the Parametrization Theorem. The Uniformity Principle gives us a recursive function g(x, y) such that W_{g(x,y)} = W_{x} ∪ {y} for all x, y. We want W_{x_{n+1}} = W_{x_{n}} ∪ {h(x_{n})} = W_{g(x_{n},h(x_{n}))} for all n, so define the infinite sequence x_{0}, x_{1}, x_{2}, … by letting x_{n+1} = g(x_{n}, h(x_{n})) for all n. Since h(x) and g(x, y) are recursive functions, the sequence x_{0}, x_{1}, …, x_{n}, … is recursive. It follows that the set {h(x_{0}), h(x_{1}), h(x_{2}), …} is an infinite r.e. subset of P.

Lecture 16: September 28, 2007

Oracles and relativization

5.1 Oracle computation

We now extend the power of register machines by considering oracle programs.

Our programs already include the increment instruction R_{i}^{+} and the decrement instruction R_{i}^{−}. We now add the oracle instruction R_{i}^{0} which replaces n in R_{i} with f(n). Here f : ℕ → ℕ is a fixed, total, 1-place, number-theoretic function, called a Turing oracle.

Example 5.1.1. Given a 1-place number-theoretic function f(x), consider the function g(x) = f(f(x) + 1). We claim that g is computable using f as an oracle. This is shown by the oracle program

[figure: start; R_{1}^{0}; R_{1}^{+}; R_{1}^{0}; R_{1}^{−}; e; stop; R_{2}^{+}]

which uses f as an oracle to compute g.

Definition 5.1.2 (Gödel numbers). We extend our Gödel numbering system to oracle programs. The definition of #(I_{m}) for an oracle instruction I_{m} is #(I_{m}) = 4 · 3^{i} · 5^{n} where I_{m} is an R_{i}^{0} instruction pointing to I_{n} as the next instruction. The definition of #(P) is then as before. Thus we now have a Gödel numbering

# : {oracle programs} ⟶^{1−1} ℕ

extending our earlier Gödel numbering

# : {programs} ⟶^{1−1} ℕ.

Definition 5.1.3. An oracle is a total, 1-place, number-theoretic function.

Notation 5.1.4. Let P be an oracle program, and let f be an oracle.

1. We denote by P^{f}(x_{1}, …, x_{k}) the run of P using oracle f starting with x_{1}, …, x_{k} in R_{1}, …, R_{k} and all other registers empty. The output of P^{f}(x_{1}, …, x_{k}) is the content of R_{k+1} if and when P^{f}(x_{1}, …, x_{k}) halts. This is an obvious generalization of Notation 1.1.9.

2. Let e = #(P). We write

φ_{e}^{(k),f}(x_{1}, …, x_{k}) ≃ the output of P^{f}(x_{1}, …, x_{k}) .

This is an obvious generalization of Notation 2.2.3.

3. We denote by W_{e}^{f} the domain of φ_{e}^{(1),f}. This is an obvious generalization of Notation 4.2.7.

Definition 5.1.5. Let f be a fixed oracle. A partial function

ψ : ⊆ ℕ^{k} → ℕ

is said to be partial f-recursive, or partial recursive relative to f, if there exists e such that

ψ(x_{1}, …, x_{k}) ≃ φ_{e}^{(k),f}(x_{1}, …, x_{k})

for all x_{1}, …, x_{k} ∈ ℕ.

Example 5.1.6. Let

f(x) = χ_{H} = \{ \begin{matrix}1 & if φ_{x}^{(1)}(0) ↓ \\ 0 & if φ_{x}^{(1)}(0) ↑ \end{matrix}

Then clearly the Halting Problem H is solvable relative to f. Further, all r.e. sets are f-recursive.

Proof. Given an r.e. set W_{e} = {x | φ_{e}^{(1)}(x) ↓}, define the function θ(x, y) = φ_{e}^{(1)}(x). By the parametrization theorem, there is a total recursive function h such that θ(x, y) ≃ φ_{h(x)}^{(1)}(y).

In particular, this is true for y = 0; so φ_{e}^{(1)}(x) ≃ φ_{h(x)}^{(1)}(0).

Then χ_{W_{e}} = χ_{H} ∘ h, which is f-recursive. The oracle program will contain the program for h, then will run the oracle instruction on h(x) to see if it halts.

Example 5.1.7. If the oracle f is itself a computable function, then the fcomputable functions are exactly the computable functions.

5.2 Relativization

Theorem 5.2.1 (Enumeration Theorem). Let f be a fixed oracle. For each k ≥ 0, the (k + 1)-place partial function

(e, x_{1}, …, x_{k}) ↦ φ_{e}^{(k),f}(x_{1}, …, x_{k})

is partial f-recursive.

Proof. Define the functions State^{f}(e, x_{1}, …, x_{k}, n), NextState^{f}(e, z), and StopTime^{f}(e, x_{1}, …, x_{k}) as before. These functions are partial f-recursive. The definition of NextState(e, z) has an extra clause for oracle instructions:

NextState^{f}(e, z) = z · p_{0}^{−m+n} · p_{i}^{−x+f(x)}
whenever (z)_{0} = m and (e)_{m} = 4 · 3^{i} · 5^{n} and (z)_{i} = x.

Then as before we have

φ_{e}^{(k),f}(x_{1}, …, x_{k}) ≃ (State^{f}(e, x_{1}, …, x_{,}StopTime^{f}(e, x_{1}, …, x_{k})))_{k+1}.

Theorem 5.2.2 (Parametrization Theorem). Given a 2-place partial f-recursive function ψ(w, x), we can find a 1-place total recursive function h(w) such that φ_{h(w)}^{(1),f}(x) ≃ ψ(w, x) for all w, x.

Note: We can find a function h which is recursive, not merely f-recursive.

Proof. Relativize the proof given earlier.

Definition 5.2.3 (Relativized Arithmetical Hierarchy). Let f be a fixed oracle. For k, n ≥ 1, a k-place predicate P (−) is Σ_{n}^{0,f} if P (−) can be written in the form

P (−) ≡ ∃y_{1} ∀y_{2} ⋯ \begin{matrix}∃ \\ ∀ \end{matrix} y_{n}R(−, y_{1}, y_{2}, …, y_{n})

where R is an f-recursive predicate.

(Similarly define Π_{n}^{0,f} and Δ_{n}^{0,f}.)

Theorem 5.2.4. Consider a set A ⊆ ℕ. Then A is Σ_{1}^{0,f} if and only if S = W_{n}^{f} for some n.

Proof. Relativization.

5.3 Turing degrees

Definition 5.3.1. Given two functions f, g : ℕ → ℕ, we say f is Turing reducible to g, denoted f ≤_{T} g, if there is an integer e such that f(x) = φ_{e}^{(1),g}(x) for all x. (i.e., f is g-recursive.)

The relation ≤_{T} is reflexive and transitive.

1. f ≤_{T} f

2. f ≤_{T} g, g ≤_{T} h ⇒ f ≤_{T} h

Pf: If P computes f using g as an oracle, and Q computes g using h as an oracle, then construct a program R with oracle h as follows:

Starting with the program P, but replace every oracle instruction R_{i}^{0} with the program Q (modified so that it takes its input from R_{i} and outputs back into R_{i}). Then R computes f with oracle h.

Definition 5.3.2. Two functions f, g : ℕ → ℕ are Turing equivalent if f ≤_{T} g and g ≤_{T} f. In this case, we write f ≡_{T} g. Note that ≡_{T} is an equivalence relation.

Definition 5.3.3 (Degrees of unsolvability). The Turing degrees are the set of ≡_{T} equivalence classes. If f : ℕ → ℕ, then deg_{T} (f) = {g : ℕ → ℕ | f ≡_{T} g} is the Turing degree of f.

The Turing degrees are partially ordered by Turing reducibility.

f ≤_{T} g ⇔ deg_{T} (f) ≤_{T} deg_{T} (g).

Lectures 11,12: September 19 and 20, 2007

Turing degrees are partially ordered by Turing reducibility. This partial ordering is an object of great interest in the study of unsolvable problems, and many papers have been written investigating its properties.

Basic properties:

1. There is a least Turing degree 0 = {f : ℕ → ℕ | f is recursive.}.

2. Any two Turing degrees have a least upper bound.

Proof. Let a = deg_{T} (f) and b = deg_{T} (g). Their least upper bound is sup(a, b) = deg_{T} (f ⊕ g) (a join b) where

\{ \begin{matrix}f ⊕ g(2n) = f(n) \\ f ⊕ g(2n + 1) = g(n) \end{matrix}

Clearly, f ≤_{T} f ⊕ g and g ≤_{T} f ⊕ g. Suppose f ≤_{T} h and g ≤_{T} h; then f ⊕ g ≤_{T} h by using the program for computing f on the even values and the program for computing g on the odd values.

3. Not every pair of Turing degrees has a greatest lower bound inf(a, b) (the infimum or meet of a b).

Proof. The construction of such a pair of Turing degrees is a homework problem.

4. For each f : ℕ → ℕ, there is a set A ⊆ ℕ such that deg_{T} (A) := deg_{T} (χ_{A}) = deg_{T} (f).

So, without loss of generality, we could just study the Turing degrees of sets.

Proof. Define A = {3^{x} · 5^{f(x)} | x ∈ ℕ} = G_{f}, the “graph” of f.

χ_{A} ≤_{T} f, since y ∈ A ≡ \underbrace{(\underbrace{y = 3^{(y)_{1}} · 5^{(y)_{2}}}_{recursive} ∧ \underbrace{f((y)_{1}) = (y)_{2}}_{f−recursive})}_{f−recursive}.

Note also that f ≤_{T} χ_{A}, since f(x) = μy (3^{x} · 5^{y} ∈ A).

5.4 The Turing jump operator

Problem: Given a Turing degree a = deg_{T} (f), find a Turing degree strictly greater than a. (In other words, given a function f, find a function g that is not f-recursive.)

Definition 5.4.1. The Halting problem relative to f is H^{f} = {x | φ_{x}^{(1),f}(0) ↓}.

The Turing jump operator maps deg_{T} (f) ↦ deg_{T} (H^{f}) a ↦ a′

Properties:

1. H^{f} is Σ_{1}^{0,f} complete.

2. f <_{T} H^{f} (f ≤_{T} H^{f} and H^{f} ≰_{T} f)

3. f ≤_{T} g ⇒ H^{f} ≤_{T} H^{g}

4. In particular, a′ ≥_{T} 0′ for all a

We present two theorems, without proof for now, that describe further the structure of Turing degrees and the Turing jump operator.

Theorem 5.4.2 (Friedberg’s Jump Theorem). For all Turing degrees c > 0′ (0′ = deg(H), the Turing degree of the halting problem), there exists a Turing degree a such that a′ = c.

In other words, the range of the Turing jump operator is {c | c ≥ 0′}.

Proof. See pages 31-33 of Spring 2004 lecture notes.

Exercise 5.4.3. Given Turing oracles f and g, prove that the following conditions are pairwise equivalent:

1. f ≤_{T} g 2. H^{f} ≤_{m} H^{g}

3. all partial f-recursive functions are partial g-recursive

4. all total f-recursive functions are g-recursive.

Solution. This is an exercise in understanding Turing reducibility, many-one reducibility, and the Turing jump operator. Basically, the problem is to prove that f ≤_{T} g if and only if H^{f} ≤_{m} H^{g}.

Recall that

H^{f} = the Halting Problem relative to the oracle f
= {x | φ_{x}^{(1),f}(0) ↓}

is the “relativization to f” of H = {x | φ_{x}^{(1)}(0) ↓} = the Halting Problem.

We have already implicitly proved that the Halting Problem is unsolvable, but let us now make this explicit:

Theorem 5.4.4. H is Σ_{1}^{0} complete, hence not recursive.

Proof. We used the Parametrization Theorem to prove that H is Σ_{1}^{0}-complete. It follows that H is not Δ_{1}^{0}, hence not recursive.

We now relativize this to H^{f} to obtain:

Theorem 5.4.5. H^{f} is not f-recursive. I.e., H^{f} ≰_{T} f. Note also that f ≤_{T} H^{f}

Proof. The point is that H^{f} is Σ_{1}^{0,f} complete, hence not Δ_{1}^{0,f}, hence not ≤_{T} f. This is just the relativization to f of the previous theorem.

It remains to show that f ≤_{T} H^{f}. By definition, for any set A ⊆ ℕ, f ≤_{T} A means f ≤_{T} χ_{A}, also deg_{T} (A) = deg_{T} (χ_{A}), etc.

For any function f, define the “graph” of f to be the set G_{f} = {3^{x}5^{y} | f(x) = y}. Then f ≡_{T} G_{f}, because f(x) = μy 3^{x}5^{y} ∈ G_{f}, and z ∈ G_{f} ≡ (∃x < z) (∃y < z) (z = 3^{x}5^{y} ∧ y = f(x)). It follows that G_{f} is Δ_{1}^{0,f}, so it is Σ_{1}^{0,f}. Hence G_{f} ≤_{m} H^{f}, hence G_{f} ≤_{T} H^{f}, hence f ≤_{T} H^{f}.

Summary: For any Turing oracle f, we have H^{f} >_{T} f.

In the above proof, we used the fact that many-one reducibility is a special case of Turing reducibility. Formally,

Lemma 5.4.6. A ≤_{m} B implies A ≤_{T} B.

Proof. Suppose A ≤_{m} B via a function h. Then χ_{A}(x) = χ_{B}(h(x)). h is recursive, so χ_{A} is χ_{B}-recursive.

Definition 5.4.7. The Turing jump operator is the operator which takes f to H^{f}.

Thus the Turing jump operator is simply the relativization of the Halting Problem. If a = deg_{T} (f), then we write a′ = deg_{T} (H^{f}). Note that a < a′ for all Turing degrees a. In other words, given any unsolvable problem, the jump operator gives us a problem which is “more unsolvable,” i.e., its degree of unsolvability is greater.

In particular, starting with the Turing degree 0 = deg_{T} (0) where 0 denotes any recursive function, we may iterate the jump operator to obtain an ascending sequence of Turing degrees 0 < 0′ < 0′′ < 0′′′ < ⋯ < 0^{(n)} < 0^{(n+1)} < ⋯ which represent higher and higher degrees of unsolvability. It can be shown that this sequence of Turing degrees is closely related to the arithmetical hierarchy. See the discussion of Post’s Theorem, below.

To solve Problem 1, use the following facts:

1. H^{f} is complete Σ_{1}^{0,f}.

2. A set A is Σ_{1}^{0,f} if and only if A is the range of a partial f-recursive function.

3. A 1-place partial function ψ(x) is partial f-recursive if and only if G_{ψ}, the graph of ψ, is Σ_{1}^{0,f}. Here of course G_{ψ} = {3^{x}5^{y} | ψ(x) ≃ y}.

All of these are straightforward relativizations of known facts about Σ_{1}^{0} sets and partial recursive functions.

5.5 Finite approximation

Definition 5.5.1. A string is a finite sequence of natural numbers.

Notation 5.5.2. We use Greek letters such as σ, τ, … to denote strings. The length of a string σ will be denoted |σ|. The elements of σ will be denoted σ(i) for i < |σ|. So, if |σ| = m, then

σ = ⟨σ(0), σ(1), …, σ(m − 1)⟩ .

We write σ ⊆ τ if and only if |σ| ≤ |τ| and σ(i) = τ(i) for all i < |σ|. The concatenation of σ and τ is

σ^{⌢}τ = ⟨σ(0), σ(1), …, σ(m − 1), τ(0), τ(1), …, τ(n − 1)⟩

where |σ| = m and |τ| = n. Note that σ^{⌢}τ is a string of length m + n.

Since strings are finite objects, they can be Gödel numbered. For concreteness we choose the Gödel numbering

#(σ) = ∏_{i<|σ|} p_{i}^{σ(i)+1}

noting that

# : {strings} ⟶^{1−1} ℕ .

For example, if σ = ⟨3, 8, 11⟩, then |σ| = 3, σ(0) = 3, σ(1) = 8, σ(2) = 11, and p_{0} = 2, p_{1} = 3, p_{2} = 5, so #(σ) = 2^{4}3^{9}5^{12}.

Note that the set of Gödel numbers of strings is recursive (use bounded quantification, etc.). Moreover, |σ| is recursive as a function of #(σ), and #(σ^{⌢}τ) is recursive as a function of #(σ) and #(τ). Also, σ(i) is recursive as a function of #(σ) and i, σ ⊆ τ is recursive as a predicate on #(σ) and #(τ), etc.

We now introduce the important concept of finite approximations.

If f is an oracle, let

f ↾ n = ⟨f(0), f(1), …, f(n − 1)⟩ .

We call this string f ↾ n a finite approximation of f. Note that f is the union of its finite approximations. We write σ ⊂ f to mean that the string σ is a finite approximation of the oracle f, i.e., σ = f ↾ n for some n.

In general, an oracle contains an infinite amount of information: f(0), f(1), f(2), f(3), f(4), . . .. However, in any particular halting computation

φ_{e}^{(1),f}(x) ≃ y

only finitely much information from the oracle is used, since the computation halts in only a finite number of steps and consults the oracle only a finite number of times. We use the following notation to describe oracle computations that use only a finite approximation to f.

Notation 5.5.3. φ_{e,s}^{(k),σ}(x_{1}, …, x_{k}) ≃ y means:

for any or all oracles f approximated by σ, φ_{e}^{(k),f}(x_{1}, …, x_{k}) ≃ y in less than or equal to s steps of computation using only oracle information from σ.

We write φ_{e}^{(k),σ}(x_{1}, …, x_{k}) ≃ y as an abbreviation for φ_{e,|σ|}^{(k),σ}(x_{1}, …, x_{k}) ≃ y.

Lemma 5.5.4. The following are pairwise equivalent:

1. φ_{e}^{(k),f}(x_{1}, …, x_{k}) ≃ y.

2. φ_{e,s}^{(k),f↾n}(x_{1}, …, x_{k}) ≃ y for some n and s.

3. φ_{e,s}^{(k),f↾n}(x_{1}, …, x_{k}) ≃ y for all sufficiently large n and s.

4. φ_{e}^{(k),f↾n}(x_{1}, …, x_{k}) ≃ y for some n.

5. φ_{e}^{(k),f↾n}(x_{1}, …, x_{k}) ≃ y for all sufficiently large n.

6. φ_{e,s}^{(k),σ}(x_{1}, …, x_{k}) ≃ y for some string σ ⊂ f and some s.

7. φ_{e,s}^{(k),σ}(x_{1}, …, x_{k}) ≃ y for all sufficiently long strings σ ⊂ f and all sufficiently large s.

8. φ_{e}^{(k),σ}(x_{1}, …, x_{k}) ≃ y for some string σ ⊂ f.

9. φ_{e}^{(k),σ}(x_{1}, …, x_{k}) ≃ y for all sufficiently long strings σ ⊂ f.

Proof. Obvious.

Note also that the above-mentioned predicates on strings have a monotonicity property:

Lemma 5.5.5. As usual let us write − instead of x_{1}, …, x_{k}.

1. If σ ⊆ τ and s ≤ t, then φ_{e,s}^{(k),σ}(−) ≃ y implies φ_{e,t}^{(k),τ} (−) ≃ y.

2. If σ ⊆ τ and s ≤ t, then φ_{e,s}^{(k),σ}(−) ↓ implies φ_{e,t}^{(k),τ} (−) ↓.

3. If σ ⊆ τ then φ_{e}^{(k),σ}(−) ≃ y implies φ_{e}^{(k),τ} (−) ≃ y.

4. If σ ⊆ τ then φ_{e}^{(k),σ}(−) ↓ implies φ_{e}^{(k),τ} (−) ↓.

Proof. Obvious.

Some of the usefulness of finite approximations lies in the fact that certain predicates associated with them are recursive. Namely:

Lemma 5.5.6. Let us write σ instead of #(σ), the Gödel number of σ. As usual we write − instead of x_{1}, …, x_{k}.

1. The (k + 4)-place predicate {⟨e, s, σ, −, y⟩ | φ_{e,s}^{(k),σ}(−) ≃ y} is recursive.

2. The (k + 3)-place predicate {⟨e, σ, −, y⟩ | φ_{e}^{(k),σ}(−) ≃ y} is recursive.

3. The (k + 3)-place predicate {⟨e, s, σ, −⟩ | φ_{e,s}^{(k),σ}(−) ↓} is recursive.

4. The (k + 2)-place predicate {⟨e, σ, −⟩ | φ_{e}^{(k),σ}(−) ↓} is recursive.

Proof. Let NextState^{σ}(e, z) be the finite approximation version of the NextState function. Note thatNextState^{σ}(e, z) is recursive as a function of the three variables #(σ), e, z. The only difference is that its definition now has an extra clause for oracle instructions:

NextState^{σ}(e, z) = z · p_{0}^{−m+n} · p_{i}^{−x+σ(x)}
whenever (z)_{0} = m and (e)_{m} = 4 · 3^{i} · 5^{n} and (z)_{i} = x and x < |σ|.

By recursion on n, it follows that State^{σ}(e, x_{1}, …, x_{k}, n) is recursive as a function of #(σ), e, x_{1}, …, x_{k}, n. Hence, the predicates

φ_{e,s}^{(k),σ}(x_{1}, …, x_{k}) ≃ y ≡ (∃n ≤ s) [ (State^{σ}(e, x_{1}, …, x_{k}, n))_{0} = 0
∧ (State^{σ}(e, x_{1}, …, x_{k}, n))_{k+1} = y]

and

φ_{e,s}^{(k),σ}(x_{1}, …, x_{k}) ↓ ≡ (∃n ≤ s) [ (State^{σ}(e, x_{1}, …, x_{k}, n))_{0} = 0 ]

etc., are recursive.

The method of finite approximation is a valuable tool in studying the structure of the Turing degrees. For example:

Theorem 5.5.7 (Kleene/Post). There exist incomparable Turing degrees below 0′. That is, there are a, b ≤ 0′ such that a ≰ b and a ≰ b.

Proof. By finite approximation.

We will build sets A, B ⊆ ℕ so that a = deg_{T} A and b = deg_{T} B, by defining longer and longer strings σ_{n}, τ_{n} ∈ 2^{<ℕ} with σ_{n} ⊆ σ_{n+1} and τ_{n} ⊆ τ_{n+1}. Then

f = χ_{A} = ⋃_{n=1}^{∞} σ_{n} and g = χ_{B} = ⋃_{n=1}^{∞} τ_{n}.

The condition a ≰ b requires that for all e, f ≠ φ_{e}^{(1),g}; the condition b ≰ a requires that for all e, g ≠ φ_{e}^{(1),f}. We will take care of one of these infinitely many requirements at each step of our construction.

Start with σ_{0}, τ_{0} = ⟨⟩ (the empty string); this is stage n = 0. Stage n = 2e + 1: Case 1: If it exists, find the least (by Gödel number) σ ∈ 2^{<ℕ} extending σ_{2e} such that φ_{e}^{(1),σ}(|τ_{2e}|) ≃ 1. Then set σ_{2e+1} = σ and τ_{2e+1} = τ_{2e}^{⌢}⟨0⟩. Case 2: Otherwise, set σ_{2e+1} = σ_{2e} and τ_{2e+1} = τ_{2e}^{⌢}⟨1⟩. The even-numbered stages will be the same as the odd-numbered stages,

except that the roles of σ_{n} and τ_{n} will be interchanged. In detail:

Stage n = 2e + 2: Case 1: If it exists, find the least (by Gödel number) τ ∈ 2^{<ℕ} extending τ_{2e+1} such that φ_{e}^{(1),τ} (|σ_{2e+1}|) ≃ 1. Set τ_{2e+2} = τ and σ_{2e+2} = σ_{2e+1}^{⌢}⟨0⟩. Case 2: Otherwise, set τ_{2e+2} = τ2e + 1 and σ_{2e+2} = σ_{2e+1}^{⌢}⟨1⟩.

Finally, define f = χ_{A} = ⋃_{n=1}^{∞} σ_{n} and g = χ_{B} = ⋃_{n=1}^{∞} τ_{n}.

Claim: f ≰_{T} g. To prove this claim, consider any e ∈ ℕ. We will show that it is not the case

that f = φ_{e}^{(1),g}. Consider step 2e + 2. Let x = |σ_{2e+1}|.

If f(x) = 0, then step 2e + 2 was case 1. Then φ_{e}^{(1),g}(x) ≃ φ_{e}^{(1),τ} (x) ≃ 1, so f ≄ φ_{e}^{(1),g}.

If f(x) = 1, then step 2e + 2 was case 2. That means there was no extension τ of τ_{2e+1} so that φ_{e}^{(1),τ} (x) ≃ 1. But g is an extension of τ_{2e+1}, so we know that f ≄ φ_{e}^{(1),g}.

Claim: g ≰_{T} f. This is proved by mirroring the argument for the previous claim. Claim: deg_{T} (f) ≤ 0′ and deg_{T} (g) ≤ 0′ This is because the entire construction is recursive relative to the Halting

problem, i.e., 0′-recursive.

Everything is recursive except the division into cases 1 and 2. The search for extensions σ and τ might never halt. Use the oracle 0′ to determine whether this search will halt. If it halts, use case 1; if it does not, use case 2.

5.6 Structure of the Turing degrees

In this section we present some exercises where the structure of the Turing degrees is further explored.

Given two Turing degrees a and b, we know that the least upper bound sup(a, b) always exists. Exercises 5.6.1, 5.6.4, and 5.6.8 below show that the greatest lower bound inf(a, b) sometimes exists and sometimes does not exist.

Exercise 5.6.1. Use the method of finite approximation to construct Turing degrees a, b such that a > 0 and b > 0 and inf(a, b) = 0.

Solution. The problem is to find a, b such that a, b > 0 and inf(a, b) = 0.

So, we want to find f, g such that f ≰_{T} g and g ≰_{T} f (this requirement is as before) and in addition, if h ≤_{T} f and h ≤_{T} g then h is recursive.

Thus, for each pair of indices i, j we have a new requirement:

if φ_{i}^{(1),f}(x) is total and φ_{j}^{(1),g}(x) is total and they are the same total function, then that function is recursive.

Here are some possible ways to bring this about at stage n:

1. ∃x ¬ ∃σ ⊇ σ_{n} ∃s φ_{i,s}^{(1),σ}(x) ↓.

[In this case, φ_{i}^{(1),f} is not total.]

2. ∃x ¬ ∃τ ⊇ τ_{n} ∃s φ_{j,s}^{(1),τ} (x) ↓.

[In this case, φ_{j}^{(1),g}(x) is not total.]

3. ∃x ∃σ ⊇ σ_{n} ∃τ ⊇ τ_{n} ∃s φ_{i,s}^{(1),σ}(x) ↓≠ φ_{j,s}^{(1),τ} (x) ↓.

[This insures that they are not the same total function.]

4. φ_{i}^{(1),f}(x) is total and recursive.

Now for the actual construction:

Stage n + 1 where n = 4 · 3^{i}5^{j}:

We are given σ_{n}, τ_{n}.

Case 1: ∃x ∃σ ⊇ σ_{n} ∃τ ⊇ τ_{n} ∃s φ_{i,s}^{(1),σ}(x) ↓ ≠ φ_{j,s}^{(1),τ} (x) ↓

In this case choose σ, τ as above and let σ_{n+1} = σ and τ_{n+1} = τ.

Case 2: Not case 1.

In this case let σ_{n+1} = σ_{n} and τ_{n+1} = τ_{n}.

Finally let f = ⋃_{n=0}^{∞} σ_{n} and g = ⋃_{n=0}^{∞} τ_{n}.

We need to prove that our requirement is satisfied. Suppose h(x) = φ_{i}^{(1),f}(x) = φ_{j}^{(1),g}(x) are the same total function. We need to prove that this function is recursive.

To see this, let n = 4 · 3^{i}5^{j} and consider what happened at stage n + 1 of the construction. If Case 1 happened, then we have an x such that φ_{i}^{(1),σ_{n+1}}(x) ↓ ≠ φ_{j}^{(1),τ_{n+1}}(x) ↓, hence φ_{i}^{(1),f}(x) and φ_{j}^{(1),g}(x) could not be the same total function. So, Case 2 must have happened at stage n + 1. In this case we claim that h(x) is computable. Namely, given x, to compute h(x), search for σ ⊇ σ_{n} and s such that φ_{i,s}^{(1),σ}(x) ↓. Then h(x) = φ_{i,s}^{(1),σ}(x), since there is exactly one possible value that we can get, no matter which extension σ we choose. If there were more than one possible value, then Case 1 would have happened at this stage.

This completes our sketch of the solution.

Exercise 5.6.2. Use the method of finite approximation to construct Turing degrees a, b such that a < 0′ and b < 0′ and sup(a, b) = 0′.

Exercise 5.6.3. Combine Exercises 5.6.1 and 5.6.2 to prove the following:

There exist two incomprable Turing degrees a, b such that inf(a, b) = 0 and sup(a, b) = 0′.

Exercise 5.6.4. Generalize Exercise 5.6.3 to prove the following:

Given two Turing degrees c, d such that c′ ≤ d, we can find two incomparable Turing degrees a, b such that inf(a, b) = c and sup(a, b) = d.

Remark 5.6.5 (iterating the Turing jump operator). For any Turing oracle f we have

f′ = H^{f} = {x | φ_{x}^{(1),f}(0) ↓} = the Halting Problem relative to f.

We know that f′ is a complete Σ_{1}^{0} set relative to the oracle f. For any Turing degree a = deg_{T} (f) we define

a′ = deg_{T} (f′) = the Turing jump of a.

Clearly a < a′ holds for all a. Thus, starting with any Turing degree a, we have an ascending sequence of Turing degrees

a < a′ < a′′ < ⋯ < a^{(n)} < a^{(n+1)} < ⋯

In particular, starting with the zero Turing degree 0, we have the ascending sequence

0 < 0′ < 0′′ < ⋯ < 0^{(n)} < 0^{(n+1)} < ⋯

corresponding to the arithmetical hierarchy.

Exercise 5.6.6. Prove the following result.

Given an ascending sequence of Turing degrees

d_{0} < d_{1} < ⋯ < d_{n} < d_{n+1} < ⋯

we can find a pair of Turing degrees a, b such that for all Turing degrees c

∃n (c ≤ d_{n}) if and only if c ≤ a and c ≤ b .

Exercise 5.6.7. Use the result of Exercise 5.6.6 to prove that no ascending sequence of Turing degrees has a least upper bound.

Exercise 5.6.8. For any pair of Turing degrees a, b as in Exercise 5.6.6, prove that the greatest lower bound inf(a, b) does not exist.

5.7 Homework #5, due October 1, 2007

Exercise 5.7.1. Recall that a simple r.e. set is neither recursive nor many-one complete. Use Post’s Theorem plus relativization to generalize this to higher levels of the arithmetical hierarchy.

Conclude that for each n ≥ 1 there exist Σ_{n}^{0} sets which are neither many-one complete (within the class of Σ_{n}^{0} sets) nor Δ_{n}^{0}.

Solution. The problem is to prove that for each n ≥ 1 there exist Σ_{n}^{0} sets which are neither Δ_{n}^{0} nor Σ_{n}^{0}-complete.

For n = 1, let A be a simple set. We have seen in a previous homework that A is not recursive and not Σ_{1}^{0}-complete.

For n = 2, it does not work to take A′ = the jump of A, where A is a simple set. The set A′ is indeed Σ_{2}^{0}, but it could be Σ_{2}^{0}-complete. For instance, suppose A is a simple set which is ≡_{T} K. Such A’s exist by Problem 4. Then A′ ≡_{m} K′ is Σ_{2}^{0}-complete.

For a correct solution, use the following definition.

Definition 5.7.2. A set A is n-simple if A is Σ_{n}^{0}, A^{c} is infinite, and A^{c} does not include an infinite Σ_{n}^{0} set.

Post’s Theorem says that Σ_{n}^{0} is the same as Σ_{1}^{0,0^{(n−1)}}. By relativizing the usual facts about simple sets to 0^{(n−1)}, we see that there exists an n-simple set and any such set is neither Σ_{n}^{0}-complete nor Δ_{n}^{0}.

Exercise 5.7.3.

1. Given a Σ_{1}^{0} predicate P (x, y) such that ∀x ∃y P (x, y) holds, prove that there exists a recursive function f(x) such that ∀x P (x, f(x)) holds.

2. Use Post’s Theorem plus relativization to generalize the previous result to higher levels of the arithmetical hierarchy.

Conclude that for all n ≥ 1, given a Σ_{n}^{0} predicate P (x, y) such that ∀x ∃y P (x, y) holds, there exists a Δ_{n}^{0} function f(x) such that ∀x P (x, f(x)) holds.

Solution. Part 1: Given a Σ_{1}^{0} predicate P (x, y) such that ∀x ∃y P (x, y), to find a recursive selector, i.e., a recursive function f(x) such that ∀x P (x, f(x)).

A wrong construction is f(x) = μy P (x, y). The wrong thing about this is that we are applying the μ operator to a nonrecursive predicate.

A correct construction is as follows. Let P (x, y) ≡ ∃z R(x, y, z) where R is a recusive predicate. Let g(x) = least 3^{y} · 5^{z} such that R(x, y, z) holds. Then, let f(x) = (g(x))_{1}.

Part 2: Using Post’s Theorem, this generalizes as follows. Given a Σ_{n}^{0} predicate P (x, y) such that ∀x ∃y P (x, y), we can find a Δ_{n}^{0} selector.

Note also that we can drop the hypothesis ∀x ∃y P (x, y). In this case the same construction gives a partial recursive selector, i.e., a partial recursive function ψ(x) such that for all x, if ∃y P (x, y) then ψ(x) is defined and is such a y. In the generalization to Σ_{n}^{0} predicates, we get a selector whose graph is Σ_{n}^{0}.

Here is an alternative solution.

Solution. Given a Σ_{1}^{0} predicate P (x, y) such that ∀x ∃y P (x, y), let P (x, y) ≡ ∃z R(x, y, z) with R recursive and define

f(x) ≃ (μw R(x, (w)_{1}, (w)_{2}))_{1} .

Then clearly f(x) is recursive and ∀x P (x, f(x)).

Now, imitate the above argument with Σ_{n}^{0} instead of Σ_{1}^{0}. This will give a direct solution without using Post’s Theorem. Assume that P (x, y) is Σ_{n}^{0} and ∀x ∃y P (x, y). Then P (x, y) ≡ ∃z R(x, y, z) where R is Π_{n−1}^{0}. Define f(x) as above. We need to show that f is Δ_{n}^{0}. We have f(x) = y ≡ ∃w [ (R(x, (w)_{1}, (w)_{2})∧¬ ∃v < w R(x, (v)_{1}, (v)_{2}))∧y = (w)_{1} ] so the graph of f is Σ_{n}^{0}. Similarly f(x) = y ≡ ∀w [ R(x, (w)_{1}, (w)_{2}) ∧ ¬ ∃v < w R(x, (v)_{1}, (v)_{2}) ] ⇒ y = (w)_{1} ] so the graph of f is Π_{n}^{0}. Thus the graph of f is Δ_{n}^{0}, i.e., f is Δ_{n}^{0}.

Exercise 5.7.4. Given a nonrecursive recursively enumerable set A, prove that we can find a simple set B such that A ≡_{T} B.

Hint: Use a deficiency set.

Exercise 5.7.5. Prove the followng theorem:

Given a Turing degree d ≥ 0′, we can find a Turing degree a such that a′ = d.

Thus, the range of the Turing jump operator consists precisely of the Turing degrees which are ≥ 0′.

Hint: Use the technique of finite approximation.

Solution. The problem was to prove the following theorem of Friedberg:

Given a Turing degree d ≥ 0′, to find a Turing degree a such that a′ = d.

In other words, given g ≥_{T} H where H is the Halting Problem, to find f such that H^{f} ≡_{T} g.

We construct f by finite approximations, f = ⋃_{n=0}^{∞} σ_{n}, σ_{0} ⊆ σ_{1} ⊆ ⋯ ⊆ σ_{n} ⊆ · · ·. Our strategy is, at even-numbered stages control H^{f}, and at oddnumbered stages code in g.

Stage 0: Let σ_{0} = ⟨⟩, the empty string.

Stage 2e + 1: Let σ_{2e+1} = σ_{2e}^{⌢}⟨g(e)⟩. The purpose here is to code in some information about g.

Stage 2e + 2:

Case 1: (∃σ ⊇ σ_{2e+1}) (φ_{e}^{(1),σ}(0) ↓). In this case choose the least such σ (according to the Gödel numbering of strings) and let σ_{2e+2} = σ. Thus we have forced e ∈ H^{f}, recalling that H^{f} = {e | φ_{e}^{(1),f}(0) ↓}.

Case 2: Not case 1. In this case let σ_{2e+2} = σ_{2e+1}. The purpose of this case distinction is to control the jump of f. The construction insures that e ∈ H^{f} if and only if Case 1 held at Stage 2e + 2.

We claim that the entire construction (i.e., the 1-place total function c where c(n) = #(σ_{n}) for all n) is ≤_{T} each of the oracles g, H ⊕ f, H^{f}.

To prove the claim, let J = {⟨e, τ⟩ | (∃σ ⊇ τ) (φ_{e}^{(1),σ}(0) ↓)}. This J is exactly the oracle that we need in order to distinguish between Case 1 and Case 2 at Stage 2e + 2. Namely, we are in Case 1 if and only if ⟨e, σ_{2e+1}⟩ ∈ J. Once we know which case we are in, we can compute σ_{2e+2} recursively given σ_{2e+1}. Now observe that J is Σ_{1}^{0}. Since H is Σ_{1}^{0} complete, it follows that J ≤_{m} H, hence J ≤_{T} H. Since H ≤_{T} g, it follows that the entire construction is ≤_{T} g. Also, the entire construction is ≤_{T} H ⊕ f, because at Stage 2e + 1 we have σ_{2e+1} = σ_{2e}^{⌢}⟨f(|σ_{2e}|)⟩. Since the entire construction is ≤_{T} H ⊕ f, it is also ≤_{T} H^{f}, because obviously H ⊕f ≤_{T} H^{f}. This completes the proof of the claim.

Note also that H^{f} ≤_{T} the entire construction. This is clear, because e ∈ H^{f} if and only if Case 1 held at Stage 2e + 2, i.e., if and only if φ_{e}^{(1),σ_{2e+2}}(0) ↓.

Finally, note that g ≤_{T} the entire construction, because g(e) = σ_{2e+1}(|σ_{2e}|). Putting everything together, we have g ≡_{T} H ⊕ f ≡_{T} H^{f} ≡_{T} the entire

construction. Letting a = deg_{T} (f), it follows that d = sup(0′, a) = a′. This completes the proof.

Lecture 14: September 24, 2007

5.8 Comments on Homework #4, continued

Finite approximation: additional explanation

A basic method in the study of Turing degrees is finite approximation. Let σ be a string. The technique of finite approximation says: φ_{e}^{(1),g}(x) ≃ y iff

(∃s) (∃σ a finite approximation to g) \underbrace{[φ_{e,s}^{(1),σ}(x) ≃ y]}_{this predicate is recursive!} .

The finite approximation method consists of defining functions f, g by f = ⋃_{n=0}^{∞} σ_{n} and g = ⋃_{n=0}^{∞} τ_{n} where σ_{0} ⊆ σ_{1} ⊆ σ_{2} ⊆ ⋯ ⊆ σ_{n} ⊆ ⋯ and τ_{0} ⊆ τ_{1} ⊆ τ_{2} ⊆ … ⊆ τ_{n} ⊆ . . .. This method was originally introduced by Kleene and Post.

At each step n, we choose extensions σ_{n+1} and τ_{n+1} to accomplish some requirement which we want to hold for f, g.

As a first application of the method, we prove the following theorem.

Theorem 5.8.1. There exist Turing degrees a, b such that a ≰ b and b ≰ a. (I.e., they are incomparable.)

In other words, there exist 1-place total functions f, g such that f ≰_{T} g and g ≰_{T} f.

Note that the requirement f ≰_{T} g can be broken down into a countable family of requirements. For each index e, we require: f(x) ≄ φ_{e}^{(1),g}(x) for some x. This requirement can be satisfied in either of two ways: φ_{e}^{(1),g}(x) ↑, or φ_{e}^{(1),g}(x) ↓ ≠ f(x).

Proof. We shall construct f and g by finite approximation. This means that, by induction on n, we shall construct infinite increasing sequences of strings

σ_{0} ⊆ σ_{1} ⊆ σ_{2} ⊆ σ_{3} ⊆ … ⊆ σ_{n} ⊆ …

and

τ_{0} ⊆ τ_{1} ⊆ τ_{2} ⊆ τ_{3} ⊆ … ⊆ τ_{n} ⊆ …

and after that we shall define f = ⋃_{n=0}^{∞} σ_{n} and g = ⋃_{n=0}^{∞} τ_{n}.

The construction:

Stage 0: Let σ_{0} = τ_{0} = ⟨⟩ = the empty string = the unique string of length 0.

Stage 2e+1: We may assume inductively that σ_{2e} and τ_{2e} are already known. Let x = |σ_{2e}|.

Case 1: ∃s ∃τ ⊇ τ_{2e} such that φ_{e,s}^{((1),τ} (x) ↓. In this case, choose such s, τ. (For example, we could choose the least such pair (s, τ) according to some fixed Gödel numbering of pairs.) Then define y = φ_{e,s}^{(1),τ} (x) and σ_{2e+1} = σ_{2e}^{⌢}⟨y + 1⟩ and τ_{2e+1} = τ.

Case 2: Not case 1. In this case, let σ_{2e+1} = σ_{2e} and τ_{2e+1} = τ_{2e}.

Stage 2e + 2: We proceed as in Stage 2e + 1 except that the roles of f and g are reversed. Here are the details. We may assume inductively that σ_{2e+1} and τ_{2e+1} are already known. Let x = |τ_{2e+1}|.

Case 1: ∃s ∃σ ⊇ σ_{2e+1} such that φ_{e,s}^{((1),σ}(x) ↓. In this case, choose such s, σ and define y = φ_{e,s}^{(1),σ}(x) and τ_{2e+2} = τ_{2e+1}^{⌢}⟨y + 1⟩ and σ_{2e+2} = σ.

Case 2: Not case 1. In this case, let σ_{2e+2} = σ_{2e+1} and τ_{2e+2} = τ_{2e+1}.

This completes the construction.

As already mentioned, we now define f = ⋃_{n=0}^{∞} σ_{n} and g = ⋃_{n=0}^{∞} τ_{n}.

We claim that f ≰_{T} g and g ≰_{T} f. To prove the claim, we argue by contradiction.

Suppose for instance that f ≤_{T} g. Then, there exists e such that f(x) = φ_{e}^{(1),g}(x) for all x. For this particular e, consider what happened at stage 2e + 1 of the construction. Let x be as in stage 2e+1, i.e., x = |σ_{2e}|. For this particular x, let n and s be sufficiently large so that φ_{e,s}^{(1),g↾n}(x) ↓ and n ≥ |τ_{2e}|. Letting τ = g ↾ n, we see that τ_{2e} ⊆ τ and φ_{e,s}^{(1),τ} (x) ↓. Thus we see that Case 1 holds. Therefore, let s and τ be as chosen in Case 1, and let y = φ_{e,s}^{(1),τ} (x). (Note that this s and τ may be different from the previous s and τ). Then, by construction, we have f(x) = σ_{2e+1}(x) = y+1 and τ_{2e+1} = τ and φ_{e}^{(1),g}(x) = φ_{e,s}^{(1),τ_{2e+1}}(x) = y, a contradiction. Thus we have proved that f ≰_{T} g.

The proof that g ≰_{T} f is similar, looking at stage 2e + 2 instead of stage 2e + 1.

Lecture 15: September 26, 2007

Hint for Homework #4, Problem 3

We want sup(a, b) = 0′. Construct f, g by finite approximation. We need

1. f ≰_{T} g and g ≰_{T} f.

2. f ⊕ g ≤_{T} H

3. H ≤_{T} f ⊕ g

We have done (1).

Essentially, we have done (2). In our finite approximation constructions, f = ⋃_{n=1}^{∞} σ_{n} where σ_{0} ⊆ σ_{1} ⊆ … ⊆ σ_{n} ⊆_{n+1}⊆ …

Stage n+1: Given σ_{n}, construct σ_{n+1}. We divide our constructions into two cases:

Case 1: ∃σ ⊇ σ_{n} ∃x ∃s (φ_{e,s}^{(1),σ}(x) ↓)

Case 2: Not case 1

Note that the division into cases is not computable. However, it is Σ_{1}^{0}. Since the Halting Problem is Σ_{1}^{0} complete, the division into cases is computable relative to the Halting Problem (0′). Thus, the construction of f is computable from 0′; that is f ≤_{T} H.

Hint for Homework #4, Problems 4 and 5

Problem 4 combines and generalizes problems 2 and 3. We need inf(a, b) = 0 and sup(a, b) = 0′. Then relativize to get inf(a, b) = c and sup(a, b) = d. Instead of finite approximation, use coinfinite approximation.

Definition 5.8.2. A coinfinite condition is a 1-place partial function, p, such that dom(p) is recursive and coinfinite.

For problems 4 and 5, construct sequences of coinfinite conditions p_{0} ⊆ p_{1} ⊆ p_{2} ⊆ ⋯ ⊆ p_{n} ⊆ p_{n+1} ⊆ ⋯ and let f = ⋃_{n=0}^{∞} p_{n}. Similarly, construct q_{0} ⊆ q_{1} ⊆ q_{2} ⊆ … ⊆ q_{n} ⊆ … and let g = ⋃_{n=0}^{∞} q_{n}. Here p ⊆ q means dom(p) ⊆ dom(q) and p(x) = q(x) for all x ∈ dom(p). It is also useful to define p ⊆_{fin} q to mean p ⊆ q and dom(q) \ dom(q) is finite.

For problem 4 use coinfinite conditions which are ≤_{T} C where c = deg_{T} (C). For problem 5 use coinfinite conditions which are ≤_{T} D_{n} for some n, where

d_{n} = deg_{T} (D_{n}).

5.9 Review for the upcoming Midterm Exam

Some topics to study are:

1. r.e. sets (various characterizations)

2. creative sets, simple sets, deficiency sets

3. Post’s Theorem

4. Parametrization Theorem, uniformity

5. the diamond: \begin{matrix}0′ \\ / \ \\ a b \\ \ / \\ 0 \end{matrix}

6. coinfinite approximation

Post’s Theorem

As usual abbreviate a k-place predicate P (x_{1}, …, x_{k}) as P (−). Recall that P is a subset of ℕ^{k}.

Theorem 5.9.1 (Post’s Theorem). P (−) is Σ_{n}^{0} if and only if P (−) is Σ_{1}^{0,0^{(n−1)}}.

n = 1: trivial.

n = 2: P (−) is Σ_{2}^{0} ⇔ P (−) is Σ_{1}^{0} relative to 0′ = the Halting Problem. An interesting consequence is: A is Δ_{2}^{0} if and only if A ≤_{T} the Halting

Problem.

Proof. We omit the proof; it is by finite approximation to 0^{(n−1)}.

The point of Post’s Theorem is that many properties of Σ_{1}^{0} sets easily relativize to become properties of Σ_{n}^{0} sets. Of course, we understand Σ_{1}^{0} sets very well, because they are the same thing as recursively enumerable sets. Post’s Theorem says that our detailed understanding of Σ_{1}^{0} sets applies also to higher levels of the arithmetical hierarchy.

Deficiency sets

Consider a 1 − 1 total recursive function f. If f is monotone increasing (x < y ⇒ f(x) < f(y)), then the range of f is recursive (not just Σ_{1}^{0}). Namely,

\begin{aligned}z ∈ range of f & ≡ & ∃xf(x) = z \\ & ≡ & ∃x ≤ zf(x) = z \end{aligned}

In general, the range of a recursive 1 − 1 function need not be recursive. In fact, any nonrecursive r.e. set is the range of a 1-1 total recursive function.

We define the deficiency set of f to be D_{f} = {x | ∃y (x < y ∧ f(y) < f(x)}. The set D_{f} measures the failure of f to be monotone increasing. Note that D_{f} is Σ_{1}^{0} by definition. It is easy to show that if rng(f) is recursive then D_{f} is recursive. Moreover, if rng(f) is nonrecursive, then D_{f} is norecursive, and in fact D_{f} is simple.

Let us show that if rng(f) is nonrecursive then D_{f} is simple. We have to show three things:

1. D_{f} is Σ_{1}^{0} (obvious).

2. D_{f} is coinfinite.

3. D_{f} is not disjoint from any infinite r.e. set.

If D_{f} is cofinite, then for all sufficiently large x we have x ∈ D_{f}, which means ∃y > x f(y) < f(x). Choose x_{0} to be such a sufficiently large x. Then we can find x_{1} > x_{0} with f(x_{1}) < f(x_{0}). Then we can find x_{2} > x_{1} with f(x_{2}) < f(x_{1}). Then we can find x_{3} > x_{2} with f(x_{3}) < f(x_{2}). . . . . This gives an infinite descending sequence of natural numbers: f(x_{0}) > f(x_{1}) > f(x_{2}) > · · ·. No such sequence exists, so D_{f} is coinfinite.

Now suppose B ∩D_{f} = ∅, where B is infinite Σ_{1}^{0}. We then argue that rng(f) is recursive. Namely, to decide whether z ∈ rng(f), search for x ∈ B such that f(x) > z. But x ∈ B implies x ∉ D_{f}, i.e., f(y) > f(x) for all y > x. So, z ∈ rng(f) ≡ ∃y ≤ x z = f(y).

Here is a more formal version of the proof. Suppose B is Σ_{1}^{0} and disjoint from D_{f}. Since B is Σ_{1}^{0}, we have x ∈ B ≡ ∃y R(x, y), where R is recursive. Consider the total recursive function g(z) = (μw (R((w)_{1}, (w)_{2}) ∧ f((w)_{1}) > z))_{1}. Then g(z) = x where x ∈ B and f(x) > z. Then, as we found above, z ∈ rng(f) ≡ [z = f(y) for some y ≤ g(z)].

A problem in Homework #5 is: Given a nonrecursive r.e. set A, find a simple set B such that A ≡_{T} B.

Here is a hint. By our characterization of r.e. sets, let f be a 1-1 recursive function whose range is A. Then we may let B = D_{f}. The proof that A ≡_{T} D_{f} is somewhat similar to the above proof that D_{f} is simple.

Coinfinite conditions

A coinfinite condition is a 1-place partial function p such that dom(p) is recursive and coinfinite. The technique of coinfinite approximation is a variant of the technique of finite approximation by strings. For a coinfinite condition p define φ_{e,s}^{(1),p}(x) ≃ y just as we did for strings, namely this means that the program with Gödel number e and input x using oracle p halts in ≤ s steps with output y consulting the oracle only for values in dom(p). Some useful notations for coinfinite conditions are:

1. p ⊆ q means: dom(p) ⊆ dom(q) and p(x) = q(x) for all x ∈ dom(p).

2. p ⊆_{fin} q means: p ⊆ q and dom(q) \ dom(p) is finite.

As an example of how to use coinfinite approximation, consider the proof of the following result concerning Turing degrees:

For all c there exist a, b > c such that inf(a, b) = c.

This is the relativization to c of the unrelativized result that there exist Turing degrees a, b > 0 such that inf(a, b) = 0. The unrelativized result was proved with finite approximations. To prove the relativization, use coinfinite approximations which are ≤_{T} h for some fixed function h such that deg_{T} (h) = c. We will have a = deg_{T} (f), b = deg_{T} (g), f = union of p_{0} ⊆_{fin} p_{1} ⊆_{fin} p_{2} ⊆_{fin} ⋯, g = union of q_{0} ⊆_{fin} q_{1} ⊆_{fin} q_{2} ⊆_{fin} · · ·. To guarantee a, b ≥ c, start with p_{0} and q_{0} defined as follows: p_{0}(2n) = h(n) and p_{0}(2n + 1) ↑ for all n, and q_{0} = p_{0}. Note that p_{0} and q_{0} are coinfinite conditions which include an infinte amount of information, namely h. We use finite extensions of p_{0} and q_{0} to accomplish a ≰ b and b ≰ a and inf(a, b) = c. This is similar to how we earlier used finite approximations to get incomparable a, b with inf(a, b) = 0.

Coinfinite conditions are useful for Homework #4, Problems 4 and 5. For Problem 4 use coinfinite conditions which are ≤_{T} C where deg_{T} (C) = c. For Problem 5 use coinfinite conditions which are ≤_{T} D_{n} for some n, where deg_{T} (D_{n}) = d_{n}.

Exercise 5.9.2. Explicitly exhibit a set which is Π_{5}^{0} and not Σ_{5}^{0}.

Exercise 5.9.3. We have seen that, given a 1-place partial recursive function ψ which is one-to-one, the inverse function ψ^{−1} is again partial recursive. The Uniformity Principle tells us that, given an index of ψ, we should expect to be able to compute an index of ψ^{−1}.

1. Give a rigorous statement of this result concerning indices.

2. Give a full proof of this result, using the Parametrization Theorem.

Exercise 5.9.4. Let A and B be subsets of ℕ. If A and B are simple, prove that A ∩ B is simple.

Solution. We are to prove that if A and B are simple then A ∩ B is simple. The proof is based on the following lemma.

Lemma 5.9.5. If A is simple, then every infinite r.e. set W_{e} has an infinite intersection with A.

Proof. Supose W_{e} infinite and W_{e} ∩ A = F finite. Then W_{e} \ F is also infinite and r.e. However, (W_{e} \ F) ∩ A ≠ ∅ since A is simple. This contradiction proves the lemma.

To see that A ∩ B is simple, let W_{e} be an infinite r.e. set. Since A is simple, we see by the previous lemma that W_{e} ∩ A is infinite. This is again an infinite r.e. set. Since B is simple, it follows that W_{e} ∩ A ∩ B ≠ ∅, Q.E.D.

Exercise 5.9.6. Let A, B, C be recursively enumerable subsets of ℕ such that A = B ∪ C and B ∩ C = ∅. Let a, b, c be the respective Turing degrees of A, B, C. Prove that a = sup(b, c).

Exercise 5.9.7. Consider the sets R = {e | W_{e} is recursive}, C = {e | W_{e} is creative}, and S = {e | W_{e} is simple}. What can you say or guess in the way of classifying R, C and S in the arithmetical hierarchy? Prove as much as you can.

Solution. Define

1. R = {e | W_{e} recursive}

2. C = {e | W_{e} creative}

3. S = {e | W_{e} simple}

The problem is to classify R, C, S in the arithmetical hierarchy.

To classify R we have

e ∈ R ≡ ∃i [W_{i} is the complement of W_{e}] ≡ ∃i [W_{i} ∩ W_{e} = ∅ ∧ W_{i} ∪ W_{e} = ℕ] ≡ \underbrace{∃i \underbrace{∀x \underbrace{[\underbrace{x ∉ W_{i} ∩ W_{e}}_{Π_{1}^{0}} ∧ \underbrace{x ∈ W_{i} ∪ W_{e}}_{Σ_{1}^{0}}]}_{Δ_{2}^{0}}}_{Π_{2}^{0}}}_{Σ_{3}^{0}} .

Thus R is Σ_{3}^{0} by this Tarski/Kuratowski computation.

Similarly for C we have

e ∈ C ≡ ∃ total recursive h(x)∀x [W_{x} ∩ W_{e} = ∅ ⇒ h(x) ∉ W_{x} ∪ W_{e}]
≡ ∃i ([∀x φ_{i}^{(1)}(x) ↓] ∧ [∀x W_{x} ∩ W_{e} ≠ ∅ ∨ ∀x ∀s φ_{i,s}^{(1)}(x) ∉ W_{i} ∪ W_{e}])
≡ Σ_{3}^{0}

A similar computation shows S is Π_{3}^{0}.

There is a useful heuristic principle. Namely, if an index set is classified in the arithmetical hierarchy by means of a Tarski/Kuratowski computation as above, then the set “ought to be” many-one complete within that class.

To complete the solution of Problem 5, one must prove that, indeed, R and C are Σ_{3}^{0}-complete and S is Π_{3}^{0}-complete. This can be done, but the proofs are rather difficult.

Exercise 5.9.8.

1. Let f_{i}, i = 0, 1, 2, … be a countable sequence of nonrecursive total 1place functions. Use the method of finite approximation to construct a nonrecursive total 1-place function g such that f_{i} ≰_{T} g for all i.

2. Deduce that for any Turing degree a > 0 we can find a Turing degree b > 0 such that inf(a, b) = 0.

Solution. Let f_{i}, i = 0, 1, 2, … be a sequence of nonrecursive functions. Then we can find g nonrecursive such that f_{i} ≰_{T} g for i = 0, 1, 2, . . .. This is done by finite approximation: g = ⋃_{n=0}^{∞} τ_{n} where τ_{0} ⊆ τ_{1} ⊆ · · ·.

Stage 0. Let τ_{0} = ⟨⟩.

Stage 2e + 1. Let x = |τ_{2e}|. If φ_{e}^{(1)}(x) ↓ let τ_{2e+1} = τ_{2e}^{⌢}⟨φ_{e}^{(1)}(x) + 1⟩. Otherwise do nothing, i.e., let τ_{2e+1} = τ_{2e}.

Stage 2e + 2 where e = 3^{i}5^{j}.

Case 1: (∃τ ⊇ τ_{2e+1}) ∃x (φ_{j}^{(1),τ} (x) ↓≠ f_{i}(x)). In this case choose such a τ and let τ_{2e+2} = τ.

Case 2: Not case 1. In this case do nothing, i.e., let τ_{2e+2} = τ_{2e+1}.

Stage 2e+1 insures that g is not recursive with index e. Stage 2e+2 insures that f_{i} ≠ φ_{j}^{(1),g} because otherwise f_{i} would be recursive. Namely, for all x we would have f_{i}(x) = φ_{j}^{(1),τ} (x) for the least τ ⊇ τ_{2e+1} such that φ_{j}^{(1),τ} (x) ↓.

Exercise 5.9.9. For each natural number n define

C_{φ}(n) = μe (φ_{e}^{(1)}(0) ≃ n) .

Intuitively, C_{φ}(n) is the smallest “description” of n in terms of our standard enumeration of the 1-place partial recursive functions, φ_{e}^{(1)}, e = 0, 1, 2, . . .. Note that C_{φ} is a total 1-place function, but it is not recursive.

Consider the set

S = {n | C_{φ}(n) < log log log n} .

Intuitively, S is the set of all n such that n has a relatively small description. For example, the number

n = (10 to the 10 to the 10 to the 10 to the 1, 000, 000, 000 power)

belongs to S because, although it is very large, it is also very easy to describe.

Prove that S is a simple set. This means:

1. S is recursively enumerable.

2. The complement of S is infinite.

3. The complement of S includes no infinite recursively enumerable set.

Solution. We define C_{φ}(n) = μe (φ_{e}^{(1)}(0) ≃ n) and S = {n | C_{φ}(n) < log log log n}. The problem is to show that S is a simple set.

1. S is r.e.

We have n ∈ S ≡ \underbrace{∃e (\underbrace{e < log^{3} n}_{recursive} ∧ \underbrace{φ_{e}^{(1)}(0) ≃ n}_{Σ_{1}^{0}})}_{Σ_{1}^{0}}.

2. S^{c} is infinite.

We know there are infinitely many e such that φ_{e}^{(1)}(0) ↑. Given k, choose x to be > at least k many such e’s. It follows that x is > at least k many n’s such that C_{φ}(n) ≥ x. For these n’s we have n < x ≤ C_{φ}(n), hence n ∉ S. Thus S^{c} includes at least k elements.

3. S^{c} ⊇ no infinite r.e. set. In other words, “any infinite r.e. set contains elements with short descriptions.”

By the Parametrization Theorem, let f(e, x) be a recursive function such that

φ_{f(e,x)}^{(1)}(0) ≃ φ_{e}^{(1)}(x)

for all e, x. Define a recursive binary relation ≪ by

m ≪ n ≡ (∀e ≤ m) (∀x ≤ m) [ f(e, x) < log^{3}(n) ] .

Note that ∀m ∃n (m ≪ n). Now let B be any infinite r.e. set. Let g be a total recursive function such that B = rng(g). Define a recursive function h by h(x) = g(μy (x ≪ g(y))) for all x. Then for all x we have x ≪ h(x) ∈ B. Let e be an index of h. Then for all x we have h(x) ≃ φ_{e}^{(1)}(x) ≃ φ_{f(e,x)}^{(1)}(0). In particular, for all x ≥ e we have C_{φ}(h(x)) ≤ f(e, x) < log^{3} h(x), hence h(x) ∈ B ∩ S, Q.E.D.

Exercise 5.9.10. Exhibit an oracle program P such that

φ_{e}^{(1),f}(x) ≃ μy (y > x ∧ f(y) = 0)

for all f ∈ ℕ^{ℕ} and all x ∈ ℕ, where e = #(P).

Exercise 5.9.11. 1. Give an explicit example of a Δ_{4}^{0} set which is neither Σ_{3}^{0} nor Π_{3}^{0}.

2. Give an example of a set which cannot be classified in the arithmetical hierarchy.

Exercise 5.9.12. Let A, B, C be recursively enumerable sets with A = B ∪ C and B ∩ C = ∅. If a, b, c are the respective Turing degrees of A, B, C prove that a = sup(b, c).

Note: The hardest part is to prove that B ≤_{T} A and C ≤_{T} A. Your proof should use the assumption that A, B, C are r.e. sets. Without this assumption, the result would not be correct.

Exercise 5.9.13. Construct an infinite descending sequence of Turing degrees

a_{0} > a_{1} > ⋯ > a_{n} > a_{n+1} > ⋯

or prove that no such sequence exists.

Lecture 18: October 8, 2007

5.10 Solutions of some homework and midterm problems

Lecture 19: October 10, 2007

Advance comments on Homework #7

We comment on each of the problems individually.

1. This is a routine exercise concerning oracle programs. We use the following notation:

Notation 5.10.1. ℕ^{ℕ} is the set of all oracles, i.e., total 1-place numbertheoretic functions.

ℕ^{ℕ} = {f : ℕ → ℕ} .

The space ℕ^{ℕ} is called the Baire space.

2. This is a routine exercise concerning the arithmetical hierarchy.

3. This is a repeat of one of the midterm problems.

4. This is another exercise on Turing degrees.

5. This exercise concerns Kolmogorov complexity. The idea of Kolmogorov complexity is, if τ is a finite sequence of 0’s and 1’s, then C(τ) = the “complexity” of τ, i.e., the “length of the shortest description” of τ. For example,

τ = ⟨\overbrace{0, 1, 0, 1, 0, 1, …, 0, 1}^{10^{10^{10^{10}}}}⟩

is a very long string of 0’s and 1’s but it has a very short description. The precise definition of C(τ) is below.

Kolmogorov complexity

6.1 Kolmogorov complexity

The purpose of this section is to introduce Kolmogorov complexity. First, some preliminaries.

Notation 6.1.1. Recall that a string is a finite sequence of natural numbers, σ = ⟨n_{1}, n_{2}, …, n_{l}⟩. Here l = length of σ = |σ|. We write

ℕ^{<ℕ} = the set of all strings
= ⋃_{l=0}^{∞} ℕ^{l} .

A bitstring (= 0,1-valued string) is a string of 0’s and 1’s. For example, σ = ⟨0, 1, 1, 0, 0, 0, 1, 0⟩ is a bitstring of length 8. We write

2^{<ℕ} = {0, 1}^{<ℕ}  = the set of all bitstrings
= ⋃_{l=0}^{∞}{0, 1}^{l} .

We often identify a string σ with its Gödel number #(σ).

Definition 6.1.2. A machine is a partial recursive function from bitstrings to bitstrings:

M : ⊆ 2^{<ℕ} → 2^{<ℕ} .

From our standard recursive enumeration φ_{e}^{(1)}, e = 0, 1, 2, … of all partial recursive functions, we obtain a recursive enumeration M_{e}, e = 0, 1, 2, … of all machines. Namely, we write

M_{e}(σ) ≃ τ ≡ φ_{e}^{(1)}(#(σ)) ≃ #(τ)

where σ, τ ∈ 2^{<ℕ}.

Definition 6.1.3. A universal machine is a machine, call it U, with the following property:

(∀ machines M) (∃ bitstring ρ) (∀ bitstrings σ) [ U(ρ^{⌢}σ) ≃ M(σ) ] .

We may think of the fixed bitstring ρ as a “code” for the machine M in terms of the universal machine U. The length of ρ is then called a “coding constant.”

Theorem 6.1.4. Universal machines exist.

Proof. Define

U(⟨\underbrace{0, …, 0}_{e}, 1⟩^{⌢}σ) ≃ M_{e}(σ) .

It is easy to check that U is a universal machine.

Now we define Kolmogorov complexity.

Definition 6.1.5 (Kolmogorov). Fix a universal machine U. The complexity of a bitstring τ is defined as

C(τ) = min{|σ| | U(σ) ≃ τ} .

Thus C(τ) is the length of the shortest “description” of τ in terms of the universal machine U. The idea of this definition is that C(τ) is supposed to measure the “amount of information” which is inherent in the bitstring τ. We shall now show that this measure of complexity is, in a sense, independent of the choice of universal machine.

Notation 6.1.6. f(x) ≤ g(x) + O(1) means:

∃c ∀x (f(x) ≤ g(x) + c) .

Theorem 6.1.7. “Up to an additive constant, C(τ) is well-defined.”

The precise statement reads as follows. If Û is another universal machine, and if we define Ĉ(τ) = min{|σ| | Û(σ) ≃ τ}, then

|C(τ) − Ĉ(τ)| ≤ O(1) .

Proof. Since Û is a machine, let ρ be a bitstring such that U(ρ^{⌢}σ) ≃ Û(σ) for all σ. Given τ, let σ, σ̂ be such that U(σ) ≃ τ, Û(σ̂) ≃ τ, |σ| = C(τ), |σ̂| = Ĉ(τ). We have Û(σ̂) ≃ U(ρ^{⌢}σ̂) ≃ τ, hence C(τ) ≤ |ρ^{⌢}σ̂| = |ρ| + |σ̂| = |ρ| + Ĉ(τ). Thus C(τ) ≤ Ĉ(τ) + O(1) because ρ is independent of τ. Similarly we can show that Ĉ(τ) ≤ C(τ) + O(1). This completes the proof.

Notation 6.1.8. We define the Kolmogorov complexity of an integer as

C(n) = C(⟨\underbrace{1, …, 1}_{n}⟩) .

Some easy facts are:

1. C(|τ|) ≤ C(τ) + O(1) .

2. C(τ) ≤ |τ| + O(1).

3. C(τ_{1}^{⌢}τ_{2}) ≤ 2C(τ_{1}) + 2C(τ_{2}) + O(1).

Question: Can we improve this to

C(τ_{1}^{⌢}τ_{2}) ≤ C(τ_{1}) + C(τ_{2}) + O(1) ?

This would be more intuitive.

Lecture 20: October 12, 2007

We have defined the complexity of a bitstring τ as

C(τ) = min{|σ| | U(σ) ≃ τ} = “the amount of information in τ”

where U is a fixed universal machine. We now prove some simple facts about this notion of complexity.

Proposition 6.1.9. The following hold for all strings τ. Recall that f(x) ≤ g(x) + O(1) means ∃c ∀x (f(x) ≤ g(x) + c).

1. C(|τ|) ≤ C(τ) + O(1) (where C(n) = C(⟨\underbrace{0, 0, …, 0}_{n}⟩) ).

Proof. Define M(σ) ≃ ⟨\underbrace{0, 0, …, 0}_{|U(σ)|}⟩. Clearly M is a machine. Let ρ be a

“code” for M in terms of U. This means that M(σ) ≃ U(ρ^{⌢}σ) for all σ. Now, given τ, let σ be such that U(σ) ≃ τ and |σ| = C(τ). Then, for this

σ, U(ρ^{⌢}σ) ≃ M(σ) ≃ ⟨\underbrace{0, 0, …, 0}_{|U(σ)|}⟩ = ⟨\underbrace{0, 0, …, 0}_{|τ|}⟩.

Hence C(|τ|) ≤ |ρ^{⌢}σ| = |ρ| + |σ| = C(τ) + |ρ| = C(τ) + O(1) since ρ is independent of τ.

2. C(τ) ≤ |τ| + O(1).

Proof. Consider the “identity machine,” I(σ) = σ. Let ρ be a “code” for I, i.e., U(ρ^{⌢}τ) ≃ I(τ) = τ for all τ. Then C(τ) ≤ |ρ^{⌢}τ| = |ρ| + |τ| = |τ| + O(1).

3. C(τ_{1}^{⌢}τ_{2}) ≤ 2C(τ_{1}) + 2C(τ_{2}) + O(1).

Proof. Temporarily define a “pairing function” for bitstrings, denoted σ∗τ. If σ = ⟨i_{1}, …, i_{m}⟩ and τ = ⟨j_{1}, …, j_{n}⟩, let

σ ∗ τ = ⟨0, i_{1}, 0, i_{2}, …, 0, i_{m}⟩^{⌢}⟨1, j_{1}, 1, j_{2}, …, 1, j_{n}⟩ .

Note that from σ ∗ τ we can recover σ and τ. Note also that |σ ∗ τ| = 2|σ| + 2|τ|. Now let M be a machine such that

M(σ_{1} ∗ σ_{2}) ≃ U(σ_{1})^{⌢}U(σ_{2})

for all bitstrings σ_{1}, σ_{2}. Let ρ be a “code” for M, i.e. U(ρ^{⌢}σ) ≃ M(σ) for all σ. Given τ_{1}, τ_{2} let σ_{1}, σ_{2} be “shortest descriptions” of τ_{1}, τ_{2}; U(σ_{1}) ≃ τ_{1}, |σ_{1}| = C(τ_{1}) and U(σ_{2}) ≃ τ_{2}, |σ_{2}| = C(τ_{2}). Then

U(ρ^{⌢}(σ_{1} ∗ σ_{2})) ≃ M(σ_{1} ∗ σ_{2})
≃ U(σ_{1})^{⌢}U(σ_{2})
≃ τ_{1}^{⌢}τ_{2}

so

C(τ_{1}^{⌢}τ_{2}) ≤ |ρ^{⌢}(σ_{1} ∗ σ_{2})|
= |ρ| + |σ_{1} ∗ σ_{2}|
= |ρ| + 2|σ_{1}| + 2|σ_{2}|
= |ρ| + 2C(τ_{1}) + 2C(τ_{2})
= 2C(τ_{1}) + 2C(τ_{2}) + O(1).

Exercise 6.1.10. Let C(σ) denote the Kolmogorov complexity of a 0, 1-valued string σ.

We have seen in class that

C(σ^{⌢}τ) ≤ 2C(σ) + 2C(τ) + O(1)

for all 0, 1-valued strings σ and τ. Improve this inequality to

C(σ^{⌢}τ) ≤ C(σ) + 2 log_{2} C(σ) + C(τ) + O(1)

where log_{2} x denotes the base 2 logarithm of x.

Can you make further improvements?

6.2 Prefix-free complexity

In this section we consider a variant of Kolmogorov complexity which is somewhat better behaved.

Recall that a machine is a partial recursive function from bitstrings to bitstrings,

M : ⊆ 2^{<ℕ} → 2^{<ℕ} .

It follows that dom(M) is a recursively enumerable set of bitstrings.

Definition 6.2.1. A prefix-free machine is a machine M such that dom(M) is prefix-free.

Intuitively, dom(M) consists of “self-delimiting programs,” i.e., each “valid program” σ ∈ dom(M) has an “end-of-program marker” and therefore cannot be an initial segment of another program in dom(M).

Definition 6.2.2. A universal prefix-free machine is a prefix-free machine, call it U, such that

(∀ prefix-free machines M) (∃ bitstring ρ) (∀ bitstrings σ) (U(ρ^{⌢}σ) ≃ M(σ)).

Theorem 6.2.3. Universal prefix-free machines exist.

Proof. Later.

Definition 6.2.4. Let U be a fixed universal prefix-free machine. The prefixfree complexity of a bitstring τ is defined as

K(τ) = min{|σ| | U(σ) ≃ τ} .

Remark 6.2.5. Prefix-free complexity, K(τ), has the same nice properties as “plain” complexity, C(τ). In fact, we shall see that it tends to have even better properties.

Theorem 6.2.6. K(τ) is well-defined up to an additive constant.

The precise statement reads as follows. If U_{1} and U_{2} are two universal prefixfree machines, and if for i = 1, 2 we define K_{i}(τ) = min{|σ| | U_{i}(σ) ≃ τ}, then |K_{1}(τ) − K_{2}(τ)| ≤ O(1) for all τ.

Proof. As before.

Also as before define K(n) = K(⟨\underbrace{1, …, 1}_{n}⟩). We then have:

Theorem 6.2.7.

1. K(|τ|) ≤ K(τ) + O(1).

2. K(τ) ≤ C(τ) + K(|τ|) + O(1). In particular K(τ) ≤ |τ| + 2 log_{2} |τ| + O(1), etc.

3. K(τ_{1}^{⌢}τ_{2}) ≤ K(τ_{1}) + K(τ_{2}) + O(1).

Note that in the case of “plain” complexity there was an annoying factor of 2 in part 3. For prefix-free complexity this factor of 2 disappears and is not needed.

Proof. Parts 1 and 2 are proved as before. Part 3 is Homework #8, Problem 1.

Remark 6.2.8. It would be interesting to compare C(τ) and K(τ). For instance, it is easy to see that C(τ) ≤ K(τ) + O(1) and K(τ) ≤ 2C(τ) + O(1). To what extent can we improve these inequalities? This would be an interesting research project.

We now prove Theorem 6.2.3 by constructing a universal prefix-free machine. As before let M_{e}, e = 0, 1, 2, … be our standard recursive enumeration of all machines, given by

M_{e}(σ) ≃ τ ≡ φ_{e}^{(1)}(#(σ)) ≃ #(τ)

where σ and τ are bitstrings. Define

M_{e,s}(σ) ≃ τ ≡ #(σ) < s and φ_{e,s}^{(1)}(#(σ)) ≃ #(τ) .

Some easy facts are:

1. M_{e}(σ) ≃ τ if and only if ∃s (M_{e,s}(σ) ≃ τ).

2. If s ≤ t and M_{e,s}(σ) ≃ τ then M_{e,t}(σ) ≃ τ.

3. The 4-place predicate M_{e,s}(σ) ≃ τ is recursive.

4. The 3-place predicate M_{e,s}(σ) ↓ is recursive.

5. For all σ ∈ dom(M_{e,s}) we have #(σ) < s. Define M̃_{e} to be the obvious “prefix-free restriction” of M_{e}, namely

M̃_{e}(σ) ≃ τ ≡ ∃s [ \underbrace{M_{e,s}(σ) ≃ τ}_{recursive} and \underbrace{dom(M_{e,s}) is prefix-free}_{recursive} ] ,

and note that M̃_{e} is partial recursive. Some easy facts are:

1. For each e, M̃_{e} is a prefix-free machine.

2. For each e, if M_{e} is a prefix-free machine then M̃_{e} = M_{e}.

3. M̃_{e}(σ) is a partial recursive function of e and σ. Thus we see that M̃_{e}, e = 0, 1, 2, … is a recursive enumeration of all prefix-free machines. It follows that the machine Ũ defined by

Ũ(⟨\underbrace{0, …, 0}_{e}, 1⟩^{⌢}σ) ≃ M̃_{e}(σ)

is a universal prefix-free machine.

6.3 The Kraft/Chaitin Theorem

Lecture 27: October 29, 2007

The following lemma is a useful technical tool in studying prefix-free complexity.

Remark 6.3.1. Given a prefix-free machine M, we know that dom(M) is a prefix-free set of bitstrings. Therefore,

∑_{σ∈dom(M)} \frac{1}{2^{|σ|}} = μ ( ⋃_{σ∈dom(M)} N_{σ}) ≤ 1 .

This is known as Kraft’s Inequality. The following lemma is a converse to Kraft’s Inequality.

Lemma 6.3.2 (Kraft/Chaitin Lemma). Let L be a Σ_{1}^{0} subset of ℕ × 2^{<ℕ} such that

∑_{(m,τ)∈L} \frac{1}{2^{m}} ≤ 1 .

Then, we can find a prefix-free machine M such that for each pair (m, τ) ∈ L there exists a bitstring σ such that |σ| = m and M(σ) ≃ τ.

Remark 6.3.3. Think of L as an abstract specification of a prefix-free machine M. Each pair (m, τ) ∈ L specifies that there should exist a bitstring σ of length m such that M(σ) ≃ τ. The pairs (m, τ) ∈ L are known as the “axioms” of the specification. The conclusion of the lemma says that we can construct a “designer” prefix-free machine M which will satisfy the given specification.

Proof. Since L is recursively enumerable, let (m_{i}, τ_{i}), i = 0, 1, 2, … be a recursive enumeration of L. Our assumption on L tells us that

∑_{i=0}^{∞} \frac{1}{2^{m_{i}}} ≤ 1 .

Sublemma. Given a recursive sequence of integers m_{i} ≥ 0, i = 0, 1, 2, … such that

∑_{i=0}^{∞} \frac{1}{2^{m_{i}}} ≤ 1 ,

we can effectively find a recursive, prefix-free sequence of bitstrings σ_{i}, i = 0, 1, 2, … such that |σ_{i}| = m_{i} for all i.

Once we have this, we can simply define M(σ_{i}) ≃ τ_{i} for all i. It is then obvious that M is a prefix-free machine as desired.

Proof of sublemma. We define σ_{i}, i = 0, 1, 2, … by recursion on i. In defining σ_{k}, we may use course-of-values recursion and assume that we already know σ_{i}, 0 ≤ i < k. We may also assume inductively that we have another finite, prefix-free set of bitstrings D_{k} with the following properties:

1. D_{k} ∩ {σ_{i} | 0 ≤ i < k} = ∅.

2. D_{k} ∪ {σ_{i} | 0 ≤ i < k} is a partition.

3. All of the bitstrings in D_{k} have different lengths.

Definition 6.3.4. A partition is a finite, prefix-free set of bitstrings, call it F, such that

2^{ℕ} = ⋃_{σ∈F} N_{σ} .

To be finished next time ….

Lecture 28: October 31, 2007

We resume our analysis of prefix-free complexity.

Remark 6.3.5. Given a prefix-free sequence of bitstrings σ_{i}, i = 0, 1, 2, …, we have

∑_{i=0}^{∞} \frac{1}{2^{|σ_{i}|}} ≤ 1

because the sum on the left is just μ(⋃_{i=0}^{∞} N_{σ_{i}}) which is ≤ 1 because ⋃_{n=0}^{∞} N_{σ_{i}} ⊆ 2^{ℕ} and μ(2^{ℕ}) = 1. This is the Kraft Inequality. The following lemma is a converse of this remark.

Lemma 6.3.6. Given a sequence of positive integers m_{i}, i = 0, 1, 2, … such that

∑_{i=0}^{∞} \frac{1}{2^{m_{i}}} ≤ 1 ,

we can find a prefix-free sequence of bitstrings σ_{i}, i = 0, 1, 2, … such that |σ_{i}| = m_{i} for all i. Moreover, if the sequence m_{i}, i = 0, 1, 2, … is recursive, we can take the sequence σ_{i}, i = 0, 1, 2, … to be recursive.

Proof. The proof is based on the following definition.

Definition 6.3.7. A partition is a finite, prefix-free set of bitstrings, F, such that

2^{ℕ} = ⋃_{σ∈F} N_{σ} .

We will construct σ_{i}, i = 0, 1, 2, … by induction. At each stage k, in defining σ_{k} we may assume that the prefix-free finite sequence of bitstrings σ_{i}, i < k, is already known. In addition we assume that we have a finite prefix-free set of bitstrings D_{k} with the following properties:

1. D_{k} ∩ {σ_{i} | i < k} = ∅.

2. D_{k} ∪ {σ_{i} | i < k} is a partition.

3. All of the strings in D_{k} are of different lengths.

Example 6.3.8. To illustrate the construction, consider m_{0} = 2, m_{1} = 4, m_{2} = 3, m_{3} = 2, … such that

∑_{i=0}^{∞} \frac{1}{2^{m_{i}}} = \frac{1}{4} + \frac{1}{16} + \frac{1}{8} + \frac{1}{4} + ⋯ ≤ 1 .

In constructing the sequence of bitstrings σ_{0}, σ_{1}, σ_{2}, σ_{3}, …, the idea is to always choose the leftmost available branch on the binary tree at each step (i.e., the branch with the most 0’s). The sets D_{k} are constructed by taking the shortest possible branches from the right side of the binary tree until D_{k} satisfies the induction hypothesis.

Given m_{0} = 2, pick σ_{0} = ⟨0, 0⟩ and D_{1} = {⟨1⟩, ⟨0, 1⟩}.

Then, given m_{1} = 4, pick σ_{1} = ⟨0, 1, 0, 0⟩ and D_{2} = {⟨1⟩, ⟨0, 1, 1⟩, ⟨0, 1, 0, 1⟩}. Then, given m_{2} = 3, pick σ_{2} = ⟨0, 1, 1⟩ and D_{3} = {⟨1⟩, ⟨0, 1, 0, 1⟩}. Then, given m_{3} = 2, pick σ_{3} = ⟨1, 0⟩ and D_{4} = {⟨1, 1⟩, ⟨0, 1, 0, 1⟩}. Etc.

Formally, at stage k +1, suppose we already have σ_{i}, i < k and D_{k} as above. We claim that m_{k} ≥ min{|ρ| | ρ ∈ D_{k}}. Otherwise m_{k} < min{|ρ| | ρ ∈ D_{k}},

hence

\frac{1}{2^{m_{k}}} > ∑_{ρ∈D_{k}} \frac{1}{2^{|ρ|}}

in view of the requirement that all strings in D_{k} are of different lengths. Hence

∑_{i=0}^{k} \frac{1}{2^{m_{i}}} = \frac{1}{2^{m_{k}}} + ∑_{i=0}^{k−1} \frac{1}{2^{m_{i}}} > ∑_{ρ∈D_{k}} \frac{1}{2^{|ρ|}} + ∑_{i=0}^{k−1} \frac{1}{2^{|σ_{i}|}} = 1

a contradiction. This proves the claim.

By the claim, let ρ_{k} ∈ D_{k} be of maximal length such that |ρ_{k}| ≤ m_{k}. Letting

σ_{k} = ρ_{k}^{⌢}⟨\underbrace{0, …, 0}_{m_{k}−|ρ_{k}|}⟩

we see that |σ_{k}| = m_{k} and σ_{k} | σ_{i} for all i < k. Letting

D_{k+1} = D_{k} \ \{ρ_{k}\} ∪ \{ρ_{k}^{⌢}⟨\underbrace{0, …, 0}_{j}, 1⟩ | j < m_{k} − |ρ_{k}|\}

it is straightforward to check that properties 1,2, and 3 hold with D_{k+1} in place of D_{k}. This completes the proof.

Theorem 6.3.9 (Kraft/Chaitin Theorem). Given a Σ_{1}^{0} set L ⊆ ℕ × 2^{<ℕ} such that

∑_{(m,τ)∈L} \frac{1}{2^{m}} ≤ 1 ,

we can find a prefix-free machine M such that for all (m, τ) ∈ L there exists σ such that |σ| = m and M(σ) ≃ τ.

Proof. Since L is recursively enumerable, let (m_{i}, τ_{i}), i = 0, 1, 2, … be a one-to-one recursive enumeration of L. Apply the previous lemma to find a recursive, prefix-free sequence of bitstrings σ_{i}, i = 0, 1, 2, … such that |σ_{i}| = m_{i} for all i. Let M(σ_{i}) ≃ τ_{i} for all i.

Remark 6.3.10. The idea of the Kraft/Chaitin Theorem is that L is an abstract specification of a prefix-free machine. The pairs (m, τ) ∈ L are called the “axioms” of the specification. Each axiom (m, τ) says that our prefix-free machine M is intended to have M(σ) ≃ τ for some σ of length m. The theorem asserts that we can find a prefix-free machine which meets all of these requirements.

The Kraft/Chaitin Theorem has the following consequence for prefix-free complexity, K.

Corollary 6.3.11. Let L be a Σ_{1}^{0} subset of ℕ × 2^{<ℕ} such that

∑_{(m,τ)} \frac{1}{2^{m}} < ∞ .

Then for all (m, τ) ∈ L we have K(τ) ≤ m + O(1).

Proof. Let c be such that

∑_{(m,τ)∈L} \frac{1}{2^{m}} ≤ 2^{c} < ∞ .

Then

∑_{(m,τ)∈L} \frac{1}{2^{m+c}} ≤ 1

so by the Kraft/Chaitin Theorem, let M be a prefix-free machine such that for all (m, τ) ∈ L there exists σ such that |σ| = m + c and M(σ) ≃ τ. Then for all (m, τ) ∈ L we have K(τ) ≤ m + c + O(1) = m + O(1).

The Cantor space

7.1 Partial recursive functionals, etc.

Remark 7.1.1. The following three spaces are important for us.

1. ℕ^{ℕ} = the Baire space.

2. 2^{ℕ} = {0, 1}^{ℕ} = the Cantor space.

3. ℕ = the natural numbers.

Recall that ℕ^{ℕ} = {f : ℕ → ℕ}, the space of total 1-place number-theoretic functions. We use letters such as f, g, h, … for points of ℕ^{ℕ}. We also consider 2^{ℕ}, the space of all 0, 1-valued total 1-place number-theoretic functions, 2^{ℕ} = {X : ℕ → {0, 1}}. Note that 2^{ℕ} ⊂ ℕ^{ℕ}. We use letters such as X, Y, Z, … to denote points in 2^{ℕ}.

We deal explicitly with the Baire space, but everything that we are saying applies also to the Cantor space as well as various “mixed” spaces.

Definition 7.1.2. A partial functional is a function

Φ : ⊆ ℕ^{ℕ} × ℕ^{k} → ℕ .

Note that dom(Φ) = the domain of Φ, a subset of ℕ^{ℕ} × ℕ^{k}, and rng(Φ) = the range of Φ, a subset of ℕ. We use notations such as Φ(f, x_{1}, …, x_{k}) ≃ y and Φ(f, x_{1}, …, x_{k}) ↓ and Φ(f, x_{1}, …, x_{k}) ↑ for partial functionals, just as for partial functions. Here f ranges over ℕ^{ℕ} and x_{1}, …, x_{k} range over ℕ.

Definition 7.1.3. A partial functional Φ as above is said to be partial recursive if and only if

∃e (∀f ∈ ℕ^{ℕ}) (∀x_{1}, …, x_{k}, y ∈ ℕ) (Φ(f, x_{1}, …, x_{k}) ≃ y ≡ φ_{e}^{(1),f}(x_{1}, …, x_{k}) ≃ y) .

In other words, Φ(f, x_{1}, …, x_{k}) ≃ φ_{e}^{(1),f}(x_{1}, …, x_{k}) for all f, x_{1}, …, x_{k}.

Example 7.1.4. An example of a partial recursive functional is

Φ(f, x) ≃ μy (y > x ∧ f(y) = 0).

Note that for this Φ we have Φ(f, x) ↓≡ ∃y (y > x ∧ f(y) = 0). Homework #7 Problem 1 is to exhibit an oracle machine computing this partial functional.

Definition 7.1.5. A predicate R ⊆ ℕ^{ℕ} × ℕ^{k} is said to be recursive if and only if its characteristic function χ_{R} : ℕ^{ℕ} × ℕ^{k} → {0, 1} is recursive.

Definition 7.1.6. For n ≥ 1, a predicate P ⊆ ℕ^{ℕ} × ℕ^{k} is said to be Σ_{n}^{0} if

P (f, x_{1}, …, x_{k}) ≡ ∃y_{1} ∀y_{1} ⋯ \begin{matrix}∀ \\ ∃ \end{matrix} y_{n} R(f, x_{1}, …, x_{k}, y_{1}, …, y_{n})

where R is a recursive predicate. Similarly we extend the definitions of Π_{n}^{0} and Δ_{n}^{0} for n ≥ 1 to predicates P ⊆ ℕ^{ℕ} × ℕ^{k}.

Remark 7.1.7. All of the usual closure properties hold in the context of predicates P ⊆ ℕ^{ℕ} × ℕ^{k}. The class of Σ_{n}^{0} predicates is closed under ∧, ∨, bounded quantification, total recursive substitution, etc. A predicate is Σ_{n}^{0} if and only if its negation is Π_{n}^{0}, etc. Thus we can perform Tarski/Kuratowski computations in this context.

We now consider how to relativize the above to a fixed Turing oracle g.

Remark 7.1.8. A useful fact about the spaces ℕ^{ℕ} and 2^{ℕ} is the existence of a 1 − 1 onto pairing function

\begin{matrix}ℕ^{ℕ} × ℕ^{ℕ} & ⟷_{onto}^{1−1} & ℕ^{ℕ} \\ (f, g) & & f ⊕ g \end{matrix}

where

\begin{aligned}(f ⊕ g)(2n) & = & f(n) \\ (f ⊕ g)(2n + 1) & = & g(n) \end{aligned}

and

\begin{matrix}2^{ℕ} × 2^{ℕ} & ⟷_{onto}^{1−1} & 2^{ℕ} \\ (X, Y) & & X ⊕ Y \end{matrix}

as a special case.

Definition 7.1.9. Let g be a fixed oracle, i.e., g ∈ ℕ^{ℕ}. A partial functional Φ : ⊆ ℕ^{ℕ} × ℕ^{k} → ℕ is said to be partial g-recursive, or partial recursive relative to g, if

∃e (∀f ∈ ℕ^{ℕ}) (∀x_{1}, …, x_{k} ∈ ℕ) (Φ(g, x_{1}, …, x_{k}) ≃ φ_{e}^{(k),f⊕g}(x_{1}, …, x_{k})) .

Similarly we define what it means for a predicate P ⊆ ℕ^{ℕ} ×ℕ^{k} to be g-recursive, Σ_{n}^{0,g}, etc.

Remark 7.1.10. We can extend all of this to “mixed” functionals and predicates on spaces such as

(ℕ^{ℕ})^{m} × (2^{ℕ})^{l} × ℕ^{k}

using the pairing function ⊕. For example, a partial functional

Φ : ⊆ (ℕ^{ℕ})^{m} × (2^{ℕ})^{l} × ℕ^{k} → ℕ

is said to be partial g-recursive if and only if it is defined by

Φ(f_{1}, …, f_{m}, X_{1}, …, X_{l}, x_{1}, …, x_{k}) ≃ φ_{e}^{(k),f_{1}⊕···⊕f_{m}⊕X_{1}⊕···⊕X_{l}⊕g}(x_{1}, …, x_{k})

for some fixed e and for all f_{1}, …, f_{m} ∈ ℕ^{ℕ}, X_{1}, …, X_{l} ∈ 2^{ℕ}, x_{1}, …, x_{k} ∈ ℕ.

Remark 7.1.11. Post’s Theorem fails in this context. For predicates P ⊆ ℕ^{ℕ} × ℕ^{k} or P ⊆ 2^{ℕ} × ℕ^{k}, it is not true in general that P ∈ Σ_{2}^{0} ≡ P ∈ Σ_{1}^{0,0′}. It is true that P ∈ Σ_{1}^{0,0′} implies P ∈ Σ_{2}^{0}, but the converse does not hold in general. See also Homework #8, Problem 2.

7.2 Homework #8, due October 22, 2007

Exercises 7.2.1.

1. Let K(τ) denote the prefix-free complexity of a bitstring τ. Prove that

K(τ_{1}^{⌢}τ_{2}) ≤ K(τ_{1}) + K(τ_{2}) + O(1) .

2. (a) Give an example of a subset of ℕ^{ℕ} which is Σ_{2}^{0} but not Σ_{1}^{0,0′}.

(b) Can you replace ℕ^{ℕ} by 2^{ℕ} here?

Note: Recall Post’s Theorem, which says (among other things) that a subset of ℕ is Σ_{2}^{0} if and only if it is Σ_{1}^{0,0′}. The point of (a) is to show that Post’s Theorem does not hold for subsets of ℕ^{ℕ}.

Hint: Recall that a set is open if and only if it is Σ_{1}^{0} relative to an oracle. Therefore, it suffices to find a set which is Σ_{2}^{0} and not open.

3. A real number is said to be left recursively enumerable (respectively right recursively enumerable) if it is the limit of an increasing (respectively decreasing) recursive sequence of rational numbers.

(a) If A is a recursively enumerable subset of ℕ, show that the real number ∑_{n∈A} 1/2^{n} is left recursively enumerable.

(b) Show that there exist real numbers which are left recursively enumerable but not recursive.

(c) Show that a real number is recursive if and only if it is both left recursively enumerable and right recursively enumerable.

4. Let P be a Π_{1}^{0} subset of 2^{ℕ}. We have seen how to construct a recursive tree T ⊆ 2^{<ℕ} such that P = {paths through T }. For each n = 0, 1, 2, … let T_{n} be the set of strings in T of length n.

(a) Show that T_{n} is prefix-free.

(b) Show that the set

V_{n} = ⋃_{τ∈T_{n}} N_{τ}

is Δ_{1}^{0}. (Note that V_{n} is a subset of 2^{ℕ}.)

(c) Show that P is the intersection of the V_{n}’s. In other words,

P = ⋂_{n=0}^{∞} V_{n} .

(d) Show that the measure of P is given by

μ(P) = lim_{n→∞} \frac{|T_{n}|}{2^{n}} .

Here |T_{n}| denotes the number of strings in T_{n}.

(e) Show that the real number μ(P) is right recursively enumerable.

(f) Show that μ(P) is not necessarily a recursive real number.

5. Given a nonempty Π_{1}^{0} set P ⊆ 2^{ℕ}, can we always find a member of P which is recursive?

Hint: Consider a recursively inseparable pair of r.e. sets.

6. Two sets P, Q ⊆ ℕ^{ℕ} are said to be Turing isomorphic if the members of P and Q have the same Turing degrees, i.e.,

{deg_{T} (f) | f ∈ P } = {deg_{T} (g) | g ∈ Q} .

(a) Prove that every Π_{2}^{0} subset of ℕ^{ℕ} is Turing isomorphic to a Π_{1}^{0} subset of ℕ^{ℕ}.

(b) Prove that every Π_{2}^{0} subset of ℕ^{ℕ} is Turing isomorphic to a Π_{2}^{0} subset of 2^{ℕ}.

(c) Is every Π_{2}^{0} subset of 2^{ℕ} Turing isomorphic to a Π_{1}^{0} subset of 2^{ℕ}? Justify your answer.

Hints: (a) If ∀x ∃y R(f, x, y) holds, map f to f⊕g where g(x) = μy R(f, x, y).

(b) Map f to the characteristic function of the set G_{f} = {3^{x}5^{y} | f(x) = y} = the “graph” of f.

7.3 Σ_{1}^{0} and Π_{1}^{0} sets in 2^{ℕ} and ℕ^{ℕ}

We have defined the three spaces ℕ, 2^{ℕ}, and ℕ^{ℕ} (the natural numbers, the Cantor space, and the Baire space). For mixed predicates, P ⊆ (ℕ^{ℕ})^{m} ×(2^{ℕ})^{l} × ℕ^{k}, we know what it means for P (−, −, −) to be recursive, or Σ_{n}^{0}, or Σ_{n}^{0} relative to an oracle, etc.

Remark 7.3.1. All of our rules about combining predicates apply in this context. If P, Q ∈ Σ_{n}^{0} then P ∧ Q ∈ Σ_{n}^{0}, P ∨ Q ∈ Σ_{n}^{0}, ¬ P ∈ Π_{n}^{0}). P is recursive if and only if P is Δ_{1}^{0}. The class of Σ_{n}^{0} predicates is closed under bounded quantification, recursive substitution, etc.

Remark 7.3.2. Post’s Theorem fails in this context. For instance, we can find a Σ_{2}^{0} set P ⊆ ℕ^{ℕ} which is not Σ_{1}^{0,0′}. See Homework #8, Problem 2.

For the sake of our later discussion of randomness, we want to focus on Σ_{1}^{0} sets and Π_{1}^{0} sets in 2^{ℕ}, the Cantor space.

Recall that

2^{ℕ} = {X | ℕ → {0, 1}}
= {infinite sequences of 0’s and 1’s} .

Each point X ∈ 2^{ℕ} is an infinite sequence of 0’s and 1’s, i.e.,

X = ⟨X(0), X(1), …, X(n), …⟩

with each X(n) = either 0 or 1. We view X as the outcome of an infinite sequence of independent coin tosses using a “fair coin”, i.e., probability of heads = probability of tails. We identify 1 as heads, 0 as tails. This corresponds to the fair coin measure on the space 2^{ℕ}.

In order to define the fair coin measure rigorously, recall that

2^{<ℕ} = {bitstrings} = {0,1-valued strings}.

Definition 7.3.3. Given a bitstring σ ∈ 2^{<ℕ} and a point X ∈ 2^{ℕ}, write σ ⊂ X to mean that σ is an initial segment of X, i.e., σ = ⟨X(0), X(1), …, X(n−1)⟩ = X ↾ n for some n. We then have n = |σ|. Define

N_{σ} = {X ∈ 2^{ℕ} | σ ⊂ X} = the neighborhood determined by σ.

The fair coin probability measure is defined as the unique measure μ on 2^{ℕ} with μ(N_{σ}) = 1/2^{|σ|} for all bitstrings σ. To motivate this definition, note that if |σ| = n then the probability of the event X ↾ n = σ is 1/2^{n} = 1/2^{|σ|}.

Definition 7.3.4. A set U ⊆ 2^{ℕ} is said to be open if U is the union of a collection of neighborhoods. In other words,

U = ⋃_{σ∈S} N_{σ}

where S ⊆ 2^{<ℕ} is a set of bitstrings.

Remark 7.3.5. Σ_{1}^{0} sets in the Cantor space are open. This is because of finite approximation. Let us elaborate.

A typical Σ_{1}^{0} set in 2^{ℕ} is

U_{e} = {X ∈ 2^{ℕ} | φ_{e}^{(1),X}(0) ↓} .

We use U_{e}, e = 0, 1, 2, … as our standard recursive enumeration of the Σ_{1}^{0} sets in 2^{ℕ}. By finite approximation, X ∈ U_{e} ≡ ∃σ (σ ⊂ X ∧ φ_{e}^{(1),σ}(0) ↓). This shows that X ∈ U_{e} depends only on a finite amount of information from X, i.e., it depends only on an initial segment of X.

Recall the notation φ_{e,s}^{(1),σ}(x) ≃ y which means that the oracle computation with input x and oracle f halts in ≤ s steps with output y using only oracle information from σ ⊂ f. This predicate is recursive. Furthermore, the predicate

φ_{e}^{(1),σ}(x) ≃ φ_{e,|σ|}^{(1),σ}(x)

is also recursive. We then have

U_{e} = ⋃_{φ_{e}^{(1),σ}(0)↓} N_{σ} .

Hence U_{e} is an open set.

We now refine the above remark to get a useful technical fact. Note that for each X ∈ U_{e} there is a unique shortest initial segment σ ⊂ X such that φ_{e}^{(1),σ}(0) ↓. Thus we have

X ∈ U_{e} ≡ ∃σ (σ ⊂ X ∧ φ_{e}^{(1),σ}(0) ↓)
≡ ∃σ (σ ⊂ X ∧ φ_{e}^{(1),σ}(0) ↓ ∧ ¬ ∃τ ⊂ σ (φ_{e}^{(1),τ} (0) ↓))
≡ X ∈ ⋃_{σ∈S_{e}} N_{σ}

where S_{e} is a recursive set of bitstrings, namely

S_{e} = {σ | φ_{e}^{(1),σ}(0) ↓ ∧ ¬ ∃τ ⊂ σ (φ_{e}^{(1),τ} (0) ↓)} .

Note also that S_{e} is prefix-free, in the following sense.

Definition 7.3.6. If σ, τ ∈ 2^{<ℕ} are bitstrings, σ ⊆ τ means that σ is an initial segment of τ (possibly σ = τ). Also σ ⊂ τ means that σ is a proper initial segment or prefix of τ, i.e., σ ⊆ τ and σ ≠ τ. A set of bitstrings S ⊆ 2^{<ℕ} is said to be prefix-free if ¬ (∃σ ∈ S) (∃τ ∈ S) (σ ⊂ τ).

Summarizing, we have proved the following theorem which says among other things that Σ_{1}^{0} sets are open.

Theorem 7.3.7. For a set U ⊆ 2^{ℕ}, the following are pairwise equivalent.

1. U is Σ_{1}^{0} .

2. U = U_{e} for some e.

3. U = ⋃_{σ∈S} N_{σ} for some recursively enumerable set of bitstrings S.

4. U = ⋃_{σ∈S} N_{σ} for some recursive, prefix-free set of bitstrings S.

Proof. 1 ⇔ 2 by definition.

2 ⇒ 4 is what we have already proved.

4 ⇒ 3 is trivial.

3 ⇒ 1: Assuming 3, we have

X ∈ U ≡ X ∈ ⋃_{σ∈S} N_{σ}, S is r.e.

≡ ∃σ \underbrace{(\underbrace{σ ∈ S}_{Σ_{1}^{0}} ∧ \underbrace{σ ⊂ X}_{recursive})}_{Σ_{1}^{0}}

Σ_{1}^{0}

which proves 1.

Remark 7.3.8. If an open set has been written as

U = ⋃_{σ∈S} N_{σ}

where S is prefix-free, then we can find the measure of U as follows. Note first that σ ⊆ τ if and only if N_{σ} ⊇ N_{τ}. On the other hand, if σ ⊈ τ and τ ⊈ σ (i.e., σ is incompatible with τ, abbreviated σ | τ), then N_{σ} ∩ N_{τ} = ∅. Thus, for any prefix-free set of bitstrings S, U = ⋃_{σ∈S} N_{σ} is a union of pairwise disjoint neighborhoods, hence

μ(U) = ∑_{σ∈S} \frac{1}{2^{|σ|}} .

We also have:

Theorem 7.3.9. For a set U ⊆ 2^{ℕ}, the following are pairwise equivalent.

1. U is open.

2. U is Σ_{1}^{0,f} for some oracle f.

3. U = ⋃_{σ∈S} N_{σ} for some set of bitstrings S.

4. U = ⋃_{σ∈S} N_{σ} for some prefix-free set of bitstrings S.

In the latter case we have μ(U) = ∑_{σ∈S} 1/2^{|σ|}.

Proof. 1 ⇔ 3 holds by definition. If 3 holds then clearly U is Σ_{1}^{0,S}, hence 2 holds. The previous theorem relativizes to prove 2 ⇒ 4. The implication 4 ⇒ 3 is trivial. This completes the proof.

Review: We have seen that a typical Σ_{1}^{0} set in the Cantor space 2^{ℕ} looks like

U_{e} = {X ∈ 2^{ℕ} | φ_{e}^{(1),X}(0) ↓} .

We have proved that

U_{e} = ⋃_{σ∈S_{e}} N_{σ}

where S_{e} is a set of bitstrings which is recursive and prefix-free, i.e., ¬ (∃σ, τ ∈ S_{e}) (σ ⊂ τ). Here the neighborhoods are defined by N_{σ} = {X ∈ 2^{ℕ} | σ ⊂ X} for all bitstrings σ.

Remark 7.3.10. The same analysis holds for Σ_{1}^{0} sets in the Baire space, ℕ^{ℕ}, using strings instead of bitstrings. In this case we would have to define the neighborhoods differently, namely we would have N_{σ} = {f ∈ ℕ^{ℕ} | σ ⊂ f} for all strings σ.

We now consider the structure of Π_{1}^{0} sets.

Of course, P ⊆ ℕ^{ℕ} is Π_{1}^{0} if and only if ℕ^{ℕ} \ P is Σ_{1}^{0}. Hence, a picture of a Π_{1}^{0} set can be obtained by viewing it as the complement of a Σ_{1}^{0} set, which in turn is described by a recursive, prefix-free set of strings.

We wish to observe that another useful picture of Π_{1}^{0} sets can be obtained in terms of trees. See also Homework #8, Problem 4.

Definition 7.3.11. A tree is a set of strings, T, which is closed when taking initial segments. In other words, ∀σ ∀τ (σ ⊆ τ, τ ∈ T ⇒ σ ∈ T).

Example 7.3.12. T = 2^{<ℕ} = the full binary tree.

[figure: ⟨0, 0⟩; ⟨0, 1⟩; ⟨1, 0⟩; ⟨1, 1⟩; ⟨0⟩; ⟨1⟩; ⟨ ⟩]

Remark 7.3.13. Trees are in a sense the opposite of prefix-free sets of strings. T is a tree if and only if all prefixes of members of T are members of T. S is prefix-free if and only if no prefix of a member of S is a member of S.

Definition 7.3.14. Let T be a tree. A path through T is a function f ∈ ℕ^{ℕ} such that f ↾ n ∈ T for all n.

Example 7.3.15. The Π_{1}^{0} set 2^{ℕ} ⊆ ℕ^{ℕ} is the set of paths through 2^{<ℕ}, the full binary tree. Theorem 7.3.16. A set P ⊆ ℕ^{ℕ} is Π_{1}^{0} if and only if P = {paths through T } for some recursive tree T. Proof. (⇒) Assume P is Π_{1}^{0}. Then

P = {f ∈ ℕ^{ℕ} | φ_{e}^{(1),f}(0) ↑}
= {f ∈ ℕ^{ℕ} | ∀n φ_{e}^{(1),f↾n}(0) ↑}

(by finite approximation)

= {f ∈ ℕ^{ℕ} | f is a path through T }

where T = {τ ∈ ℕ^{<ℕ} | φ_{e}^{(1),τ} (0) ↑}. Note that T is a recursive tree.

(⇐) Assume P = {paths through T } for some recursive tree T. Then

P = \{f ∈ ℕ^{ℕ} | ∀n \underbrace{f ↾ n ∈ T}_{recursive}\}
= {f ∈ ℕ^{ℕ} | ∀n R(f, n)}
= Π_{1}^{0}

Remark 7.3.17. The same applies to Π_{1}^{0} sets in the Cantor space, 2^{ℕ}. Note also that if P ⊆ 2^{ℕ} is Π_{1}^{0} then we may restrict our attention to bitstrings, so we may take our recursive tree T to be a subtree of 2^{<ℕ}. Definition 7.3.18. A set P ⊆ ℕ^{ℕ} is defined to be closed if it complement ℕ^{ℕ} \ P is open.

In analogy with Theorems 7.3.9 and 7.3.16 we have Theorem 7.3.19. For P ⊆ ℕ^{ℕ} the following are pairwise equivalent.

1. P is closed.

2. P is Π_{1}^{0,f} for some oracle f.

3. P = {paths through T } for some tree T. Moreover, if P ⊆ 2^{ℕ} then we may take T to be a subtree of 2^{<ℕ}. Proof. This follows easily from Theorems 7.3.9 and 7.3.16 and relativization.

7.4 Compactness of 2^{ℕ}

In order to study the Turing degrees of random sequences, it is convenient to use the fact that the Cantor space 2^{ℕ} is compact. This is embodied in the following theorem.

Theorem 7.4.1 (compactness of 2^{ℕ}). If S is a set of bitstrings such that 2^{ℕ} = ⋃_{σ∈S} N_{σ}, then 2^{ℕ} = ⋃_{σ∈F} N_{σ} for some finite F ⊆ S. In other words,

“Every covering of 2^{ℕ} by neighborhoods contains a finite subcovering.”

Note that this does not hold for the Baire space, ℕ^{ℕ}. For example, ℕ^{ℕ} = ⋃_{i=0}^{∞} N_{⟨i⟩} but obviously there is no finite subcovering.

In order to prove Theorem 7.4.1, we shall first prove a combinatorial lemma concerning trees. Recall that a tree is a set T ⊆ ℕ^{<ℕ} which is closed under initial segments, i.e., σ ⊂ τ, τ ∈ T ⇒ σ ∈ T.

Definition 7.4.2. A tree T is said to be finitely branching if for every σ ∈ T there are only finitely many n such that σ^{⌢}⟨n⟩ ∈ T.

Define an immediate extension of a string σ to be any string of the form σ^{⌢}⟨n⟩ for some n. Note that a tree T is finitely branching if and only if each string in T has only finitely many immediate extensions in T.

Example 7.4.3. The full binary tree 2^{<ℕ} is finitely branching, because for any bitstring σ the only immediate extensions of σ which are bitstrings are σ^{⌢}⟨0⟩ and σ^{⌢}⟨1⟩.

Note also that any subtree of a finitely branching tree is finitely branching. In particular, any subtree of the full binary tree is finitely branching.

Lemma 7.4.4 (König’s Lemma). Let T be a finitely branching tree. Then T is infinite ⇔ T has a path.

Note that König’s Lemma fails badly for trees that are not finitely branching. For example, the tree {⟨⟩, ⟨0⟩, ⟨1⟩, ⟨2⟩, …} is infinite and not finitely branching and has no path, in fact it contains no string of length 2.

Proof of König’s Lemma. The ⇐ is obvious. If f is a path through T, then T contains the strings f ↾ n, n = 0, 1, 2, …, hence T is infinite.

To prove ⇒, let T be an infinite, finitely branching tree. Define

T̂ = {σ ∈ T | T contains infinitely many extensions of σ} .

Note that T̂ is a subtree of T.

We claim that the empty string ⟨⟩ belongs to T̂. This is obvious, because T is infinite.

We claim that any σ ∈ T̂ has an immediate extension belonging to T̂. This is because any extension of σ in T is an extension of some immediate extension of σ in T. Since σ has infinitely many extensions in T, and σ has only finitely many immediate extensions in T (because T is finitely branching), it follows by the Pigeonhole Principle that at least one of the immediate extensions of σ has infinitely many extensions in T, i.e., it belongs to T̂.

Now, to construct a path through T, it suffices to construct a path through T̂. Start with σ_{0} = ⟨⟩ ∈ T̂. Given σ_{i} ∈ T̂, let n_{i} = some n such that σ_{i}^{⌢}⟨n⟩ ∈ T̂, and let σ_{i+1} = σ_{i}^{⌢}⟨n_{i}⟩. Thus σ_{0}, σ_{1}, σ_{2}, … is an infinite path through T̂. Actually, the infinite path f is defined by f(i) = n_{i} for all i.

Proof of Theorem 7.4.1. Given a covering 2^{ℕ} = ⋃_{σ∈S} N_{σ} where S is a set of bitstrings, let

T = {τ ∈ 2^{<ℕ} | ¬ (∃m ≤ |τ|) (τ ↾ m ∈ S)} .

Clearly T is a tree. T is finitely branching, because T ⊆ 2^{<ℕ}. T has no path, because if X ∈ 2^{ℕ} were a path through T we would have X ↾ n ∈ T for all n, hence X ↾ n ∉ S for all n, hence X ∉ N_{σ} for all σ ∈ S, a contradiction.

Therefore, by König’s Lemma, T is finite. Let n be such that T contains no bitstring of length n. In other words, every bitstring of length n has an initial segment belonging to S. Hence,

2^{ℕ} = ⋃_{σ∈S,|σ|≤n} N_{σ} = ⋃_{σ∈F} N_{σ}

where F = {σ ∈ S | |σ| ≤ n}. Note that F is finite of cardinality ≤ 2^{n}.

Corollary 7.4.5. If 2^{ℕ} = ⋃_{n=0}^{∞} V_{n} with V_{n} open, then 2^{ℕ} = ⋃_{n=0}^{k} V_{n} for some k. In other words, 2^{ℕ} is compact in the usual topological sense:

“Every open covering of 2^{ℕ} has a finite subcovering.”

Proof. This follows easily from Theorem 7.4.1, because an open set is the union of a sequence of neighborhoods.

Lecture 31: November 7, 2007

Remark 7.4.6. Here are a few announcements about upcoming seminars and colloquia relevant to this course.

• Tomorrow, MASS Colloquium: Professor Alexandra Shlapentokh, expert on Hilbert’s 10th Problem.

• Tuesday, MASS Seminar: Professor Peter Cholak, expert on randomness, Kolmogorov complexity, etc.

• Tuesday, Logic Seminar: Professor Peter Cholak (2:30-3:45 in 106 McAllister).

We now continue with our discussion of the fact that 2^{ℕ} is compact.

We have proved:

Corollary 7.4.7. If 2^{ℕ} = ⋃_{n=0}^{∞} V_{n} where each V_{n} is an open subset of 2^{ℕ}, then 2^{ℕ} = ⋃_{n=0}^{k} V_{n} for some k. In other words:

“Every open covering of 2^{ℕ} has a finite subcovering.”

Since a closed set is the complement of an open set, we can restate the previous corollary in terms of closed sets, as follows:

Corollary 7.4.8. Assume that

Q_{0} ⊇ Q_{1} ⊇ ⋯ ⊇ Q_{n} ⊇ Q_{n+1} ⊇ ⋯

is a descending sequence of closed sets in 2^{ℕ} and Q_{n} ≠ ∅ for all n. Then ⋂_{n=0}^{∞} Q_{n} ≠ ∅.

Proof. Look at the open sets V_{n} = 2^{ℕ} \Q_{n} and apply the previous corollary.

Remark 7.4.9. Recall also that Σ_{1}^{0} sets are open, and Π_{1}^{0} sets are closed. Hence, the above corollaries apply to these sets as well.

7.5 Σ_{1}^{0} and Π_{1}^{0} predicates in 2^{ℕ}

We now use the compactness of 2^{ℕ} to draw some interesting consequences concerning Σ_{1}^{0} and Π_{1}^{0} predicates.

Recall that we are dealing with three important spaces: ℕ^{ℕ} (the Baire space), 2^{ℕ} (the Cantor space), and ℕ (the natural numbers). Actually we are dealing with “mixed” predicates S ⊆ (ℕ^{ℕ})^{m} × (2^{ℕ})^{l} × ℕ^{k}. For convenience in stating the following lemma, let us abbreviate S(f_{1}, …, f_{m}, X_{1}, …, X_{l}, n_{1}, …, n_{k}) as S(−, −, −).

Lemma 7.5.1. Let f be a variable ranging over ℕ^{ℕ}. Then any Σ_{1}^{0} predicate S(f, −, −, −) can be written in the form

S(f, −, −, −) ≡ ∃n R(f ↾ n, −, −, −)

where R(σ, −, −, −) is a recursive predicate and σ is a variable ranging over strings.

Proof. This follows easily from the idea of finite approximation. In detail we have

S(f, −, −, −) ≡ φ_{e}^{(k),f⊕−⊕−}(−) ↓
≡ ∃s φ_{e,s}^{(k),f↾s⊕−↾s⊕−↾s}(−) ↓
≡ ∃s R(f ↾ s, −, −, −)

where R(σ, −, −, −) is the recursive predicate φ_{e,|σ|}^{(k),σ⊕−↾|σ|⊕−↾|σ|}(−) ↓.

Lemma 7.5.2 (boundedness principle). Let X be a variable ranging over 2^{ℕ}. Let S(X, n, −) be a Σ_{1}^{0} predicate. Then

∀X ∃n S(X, n, −) ≡ ∃k ∀X (∃n < k) S(X, n, −) .

In other words, the n’s on the left hand side are bounded. Proof. Fix − and let V_{n} = {X ∈ 2^{ℕ} | S(X, n, −)}. Since S is Σ_{1}^{0}, V_{n} is open. By Lemma 7.5.1 we have

∀X ∃n S(X, n, −) ≡ ∀X ∃n (X ∈ V_{n})
≡ 2^{ℕ} = ⋃_{n=0}^{∞} V_{n}
≡ ∃k 2^{ℕ} = ⋃_{n=0}^{k−1} V_{n}  (by compactness)
≡ ∃k ∀X (∃n < k) S(X, n, −) .

Lemma 7.5.3 (the Magic Lemma). Let X be a variable ranging over 2^{ℕ}. 1. If S(X, −) is Σ_{1}^{0} then ∀X S(X, −) is Σ_{1}^{0}.

2. If P (X, −) is Π_{1}^{0} then ∃X P (X, −) is Π_{1}^{0}. In other words,

1. The class of Σ_{1}^{0} predicates is closed under ∀X.

2. The class of Π_{1}^{0} predicates is closed under ∃X. These closure properties are useful in Tarski/Kuratowski computations. Proof. We prove only part 1. Part 2 follows by duality. Let S(X, −) be a Σ_{1}^{0} predicate. Use Lemma 7.5.1 to write

S(X, −) ≡ ∃n R(X ↾ n, −)

where R(σ, −) is a recursive predicate. By Lemma 7.5.2 we have

∀X S(X, −) ≡ ∀X ∃n R(X ↾ n, −)
≡ ∃k ∀X (∃n < k) R(X ↾ n, −)
≡ ∃k \underbrace{(∀ bitstrings σ of length k) (∃n < k)}_{bounded number quantifiers} \underbrace{R(σ ↾ n, −)}_{recursive}
≡ Σ_{1}^{0}

Remark 7.5.4. Lemmas 7.5.1, 7.5.2, and 7.5.3 will be useful in Homework #11, Problems 1, 3, and 4. They will also be useful in the proof of the Low Basis Theorem, below.

Corollary 7.5.5. Let

P_{e} = {X ∈ 2^{ℕ} | φ_{e}^{(1),X}(0) ↑}, e = 0, 1, 2, …

be our standard recursive enumeration of all Π_{1}^{0} subsets of 2^{ℕ}. Then, the set {e | P_{e} ≠ ∅} is Π_{1}^{0}. Also, the sets {3^{i}5^{j} | P_{i} ∩ P_{j} ≠ ∅}, etc., are Π_{1}^{0}.

Proof. By the Magic Lemma 7.5.3 we have

P_{e} ≠ ∅ ≡ \underbrace{∃X \underbrace{(X ∈ P_{e})}_{Π_{1}^{0}}}_{Π_{1}^{0}}

and similarly

P_{i} ∩ P_{j} ≠ ∅ ≡ ∃X (X ∈ P_{i} ∧ X ∈ P_{j}) ≡ Π_{1}^{0}

etc.

Randomness

In this chapter we study Martin-Löf’s concept of randomness. We shall see that it is closely related to Kolmogorov’s concept of complexity.

Lecture 23: October 19, 2007

8.1 Foundations of mathematics

The purpose of foundations of mathematics is to understand very clearly and precisely the most basic concepts of mathematics. We wish to answer questions such as:

1. What is a number?

2. What is a shape?

3. What is a set?

4. What is a function?

5. What is an axiom?

6. What is a theorem?

7. What is a proof?

8. What is an algorithm? All of mathematics is built on these concepts, yet often we proceed without having a precise idea of what they mean. There is a lot of interest in these questions. Moreover, the issue of how these questions should be answered is an important topic in philosophy of mathematics.

Researchers in foundations of mathematics have made a lot of progress on these questions. Concepts such as set, function, and number have been grounded and made precise in terms of set theory. Concepts such as theorem, proof, and axiom have been made precise in mathematical logic.

One example we have seen of defining a fundamental concept is Turing’s definition of a computable function, which is widely recognized as the “right definition”, i.e., the right answer to the question

What is a computable function?

Turing’s definition has intuitive appeal. The functions which we believe should be computable satisfy the definition, and conversely. Before Turing, the idea of a computable function had not been made precise.

In a similar vein, we now wish to clarify the concept of randomness, i.e., to answer the question

What is a random point in a probability space?

8.2 Definitions of randomness

We wish to define what we mean by a random point in a probability space.

To keep things simple, we consider only one probability space: the Cantor space, 2^{ℕ}, with the fair coin probability measure, μ. Recall that each bitstring σ ∈ 2^{<ℕ} determines a neighborhood N_{σ} = {X ∈ 2^{ℕ} | σ ⊂ X} in 2^{ℕ}. The measure of N_{σ} is μ(N_{σ}) = 1/2^{|σ|} = Prob(σ is an initial segment of X). The measure μ has the following properties:

1. μ(2^{ℕ}) = 1 and μ(∅) = 0.

2. μ(⋃_{i=0}^{∞} S_{i}) = ∑_{i=0}^{∞} μ(S_{i}) provided the sets S_{i}, i = 0, 1, 2, … are pairwise disjoint.

3. μ(2^{ℕ} \ S) = 1 − μ(S).

We wish to define what we might mean by saying that a point X ∈ 2^{ℕ}, X = an infinite sequence of 0’s and 1’s, is random. Our first attempt is as follows.

Definition 8.2.1 (non-rigorous). A point X ∈ 2^{ℕ} is said to be random if it is the outcome of an infinite sequence of tosses of an unbiased coin, identifying heads as 1 and tails as 0.

This definition, although not rigorous, provides some guidance. For example, we would not expect a random X to have X(2n) = 1 for all n, because this would mean that all of the even-numbered coin tosses result in heads, an event which is highly unlikely. This corresponds to the fact that

Prob(∀n (X(2n) = 1)) = μ{X ∈ 2^{ℕ} | ∀n (X(2n) = 1)} = 0.

These considerations suggest the following attempt at defining randomness, using simple concepts from measure theory.

Definition 8.2.2 (temporary). A point X ∈ 2^{ℕ} is said to be random if X does not belong to any set S ⊆ 2^{ℕ} which is of measure 0.

This definition of randomness has the advantage of being perfectly rigorous. Furthermore, if X is random according to this definition, then X obviously has many properties which are intuitively associated with sequences of coin tosses. For instance, if X is random then ¬ ∀n (X(2n) = 1) as desired. In fact, if X is random, then no event of probability 0 occurs.

A fatal difficulty with this definition is that, under this definition, random points do not exist! For any point X ∈ 2^{ℕ}, the singleton set {X} is a null set, and X ∈ {X}, so X is not random. Thus, the above definition of randomness turns out to be uninteresting.

We therefore discard the above definition and replace it be another definition in which we consider only “nice” sets of measure 0, instead of arbitrary sets of measure 0. In order to define what we mean by a “nice” set of measure 0, we use concepts from recursion theory.

Definition 8.2.3. For each n ≥ 1, a point X ∈ 2^{ℕ} is said to be weakly n-random if X ∉ any Π_{n}^{0} set of measure 0. Equivalently, X ∉ any Σ_{n+1}^{0} set of measure 0.

Remark 8.2.4. Note that X ∉ any Π_{n}^{0} set of measure 0 if and only if X ∉ any Σ_{n+1}^{0} set of measure 0. This is because a Σ_{n+1}^{0} set is a union of Π_{n}^{0} sets.

Remark 8.2.5. In the previous definition, we do not consider Σ_{1}^{0} sets of measure 0. The reason for this restriction is that the only Σ_{1}^{0} set of measure 0 is the empty set. If we were to make the above definition with Σ_{1}^{0} instead of Π_{n}^{0}, then all points of 2^{ℕ} would be random, so the definition would be uninteresting.

Lemma 8.2.6. There exist points X ∈ 2^{ℕ} which are weakly n-random.

Proof. There are only countably many Π_{n}^{0} sets. Hence, there are only countably many Π_{n}^{0} sets of measure 0. Let S_{n} = the union of all Π_{n}^{0} sets of measure 0. Equivalently, S_{n} is the union of all Σ_{n+1}^{0} sets of measure 0. By countable additivity, μ(S_{n}) = 0. Any X ∉ S_{n} is weakly n-random.

Remark 8.2.7. Note that S_{n} itself is not Π_{n}^{0} or even Σ_{n+1}^{0}.

Remark 8.2.8. It is clear that, for each n ≥ 1, weak n + 1-randomness implies weak n-randomness. We shall see later that the converse does not hold.

Definition 8.2.9. A point X ∈ 2^{ℕ} is said to be arithmetically random if X ∉ any Π_{n}^{0} set of measure 0 for any n ≥ 1. Equivalently, X is weakly n-random for all n ≥ 1.

Remark 8.2.10. They exist because, by countable additivity, μ(⋃_{n=1}^{∞} S_{n}) = 0.

Remark 8.2.11. If X is weakly 1-random, then X has at least some of the desirable properties which we would normally expect of sequences of coin tosses. For example, ¬ ∀n (X(2n) = 1). This is because the set {X ∈ 2^{ℕ} | ∀n (X(2n) = 1)} is Π_{1}^{0} of measure 0.

Similarly we can show that if X is weakly 1-random then X is the characteristic function of a biimmune set. (A set B ⊆ ℕ is said to be biimmune if both B and the complement of B are immune.) See also Homework #9 Problem 6. Note also that this gives an example of two immune sets whose union is not immune. Other examples of biimmune sets can be constructed using finite approximation, but the present example in terms of weak 1-randomness is perhaps easier and more interesting.

Remark 8.2.12. From the previous remark, we see that the concept of weak 1-randomness is useful. However, we shall see later that this concept is not really what we want. For example, it is possible for X to be weakly 1-random yet not obey the Strong Law of Large Numbers.

A better concept of randomness is due to P. Martin-Löf in a paper published in 1966. We shall see that Martin-Löf’s concept of randomness is intermediate between weak 1-randomness and weak 2-randomness and implies essentially all desirable statistical properties which would normally be expected of an infinite sequence of coin tosses.

Our goal now is to present Martin-Löf’s definition of randomness. First we need some preliminary definitions.

Definition 8.2.13. A null set is a set S ⊆ 2^{ℕ} which is of measure 0, i.e., μ(S) = 0.

Remark 8.2.14. A well known fact is that S is null if and only if

(∀ϵ > 0) (∃ open set V) (S ⊆ V ∧ μ(V) < ϵ) .

This follows from the fact that the fair coin probability measure μ is regular.

Next we are going to “effectivize” the concept of a null set. This means that we are going to define a more restricted concept which pays more attention to computability.

Definition 8.2.15.

1. A set V ⊆ 2^{ℕ} is said to be effectively open if V is Σ_{1}^{0}. (See also Theorem 7.3.9.)

2. A set S ⊆ 2^{ℕ} is said to be effectively null if there exists a recursive sequence of effectively open sets V_{n}, n = 0, 1, 2, …, such that

∀n (S ⊆ V_{n} ∧ μ(V_{n}) ≤ 1/2^{n}) .

Remark 8.2.16. Recall that our standard recursive enumeration of all Σ_{1}^{0} subsets of 2^{ℕ} is given by

U_{e} = {X ∈ 2^{ℕ} | φ_{e}^{(1),X}(0) ↓}

where e = 0, 1, 2, . . .. By a recursive sequence of effectively open sets we mean a sequence V_{n} = U_{f(n)}, n = 0, 1, 2, …, where f(n) is a total recursive function. In this case we could also say that the sequence V_{n}, n = 0, 1, 2, … is uniformly effectively open, or we could say that the sequence is uniformly Σ_{1}^{0}.

Remark 8.2.17. Obviously every effectively null set is a null set, but the converse does not hold. Later we shall prove the surprising result that the union of all effectively null sets is an effectively null set. Thus, there is a unique largest effectively null set.

Finally we are able to present Martin-Löf’s definition of randomness.

Definition 8.2.18 (Martin-Löf). A point X ∈ 2^{ℕ} is said to be random if X ∉ any effectively null set. Equivalently, the singleton set {X} is not effectively null.

Exercise 8.2.19. Let X ∈ 2^{ℕ} be random in the sense of Martin-Löf. Prove that for all bitstrings σ there exist infinitely many n such that X(n + i) = σ(i) for all i < |σ|.

Hint: Given a bitstring σ, construct a Martin-Löf test appropriate for σ. In other words, construct an effectively null set S_{σ} such that every X ∉ S_{σ} has the desired property.

8.3 Homework #9, due October 29, 2007

Exercises 8.3.1.

1. Hoeffding’s Inequality says that the probability space 2^{ℕ} with the fair coin probability measure satisfies

Prob (|\frac{∑_{i=0}^{n−1} X(i)}{n} − \frac{1}{2}| > ϵ) < \frac{2}{exp 2nϵ^{2}} .

Use Hoeffding’s Inequality to prove that if a point X ∈ 2^{ℕ} is random (i.e., random in the sense of Martin-Löf), then X obeys the Strong Law of Large Numbers:

\frac{∑_{i=0}^{n−1} X(i)}{n} → \frac{1}{2}  as  n → ∞ .

2. Prove that there exist weakly 1-random points in 2^{ℕ} which do not obey the Strong Law of Large Numbers.

Hint: Use finite approximation.

3. In problem 1, can you say anything about the rate of convergence to 1/2?

4. Prove that if X ⊕ Y ∈ 2^{ℕ} is random (i.e., random in the sense of Martin-Löf), then X ≰_{T} Y and Y ≰_{T} X.

5. Prove that there exist points X, Y ∈ 2^{ℕ} such that X ⊕ Y is weakly 1random yet X ≡_{T} Y.

6. A set B ⊆ ℕ is said to be biimmune if both B and its complement ℕ \ B are immune. Prove that if X ∈ 2^{ℕ} is weakly 1-random then X is the characteristic function of a biimmune set.

7. Let f be a Turing oracle.

For each i ∈ ℕ define

U_{i}^{f} = {X ∈ 2^{ℕ} | φ_{i}^{(1),f⊕X}(0) ↓}.

Thus U_{i}^{f}, i = 0, 1, 2, … is the standard recursive enumeration of all Σ_{1}^{0,f} subsets of 2^{ℕ}.

Given a sequence of sets V_{n} ⊆ 2^{ℕ}, n = 0, 1, 2, …, prove that the following are pairwise equivalent.

(a) There exists a total recursive function g such that V_{n} = U_{g(n)}^{f} for all n.

(b) There exists a total f-recursive function h such that V_{n} = U_{h(n)}^{f} for all n.

(c) The predicate P ⊆ 2^{ℕ} × ℕ given by

P (X, n) ≡ X ∈ V_{n}

is Σ_{1}^{0,f}.

In this case we say that the sequence of sets V_{n}, n = 0, 1, 2, … is uniformly Σ_{1}^{0,f} or uniformly Σ_{1}^{0} relative to f.

Note: This concept will be part of the definition of what it means for a point X ∈ 2^{ℕ} to be random relative to the oracle f.

Lecture 24: October 22, 2007

8.4 Properties of Martin-Löf randomness

Review:

We defined a set S ⊆ 2^{ℕ} to be effectively null if S ⊆ ⋂_{n=0}^{∞} V_{n} where V_{n}, n = 0, 1, 2, … is uniformly Σ_{1}^{0} and μ(V_{n}) ≤ 1/2^{n}. We defined a point X ∈ 2^{ℕ} to be random (in the sense of Martin-Löf) if X ∉ any effectively null set.

Remark 8.4.1. Homework #9 Problems 1, 3, and 4 show that if X is random then X has various desirable properties which we would attribute to a sequence of coin tosses:

1. SLLN = Strong Law of Large Numbers:

lim_{n→∞} \frac{∑_{i=0}^{n−1} X(i)}{n} = \frac{1}{2} .

In other words, “the proportion of heads in the first n coin tosses goes to 1/2 as n goes to infinity.”

2. For X ∈ 2^{ℕ} we can write X uniquely as X = X_{0} ⊕ X_{1} where X_{0} and X_{1} are the even and odd parts of X, i.e., X_{0}(n) = X(2n) and X_{1} = X(2n+1) for all n. Then, X random implies X_{0} ≰_{T} X_{1} and X_{1} ≰_{T} X_{0}. In other words, “the even part does not help us to compute the odd part, and vice versa.”

Results such as these tend to justify the Martin-Löf definition of randomness.

We now compare Martin-Löf’s concept of randomness to weak n-randomness, n = 1, 2, . . ..

Lemma 8.4.2. Let P be a subset of 2^{ℕ}.

1. If P is Π_{1}^{0} and null, then P is effectively null.

2. If P is effectively null, then P ⊆ S for some S which is Π_{2}^{0} and effectively null.

Proof. 1. If P is Π_{1}^{0}, we can write P = {paths through T } where T is a recursive tree. Then

P = ⋂_{n=0}^{∞} V_{n}, where V_{n} = ⋃_{\begin{matrix}τ ∈ T \\ |τ| = n \end{matrix}} N_{τ}

with V_{0} ⊇ V_{1} ⊇ V_{2} ⊇ ⋯ ⊇ V_{n} ⊇ ⋯ and V_{n}, n = 0, 1, 2, … is uniformly Σ_{1}^{0}. (In fact, V_{n} is uniformly Δ_{1}^{0}.) Hence μ(P) = lim_{n→∞} μ(V_{n}). Clearly the function μ(V_{n}) is a recursive function of n, because

μ(V_{n}) = ∑_{\begin{matrix}τ ∈ T \\ |τ| = n \end{matrix}} \frac{1}{2^{n}} .

See also Homework #8 Problem 4. If P is null, then

μ(P) = lim_{n→∞} μ(V_{n}) = 0

so let f(k) = the least n such that μ(V_{n}) ≤ 1/2^{k}. Then f is a recursive function, so the sets V_{k}^{∗} = V_{f(k)}, k = 0, 1, 2, … are uniformly Σ_{1}^{0} and μ(V_{k}^{∗}) ≤ 1/2^{k}. We now see that

P = ⋂_{n=0}^{∞} V_{n} = ⋂_{k=0}^{∞} V_{k}^{∗}

is an effectively null set.

2. If P is effectively null, we have P ⊆ ⋂_{n=0}^{∞} V_{n}, μ(V_{n}) ≤ 1/2^{n}, V_{n} uniformly Σ_{1}^{0}. Letting S = ⋂_{n=0}^{∞} V_{n} we see that P ⊆ S, S is effectively null, and

X ∈ S ≡ \underbrace{∀n (\underbrace{X ∈ V_{n}}_{Σ_{1}^{0}})}_{Π_{2}^{0}}

so S is Π_{2}^{0}.

Theorem 8.4.3. Let X ∈ 2^{ℕ}.

1. X random ⇒ X weakly 1-random.

2. X weakly 2-random ⇒ X random.

Proof. 1. Suppose X is random. To show X is weakly 1-random, consider any null Π_{1}^{0} set P. By part 1 of Lemma 8.4.2, P is effectively null. It follows that X ∉ P.

2. Assume that X is weakly 2-random, i.e., X ∉ any null Π_{2}^{0} set. It follows by part 2 of Lemma 8.4.2 that X ∉ any effectively null set, i.e., X is random.

Remark 8.4.4. From now on we shall write

random  ≡ random in the sense of Martin-Löf,
weakly random ≡ weakly 1-random,
strongly random ≡ weakly 2-random.

The previous theorem tells us that strongly random ⇒ random, and random ⇒ weakly random. We shall see later that the converses do not hold.

Remark 8.4.5. Random ⇒ not recursive. In fact, weakly random ⇒ not recursive. To see this, note that for all X ∈ 2^{ℕ} we have {X} = P = {paths through T } where T = {X ↾ n | n ∈ ℕ} is a tree and P is a null set. If X is recursive, then T is recursive, hence P is a Π_{1}^{0} null set, hence X is not weakly random.

An important technical lemma is:

Lemma 8.4.6 (Solovay’s Lemma). Let X ∈ 2^{ℕ} be random. Let V_{n}, n = 0, 1, 2, … be uniformly Σ_{1}^{0} subsets of 2^{ℕ} such that

∑_{n=0}^{∞} μ(V_{n}) < ∞ .

Then X ∈ V_{n} for only finitely many n. In other words, X ∉ V_{n} for all sufficiently large n.

Note: This lemma may be useful in Homework #9, Problem 1.

Proof. By assumption, let c be a constant such that

∑_{n=0}^{∞} μ(V_{n}) ≤ 2^{c} < ∞ .

For k = 0, 1, 2, … consider the sets

W_{k} = {X ∈ 2^{ℕ} | X ∈ V_{n} for at least k many n’s}

and note that these sets are uniformly Σ_{1}^{0}:

X ∈ W_{k} ≡ \underbrace{∃n_{1} < ⋯ < n_{k} \underbrace{∀i ≤ k}_{bounded quantification} (\underbrace{X ∈ V_{n_{i}}}_{Σ_{1}^{0}})}_{Σ_{1}^{0}}

We claim that μ(W_{k}) ≤ 2^{c}/k for all k.

Assuming this claim, we have μ(W_{2^{c+k}}) ≤ 2^{c}/2^{c+k} = 1/2^{k} and these sets are also uniformly Σ_{1}^{0}. Therefore, since X is random, X ∉ W_{2^{c+k}} for some k. It follows that X ∈ V_{n} for < 2^{c+k} many n’s. This proves Solovay’s Lemma.

It remains to prove the claim. We have

W_{k} = {X ∈ 2^{ℕ} | (∃^{≥k}n) (X ∈ V_{n})}

so for all s let

W_{k,s} = {X ∈ 2^{ℕ} | (∃^{≥k}n ≤ s) (X ∈ V_{n})} .

To simplify the calculations, let us identify the sets V_{n}, W_{k}, W_{k,s} with their characteristic functions. Thus, μ(V_{n}) = ∫_{X∈2^{ℕ}} V_{n}(X) dX, etc. We have

2^{c} ≥ ∑_{n=0}^{∞} μ(V_{n}) ≥ ∑_{n=0}^{s} μ(V_{n})

= ∑_{n=0}^{s} ∫_{X} V_{n}(X) dX

= ∫_{X} ∑_{n=0}^{s} V_{n}(X) dX  (the sum is ≥ k if X ∈ W_{k,s})
≥ ∫_{X} kW_{k,s}(X) dX = kμ(W_{k,s}) .

But obviously W_{k} = ⋃_{s=0}^{∞} W_{k,s}, hence μ(W_{k}) = lim_{s→∞} μ(W_{k,s}), hence our calculation above shows that 2^{c} ≥ kμ(W_{k}). This proves the claim and completes the proof of Solovay’s Lemma.

Remark 8.4.7. Solovay’s Lemma is a recursion-theoretic refinement of the Borel/Cantelli lemma in probability theory. Solovay’s Lemma is frequently used in order to prove that a random X behaves as we would expect.

We now prove another important theorem about Martin-Löf’s concept of randomness.

Theorem 8.4.8. The union of all effectively null sets is effectively null.

Remark 8.4.9. We can rephrase the theorem as, “there is a universal effectively null set.” Or, “there is a universal test for randomness.”

By a test for randomness we mean a uniformly Σ_{1}^{0} sequence of sets V_{n}, n = 0, 1, 2, … such that μ(V_{n}) ≤ 1/2^{n} for all n. We say that a point X ∈ 2^{ℕ} passes the test if X ∉ ⋂_{n=0}^{∞} V_{n}. Our definition of randomness says that X is random if and only if it passes all tests for randomness. The theorem says that there is a particular test for randomness which is universal in the following sense: if X passes this particular test for randomness, then it passes all tests for randomness and is therefore random.

Proof. Recall our standard recursive enumeration of all Σ_{1}^{0} subsets of 2^{ℕ},

U_{e} = {X ∈ 2^{ℕ} | φ_{e}^{(1),X}(0) ↓} .

Define

U_{e,s} = {X ∈ 2^{ℕ} | φ_{e,s}^{(1),X↾s}(0) ↓}

and note that U_{e,s} is uniformly Σ_{1}^{0} (in fact, uniformly Δ_{1}^{0}) and U_{e} = ⋃_{s=0}^{∞} U_{e,s}. Furthermore, μ(U_{e,s}) is a rational number and is recursive as a function of e, s. This is because

U_{e,s} = ⋃_{\begin{matrix}φ_{e,|σ|}^{(1),σ}(0) ↓ \\ |σ| = s \end{matrix}}N_{σ}

and

μ(U_{e,s}) = ∑_{\begin{matrix}φ_{e,|σ|}^{(1),σ}(0) ↓ \\ |σ| = s \end{matrix}}\frac{1}{2^{s}}

and the predicates φ_{e,s}^{(1),σ}(0) ↓ and |σ| = s are recursive. Note that σ ranges over bitstrings of length s.

Given a rational number r, define

U_{e}[r] = ⋃_{μ(U_{e,s})≤r} U_{e,s} .

Intuitively U_{e}[r] is “U_{e} enumerated so long as its measure is ≤ r”.

Some easily verified facts are:

1. U_{e}[r] ⊆ U_{e}.

2. μ(U_{e}[r]) ≤ r.

3. If μ(U_{e}) ≤ r then U_{e}[r] = U_{e}.

4. The sets U_{e}[r] are uniformly Σ_{1}^{0}. In other words, the 3-place predicate P (X, e, r) ≡ X ∈ U_{e}[r] is Σ_{1}^{0}.

Now define

V_{e,n} = \{ \begin{matrix}U_{i}[1/2^{n}] & if φ_{e}^{(1)}(n) ≃ i , \\ ∅ & if φ_{e}^{(1)}(n) ↑ . \end{matrix}

Some easy facts are:

1. V_{e,n} is uniformly Σ_{1}^{0}.

Namely, X ∈ V_{e,n} ≡ ∃i (φ_{e}^{(1)}(n) ↓ ≃ i ∧ X ∈ U_{i}[1/2^{n}]) which is obviously Σ_{1}^{0}.

2. μ(V_{e,n}) ≤ 1/2^{n} for all n.

Therefore, for each e, the sequence of sets V_{e,n}, n = 0, 1, 2, … is a test for randomness. Moreover, we claim that all tests for randomness are among these. To see this, suppose that V_{n}, n = 0, 1, 2, … is a test for randomness, say V_{n} = U_{f(n)} = U_{f(n)}[1/2^{n}] for some recursive function f(n). Let e be an index of f, so that f(n) ≃ φ_{e}^{(1)}(n) for all n. Then clearly V_{n} = V_{e,n} for all n.

Now, to obtain a universal test for randomness, we diagonalize over all tests for randomness by letting

Ṽ_{n} = ⋃_{e=0}^{∞} V_{e,e+n+1} .

Then X ∈ Ṽ_{n} ≡ ∃e (X ∈ V_{e,e+n+1}) so the sequence Ṽ_{n}, n = 0, 1, 2, …, is uniformly Σ_{1}^{0}. Moreover

μ(Ṽ_{n}) ≤ ∑_{e=0}^{∞} μ(V_{e,e+n+1})

≤ ∑_{e=0}^{∞} \frac{1}{2^{e+n+1}}
= \frac{1}{2^{n}} (\frac{1}{2} + \frac{1}{4} + ⋯)
= \frac{1}{2^{n}}

so Ṽ_{n}, n = 0, 1, 2, … is a test for randomness. We claim that it is a universal test for randomness. In other words, for all e,

⋂_{n=0}^{∞} V_{e,n} ⊆ ⋂_{n=0}^{∞} Ṽ_{n} .

This is easily verified: if X ∈ ⋂_{n=0}^{∞} V_{e,n} then X ∈ V_{e,e+n+1} for all n, hence X ∈ Ṽ_{n} for all n, hence X ∈ ⋂_{n=0}^{∞} Ṽ_{n}. This completes the proof.

Remark 8.4.10. We view the existence of a universal test for randomness as providing good evidence for the “naturalness” of our concept of randomness. Similarly, Turing’s theorem stating the existence of a universal partial recursive function provides good evidence for the “naturalness” of our concept of computable function.

Questions about the “naturalness” of various concepts in mathematics are extremely important. This is because, as mathematicians, we have only a finite amount of time to spend on mathematical research, and therefore it is extremely important to choose the right research topics. If a mathematical concept is “natural” or “interesting”, then this suggests that time spent studying the concept will be well spent.

Lecture 26: October 26, 2007

8.5 Comments on Homework #8

Problem 1

Recall that the machine M is prefix-free if dom(M) is prefix-free. Define a prefix-free machine, M, by M(σ_{1}^{⌢}σ_{2}) ≃ U(σ_{1})^{⌢}U(σ_{2}). You must check that

1. M is single-valued; i.e., M(σ) is well-defined.

2. M is partial recursive.

3. dom(M) is prefix-free.

Problem 3

Let r ∈ ℝ be a real number which is both left r.e. and right r.e. In other words,

r = lim_{n} a_{n}, a_{n} ↗ (increasing)

and

r = lim_{n} b_{n}, b_{n} ↘ (decreasing)

where a_{n} and b_{n} are recursive sequences of rational numbers. We need to show that r is a recursive real number.

One solution is to define f(k) = μn (|a_{n} − b_{n}| < 1/2^{k}). Then f is a recursive function, r = lim_{k→∞} a_{f(k)}, and |a_{f(k)} − r| < 1/2^{k}. So the recursive sequence of rational numbers a_{f(k)}, k = 0, 1, 2, … witnesses that r is a recursive real number.

Alternatively, we can use the characterization of recursive real numbers as a real number r such that g(n) =“nth decimal digit of r” is recursive. It is natural to try something like this:

“Let h(k) = the least n such that a_{n} and b_{n} have the same first k +1 digits, and then let g(k) = the kth digit of a_{h(k)}.”

However, this does not always work! Consider r = 0.1 with a_{n} = 0.0 \underbrace{99 … 9}_{n} and

b_{n} = 0.1 \underbrace{00 … 0}_{n} 1. In this case and for many other rational r, h(k) is undefined.

We can get around this difficulty by considering rational and irrational r as separate cases.

Problem 4

By part (e) we know that for any Π_{1}^{0} set P ⊆ 2^{ℕ} the real number μ(P) is right recursively enumerable. Part (f) asks us to find a P such that μ(P) is not recursive. The easiest example is

P = 2^{ℕ} \ ⋃_{e∈H} N_{⟨\underbrace{0, …, 0}_{e},1⟩}

where H is the Halting Problem. Clearly P is Π_{1}^{0} and

μ(P) = 1 − ∑_{e∈H} \frac{1}{2^{e+1}}

which is a nonrecursive real number.

Problem 5

To find an example of a nonempty Π_{1}^{0} subset of 2^{ℕ} which has no recursive element.

Consider

P = {X ∈ 2^{ℕ} | X separates A, B}

where A, B is a disjoint pair of recursively inseparable r.e. subsets of ℕ. Obviously P is nonempty has no recursive element. For all X ∈ 2^{ℕ} we have

X ∈ P ≡ ¬ ∃n ((X(n) = 0 ∧ n ∈ A) ∨ (X(n) = 1 ∧ n ∈ B)) .

Since A and B are Σ_{1}^{0}, a Tarski/Kuratowski computation shows that P is Π_{1}^{0}. Alternatively, consider

Q = {X ∈ 2^{ℕ} | ∀n (X(n) ≄ φ_{n}^{(1)}(n))}
= {X ∈ 2^{ℕ} | X is diagonally nonrecursive}.

Again, Q is nonempty Π_{1}^{0} and has no recursive element.

Problem 6

Let P ⊆ ℕ^{ℕ} be Π_{2}^{0}. Say f ∈ P ≡ ∀x ∃y R(f, x, y) where R is recursive.

For part (a), let Q = {f ⊕ g | ∀x (g(x) = μyR(f, x, y))}. Clearly Q is Turing isomorphic to P because g ≤_{T} f, hence f ≡_{T} f ⊕ g. To check that Q is Π_{1}^{0}, we have f ⊕ g ∈ Q ≡ ∀x (R(f, x, g(x)) ∧ ¬ ∃y < g(x) R(f, x, y)).

For part (b), let Q = {χ_{G_{f}} | f ∈ P } where G_{f} = {3^{i}5^{j} | f(i) = j} = the “graph” of f. Obviously Q is Turing isomorphic to P, because f ≡_{T} G_{f} ≡_{T} χ_{G_{f}}. It remains to check that Q is Π_{2}^{0}. Recall that f ∈ P ≡ ∀x ∃y R(f, x, y). Define the partial recursive functional Φ(f, x) ≃ μy R(f, x, y). Let e be an index of Φ. By finite approximation we have

f ∈ P ≡ ∀x (φ_{e}^{(1),f}(x) ↓) ≡ ∀x ∃n (φ_{e,n}^{(1),f↾n}(x) ↓) .

We know that X ∈ Q if and only if X is the characteristic function of the “graph” of some function f which belongs to P. Writing this out in detail in terms of e using finite approximation, we have

X ∈ Q ≡ \{ \begin{matrix}∀n [(X(n) = 1) ⇒ (n = 3^{(n)_{1}} · 5^{(n)_{2}})] \\ ∧ ∀m ∀n [(X(m) = 1 ∧ X(n) = 1 ∧ (m)_{1} = (n)_{1}) ⇒ (m)_{2} = (n)_{2}] \\ ∧ ∀i ∃j ∃n [X(n) = 1 ∧ n = 3^{i} · 5^{j}] \\ ∧ ∀x ∃σ (∀i < |σ|) [X(3^{i} · 5^{σ(i)}) = 1 ∧ φ_{e,|σ|}^{(1),σ}(x) ↓] . \end{matrix}

These four lines say that X has the form χ_{G_{f}} where f is single-valued, f is total, and f ∈ P. Thus we see that Q is Π_{2}^{0}.

For part (c), the answer is NO! Not every Π_{2}^{0} subset of 2^{ℕ} is Turing isomorphic to a Π_{1}^{0} subset of 2^{ℕ}. For an example illustrating this, recall the hierarchy based on the jump operator, 0, 0′, 0′′, …, 0^{(n)}, . . .. Post’s Theorem tells us that a set A ⊆ ℕ is Σ_{n}^{0} if and only if A ≤_{m} 0^{(n)}. Define

0^{(ω)} = ⨁_{n=1}^{∞} 0^{(n)} = {3^{m} · 5^{n} | m ∈ 0^{(n)}} .

Thus 0^{(ω)} is outside the arithmetical hierarchy. Identifying the set 0^{(ω)} with its characteristic function in 2^{ℕ}, we can prove that the singleton set {0^{(ω)}} ⊆ 2^{ℕ} is Π_{2}^{0}. On the other hand, we can prove that if P ⊆ 2^{ℕ} is Π_{1}^{0} and nonempty, then P contains some X which belongs to the arithmetical hierarchy. See also Homework #10, Problems 2 and 3.

8.6 Homework #10, due November 5, 2007

Exercises 8.6.1.

1. Let f and g be Turing oracles. Define f ≤_{LK} g to mean that

K^{g}(τ) ≤ K^{f}(τ) + O(1)

for all bitstrings τ. Define f ≤_{LR} g to mean that

(∀X ∈ 2^{ℕ}) (if X is g-random then X is f-random).

(a) Show that f ≤_{T} g implies both f ≤_{LK} g and f ≤_{LR} g.

(b) Let X ∈ 2^{ℕ} be such that X ≤_{LK} 0. Show that X is K-trivial, i.e., K(X ↾ n) ≤ K(n) + O(1) for all n.

Note: It can be shown that the properties f ≤_{LK} g and f ≤_{LR} g are equivalent to each other. However, they are not equivalent to f ≤_{T} g. In fact, we can find a nonrecursive X ∈ 2^{ℕ} such that X ≤_{LK} 0. It can be shown that X ≤_{LK} 0 if and only if X is K-trivial.

2. For convenience in stating this problem, let us identify subsets of ℕ with their characteristic functions. In other words, we identify A ⊆ ℕ with χ_{A} ∈ 2^{ℕ}. Thus 2^{ℕ} is the set of all subsets of ℕ.

Let J : 2^{ℕ} → 2^{ℕ} be the Turing jump operator:

J(X) = X′ = H^{X} = the Halting Problem relative to X.

Recall that 0^{(1)} = 0′ = J(0) and in general 0^{(n+1)} = (0^{(n)})′ = J(0^{(n)}) for all n. By Post’s Theorem we know that for each n ≥ 1 the set 0^{(n)} is Σ_{n}^{0} and not Δ_{n}^{0}. Define

0^{(ω)} = ⨁_{n=1}^{∞} 0^{(n)} = {3^{m}5^{n} | m ∈ 0^{(n)}} .

Note that the set 0^{(ω)} is not arithmetical, i.e., it is not Δ_{n}^{0} for any n.

(a) Show that the 2-place predicate P ⊆ 2^{ℕ} × 2^{ℕ} given by

P (X, Y) ≡ J(X) = Y

is Π_{2}^{0}.

(b) Show that for each n ≥ 1 the singleton set {0^{(n)}} is Π_{2}^{0}.

(c) Show that the singleton set {0^{(ω)}} is Π_{2}^{0}.

Note: These singleton sets are subsets of 2^{ℕ}.

3. (a) Show that every nonempty Π_{1}^{0} subset of 2^{ℕ} contains a member which is Δ_{n}^{0} for some n.

(b) In part (a), what is the optimal value of n?

(c) In parts (a) and (b), what if we replace Π_{1}^{0} sets by Π_{2}^{0} sets?

(d) Is every Π_{2}^{0} subset of 2^{ℕ} Turing isomorphic to a Π_{1}^{0} subset of 2^{ℕ}?

4. Let X ∈ 2^{ℕ}. We say that X is 2-random if X is random relative to 0′. Recall also that X is weakly 2-random if X ∉ any Π_{2}^{0} set of measure 0. Let a = deg_{T} (X) = the Turing degree of X.

(a) Show that if X is 2-random then X is weakly 2-random.

(b) Show that if X is weakly 2-random then inf(a, 0′) = 0.

(c) In part (b) what if we assume only that X is random?

(d) Show that if X is 2-random then sup(a, 0′) = a′.

(e) In part (d) what if we assume only that X is weakly 2-random?

5. Show that every Π_{2}^{0} subset of 2^{ℕ} includes a Σ_{2}^{0,0′} set of the same measure.

8.7 Initial segment complexity

Let X ∈ 2^{ℕ}, an infinite sequence of 0’s and 1’s. We can consider the complexity or prefix-free complexity of the finite initial segments of X: C(X ↾ n) and K(X ↾ n). These quantities are called the initial segment complexity of X. The asymptotic behavior of the initial segment complexity of X as n goes to infinity may be viewed as a measure of the “amount of complexity” inherent in X.

Recall that, roughly speaking, K(|τ|) ≤ K(τ) ≤ |τ| for all strings τ. In other words, K(n) ≤ K(X ↾ n) ≤ n for X ∈ 2^{ℕ} and all positive integers n. And similarly for C. All of these inequalities are modulo an additive constant O(1). Roughly speaking, we have two extreme possibilities for the initial segment complexity of X, given by the following two definitions.

Definition 8.7.1. We say that X is K-trivial if K(X ↾ n) = K(n) ± O(1) for all n. In other words, ∃c ∀n (K(X ↾ n) ≤ K(n) + c). We define C-trivial similarly.

Definition 8.7.2. We say that X is K-random if K(X ↾ n) = n ± O(1) for all n. In other words, ∃c ∀n (K(X ↾ n) ≥ n − c). We define C-random similarly.

The following facts are known and we shall prove some of them.

1. X is C-trivial ⇔ X is recursive.

2. ∃ K-trivial X such that X is not recursive.

3. X is K-random ⇔ X is random.

4. X is C-maximal ⇔ ???

In addition to the two extremes of K-triviality and K-randomness, there are many intermediate possibilities. This leads to a fine classification of X in terms of the “amount of complexity” inherent in X, as measured by initial segment complexity.

Lecture 29: November 2, 2007

We now exhibit a close connection between randomness and Kolmogorov complexity. Recall that X ∈ 2^{ℕ} is said to be K-random if K(X ↾ n) ≥ n − O(1). In other words, ∃c ∀n (K(X ↾ n) ≥ n − c).

Theorem 8.7.3 (Schnorr’s Theorem). X is random ⇔ X is K-random.

Proof. (⇒) Assume X is random. Let V_{c} = {X ∈ 2^{ℕ} | ∃n (K(X ↾ n) < n − c)}. Note V_{c} is uniformly Σ_{1}^{0} for all c = 0, 1, 2, …, namely

X ∈ V_{c} ≡ ∃τ (τ ⊂ X ∧ K(τ) < |τ| − c)
≡ ∃τ (τ ⊂ X ∧ ∃σ (|σ| < |τ| − c ∧ U(σ) ≃ τ))
≡ Σ_{1}^{0}

where U is a universal prefix-free machine.

We claim that μ(V_{c}) < 1/2^{c}. To see this, for each τ such that K(τ) < |τ| − c choose a σ such that U(σ) ≃ τ and |σ| < |τ| − c. We then have

∑_{σ} \frac{1}{2^{|σ|}} ≤ 1

by the Kraft inequality. It follows that

μ(V_{c}) ≤ ∑_{τ} \frac{1}{2^{|τ|}} < ∑_{σ} \frac{1}{2^{|σ|+c}} ≤ \frac{1}{2^{c}}

which proves the claim.

Thus, X random implies X ∉ V_{c} for some c, which means that ∀n K(X ↾ n) ≥ n − c. Hence K(X ↾ n) ≥ n − O(1), i.e., X is K-random, Q.E.D.

(⇐) Assume X is not random. Then X ∈ ⋂_{n=0}^{∞} V_{n} where V_{n} is uniformly Σ_{1}^{0} and μ(V_{n}) ≤ 1/2^{n}. Let T_{n}, n = 0, 1, 2, … be a uniformly recursive, prefix-free set of bitstrings such that V_{n} = ⋃_{τ∈T_{n}} N_{τ}. We have

∑_{n=0}^{∞} ∑_{τ∈T_{2n}} \frac{1}{2^{|τ|−n}} = ∑_{n=0}^{∞} 2^{n}μ(V_{2n}) ≤ ∑_{n=0}^{∞} 2^{n} (\frac{1}{2^{2n}} ) = ∑_{n=0}^{∞} \frac{1}{2^{n}} = 2 .

It follows by Kraft/Chaitin (see Corollary 6.3.11) that K(τ) ≤ |τ| − n + O(1) for all such pairs (τ, n). That is, for some fixed c, K(τ) ≤ |τ| − n + c for all n and all τ ∈ T_{2n}. But then, since X ∈ V_{2n} for all n, we have ∀n ∃m (K(X ↾ m) ≤ m − n + c). In other words, it is not the case that K(X ↾ m) > m − c − O(1) for all m. Thus X is not K-random, Q.E.D.

Remark 8.7.4. Schnorr’s Theorem exhibits a close relationship between our two approaches to randomness. We may view prefix-free complexity K(τ) as the length of the smallest compressed version of τ. So randomness can be seen not only in terms of probability, but also in terms of compressibility of strings.

Remark 8.7.5. As usual, we can relativize all of these concepts and theorems to an arbitrary oracle f ∈ ℕ^{ℕ}. This goes as follows.

Definition 8.7.6. A sequence of sets V_{n} ⊆ 2^{ℕ}, n = 0, 1, 2, … is said to be uniformly Σ_{1}^{0,f} if . . .. The final problem in Homework #9 was to show that three reasonable definitions of this concept coincide.

Definition 8.7.7. We say that X is f-random (i.e., random relative to f) if there is no uniformly Σ_{1}^{0,f} sequence of sets V_{n}, n = 0, 1, 2, … such that X ∈ ⋂_{n=0}^{∞} V_{n} and μ(V_{n}) ≤ 1/2^{n} for all n.

Definition 8.7.8. An f-machine is a partial f-recursive function from bitstrings to bitstrings, M : ⊆ 2^{<ℕ} → 2^{<ℕ}.

Definition 8.7.9. M is prefix-free if ….

Definition 8.7.10. A universal prefix-free f-machine is ….

Definition 8.7.11. We define K^{f}(τ) = min{|σ||U^{f}(σ) ≃ τ} where U^{f} is a universal prefix-free f-machine.

Theorem 8.7.12 (relativization of Schnorr’s Theorem). X is f-random ⇔ X is K^{f}-random.

8.8 Solutions for Homework #9

Problem 1

For all n ≥ 1 and all rational ϵ > 0, let V_{n,ϵ} be the set of X ∈ 2^{ℕ} such that

|\frac{∑_{i=0}^{n−1} X(i)}{n} − \frac{1}{2}| > ϵ .

Note that V_{n,ϵ} is uniformly Σ_{1}^{0}. Hoeffding’s Inequality says that

μ(V_{n,ϵ}) ≤ \frac{2}{e^{2nϵ^{2}}} = 2 (\frac{1}{e^{2ϵ^{2}}} )^{n} .

Since 1/e^{2ϵ^{2}} is < 1, we have

∑_{n=0}^{∞} (\frac{1}{e^{2ϵ^{2}}} )^{n} < ∞ (geometric series)

hence ∑_{n=0}^{∞} μ(V_{n,ϵ}) < ∞. Thus, by Solovay’s Lemma, if X is random then X ∈ V_{n,ϵ} for only finitely many n. Since this holds for all ϵ > 0, we see that X satisfies the Strong Law of Large Numbers,

lim_{n→∞} \frac{∑_{i=0}^{n−1} X(i)}{n} = \frac{1}{2} .

Problem 3

We modify the argument for Problem 1 by letting ϵ depend on n. For instance, we can define

ϵ_{n} = \sqrt{\frac{log n}{n}} .

Letting V_{n} = V_{n,ϵ_{n}} we see that

∑_{n=0}^{∞} μ(V_{n}) ≤ ∑_{n=0}^{∞} \frac{2}{e^{2nϵ_{n}^{2}}} = ∑_{n=0}^{∞} \frac{2}{e^{2 log n}} = ∑_{n=0}^{∞} \frac{2}{n^{2}} < ∞

(p-series, p = 2). So again X ∈ V_{n} for only finitely many n. In other words,

|\frac{∑_{i=0}^{n−1} X(i)}{n} − \frac{1}{2}| ≤ \sqrt{\frac{log n}{n}}

for all sufficiently large n.

Problem 5

For all X ∈ 2^{ℕ} we have X = X_{0} ⊕ X_{1} where X_{0}(n) = X(2n) and X_{1}(n) = X(2n + 1). Thus X_{0} and X_{1} are the even and odd parts of X, respectively.

Problem 4 was to show that if X is random then X_{0} and X_{1} are Turing incomparable.

Problem 5 was to find a weakly random X such that X_{0} ≡_{T} X_{1}. We construct X by finite approximation.

Stage 0. Let σ_{0} = ⟨⟩.

Stage e + 1. Assume we already know σ_{e}.

Case 1: ∃σ ⊇ σ_{e}^{⌢}⟨1, 1⟩ such that φ_{e,|σ|}^{(1),σ}(0) ↓. Let σ_{e+1} = the least such σ which is of even length.

Case 2: not Case 1. Let σ_{e+1} = σ_{e}^{⌢}⟨0, 0⟩.

Finally let X = ⋃_{e=0}^{∞} σ_{e}. Note that |σ_{e}| is even for all e.

Recall our standard recursive enumeration of all Π_{1}^{0} subsets of 2^{ℕ}, namely P_{e} = {X ∈ 2^{ℕ} | φ_{e}^{(1),X}(0) ↑}. Consider what happened at stage e + 1. If Case 1 holds, then X ∉ P_{e}. If Case 2 holds, then N_{σ_{e}^{⌢}⟨1,1⟩} ⊆ P_{e}, hence μ(P_{e}) > 0. Thus X is weakly random.

To see that X_{0} ≡_{T} X_{1}, it suffices to show that the entire construction σ_{e}, e = 0, 1, 2, … is both ≤_{T} X_{0} and ≤_{T} X_{1}. Assume that we are using one of the oracles X_{0} or X_{1} and we have already computed σ_{e}. Note that Case 1 holds at stage e + 1 if and only if X_{0}(|σ_{e}|/2) = 1, if and only if X_{1}(|σ_{e}|/2) = 1. Thus we can use our oracle to tell which case we are in. If we are in Case 1, we can recursively search for σ_{e+1} = the least σ ⊇ σ_{e}^{⌢}⟨1, 1⟩ of even length such that φ_{e,|σ|}^{(1),σ}(0) ↓. Otherwise we are in Case 2 and σ_{e+1} = σ_{e}^{⌢}⟨0, 0⟩. Either way we have now computed σ_{e+1}.

Problem 6

Let X be weakly random. Suppose W_{e} is an infinite r.e. set. Since W_{e} is infinite,

P = {X ∈ 2^{ℕ} | ∀n (n ∈ W_{e} ⇒ X(n) = 1)}

is of measure 0. Since W_{e} is Σ_{1}^{0}, a Tarski/Kuratowski computation shows that P is Π_{1}^{0}. Hence X ∉ P, hence X(n) = 0 for at least one n ∈ W_{e}. Similarly we can show that X(n) = 1 for at least one n ∈ W_{e}. Thus X is the characteristic function of a biimune set.

8.9 Homework #11, due November 12, 2007

Exercises 8.9.1. For f, g ∈ ℕ^{ℕ} say that f is majorized by g if f(n) < g(n) for all n.

1. If P (f, g, −) is a Π_{1}^{0} predicate, prove that the predicate

Q(g, −) ≡ ∃f (P (f, g, −) ∧ f is majorized by g)

is again Π_{1}^{0}.

Note: This is a generalization of the Magic Lemma, Lemma 7.5.3 in the Lecture Notes. You can prove it by imitating the the proof of Lemma 7.5.3.

2. (a) Show that the result of Problem 1 holds if we replace Π_{1}^{0} by Σ_{2}^{0}.

(b) Show that the result does not hold if we replace Π_{1}^{0} by Π_{2}^{0}. In fact, we can find a Π_{2}^{0} predicate P (X, −) with X ranging over 2^{ℕ} such that the predicate ∃X P (X, −) is not arithmetical, i.e., it is not Π_{n}^{0} or Σ_{n}^{0} for any n.

3. Let P ⊆ 2^{ℕ} be Π_{1}^{0}. Let Φ(X, n) be a partial recursive functional such that Φ(X, n) ↓ for all X ∈ P and all n. Find a total recursive function g(n) which majorizes Φ(X, n) for all X ∈ P and all n.

4. An oracle X is said to be hyperimmune-free (sorry for the awkward terminology) if each f ≤_{T} X is majorized by some recursive function.

Note: This is another example of a “lowness property” of X.

(a) Let P ⊆ 2^{ℕ} be nonempty and Π_{1}^{0}. Prove that there exists X ∈ P such that X is hyperimmune-free. This result is known as the Hyperimmune-Free Basis Theorem. Hint: Use Π_{1}^{0} approximation as in the Low Basis Theorem.

(b) Deduce that we can find a random X which is hyperimmune-free.

5. Prove that if 0 <_{T} X ≤_{T} 0′ then X is not hyperimmune-free.

Note: This prevents us from combining the Low Basis Theorem and the Hyperimmune-Free Basis Theorem into one theorem.

Hint for the proof: By Post’s Theorem X is Δ_{2}^{0}. Deduce that the singleton set {X} is Π_{2}^{0}. Use this to find f ≡_{T} X such that the singleton set {f} is Π_{1}^{0}. If such an f is majorized by a recursive function, use the result of Problem 1 to show that f is recursive.

6. (Extra Credit)

(a) Prove that if X is 2-random then X is not hyperimmune-free.

(b) What if we assume only that X is weakly 2-random?

7. (a) Prove that if Y is nonrecursive then μ({X ∈ 2^{ℕ} | Y ≰_{T} X}) = 1.

(b) Deduce that for each nonrecursive Y we can find a random X such that Y ≰_{T} X.

(c) More generally, prove the following. Given a sequence of nonrecursive oracles Y_{i}, i = 0, 1, 2, …, we can find an X which is n-random for all n and such that Y_{i} ≰_{T} X for all i.

Note: It can be shown that for all Y we can find a random X such that Y ≤_{T} X. In fact, each Turing degree ≥ 0′ contains a random X. However, this does not hold for weakly 2-random X’s, because all such X’s are Turing incomparable with 0′.

8. (a) Assume that P ⊆ 2^{ℕ} is Π_{1}^{0} and

¬ ∃X (X ∈ P ∧ X is recursive).

Find a nonrecursive Y such that

¬ ∃X (X ∈ P ∧ X ≤_{T} Y).

Hint: Use finite approximation.

(b) Find a nonrecursive Y such that

¬ ∃X (X is random ∧ X ≤_{T} Y).

Hint: Use the fact that {X | X is random} is the union of a sequence of Π_{1}^{0} sets.

Some advanced topics

In this chapter we present some advanced topics related to computability, unsolvability, and randomness.

Lecture 30: November 5, 2007

9.1 Turing degrees of random sequences

Let X be an infinite sequence of 0’s and 1’s which is random in the sense of Martin-Löf. What can we say about the Turing degree of X? We shall obtain some answers to this question.

Remark 9.1.1. We already know that if X is random then X is nonrecursive, i.e., the Turing degree of X is nonzero. We shall prove:

1. There exist random X’s whose Turing degree is < 0′.

2. There exist random X’s whose Turing degree is incomparable with 0′ (see below for more details).

3. There exist nonzero Turing degrees b such that no Turing degree ≤ b contains a random X.

4. Every Turing degree ≥ 0′ contains a random X (if time permits).

Definition 9.1.2. A Turing degree a is said to be low if a′ = 0′.

Definition 9.1.3. For f, g ∈ ℕ^{ℕ} we say f is dominated by g if f(n) < g(n) for all sufficiently large n. We say that f is majorized by g if f(n) < g(n) for all n.

Definition 9.1.4. A Turing degree a is said to be hyperimmune-free if every function of degree a is dominated by a recursive function. Equivalently, every function of degree a is majorized by a recursive function.

Remark 9.1.5. Both of these properties, lowness and hyperimmune-freeness, say that the Turing degree a is in some sense close to 0. We shall see:

1. There exist random X’s which are low.

2. There exist random X’s which are hyperimmune-free.

3. There do not exist random X’s which are both low and hyperimmune-free.

9.2 The Low Basis Theorem

Definition 9.2.1. X is low if X′ ≡_{T} 0′. Note that this implies X <_{T} 0′.

Theorem 9.2.2 (Low Basis Theorem). Given a nonempty Π_{1}^{0} set P ⊆ 2^{ℕ}, we can find an element X ∈ P such that X is low.

Remark 9.2.3. The Low Basis Theorem should be compared with the result from Homework #8 that there is a nonempty Π_{1}^{0} set P ⊆ 2^{ℕ} with no recursive elements.

In general, a “basis theorem” is a theorem asserting that any nonempty “nice” set must contain a “nice” element. Thus, the low elements of 2^{ℕ} form a “basis” for the Π_{1}^{0} sets, but the recursive elements of 2^{ℕ} do not.

Proof of the Low Basis Theorem. Let P ⊆ 2^{ℕ} be nonempty Π_{1}^{0}. We shall obtain X ∈ P by a technique known as “Π_{1}^{0} approximation.” This means that, starting with P, we shall construct a descending sequence of nonempty Π_{1}^{0} sets

P = Q_{0} ⊇ Q_{1} ⊇ ⋯ ⊇ Q_{e} ⊇ Q_{e+1} ⊇ ⋯

and then let X ∈ ⋂_{e=0}^{∞} Q_{e}. Note that ⋂_{e=0}^{∞} Q_{e} is nonempty in view of Corollary 7.4.8.

We shall perform this construction in such a way as to insure that X is low, i.e., H^{X} ≤_{T} H where H is the Halting Problem. Recall that H^{X} = {e | φ_{e}^{(1),X}(0) ↓}. Actually, the entire construction will be ≤_{T} H.

Here is the construction.

Stage 0: Let Q_{0} = P.

Stage e + 1: The purpose of this stage is to decide whether e ∈ H^{X} or not. Assume inductively that Q_{e} is already known and is a nonempty Π_{1}^{0} set.

Case 1: (∃X ∈ Q_{e}) (φ_{e}^{(1),X}(0) ↑). In this case let

Q_{e+1} = {X ∈ Q_{e} | φ_{e}^{(1),X}(0) ↑}

and note that Q_{e+1} is a nonempty Π_{1}^{0} set. Moreover, e ∉ H^{X} for all X ∈ Q_{e+1}.

Case 2: Not case 1. I.e., (∀X ∈ Q_{e}) (φ_{e}^{(1),X}(0) ↓). In this case let Q_{e+1} = Q_{e}. Again Q_{e+1} is a nonempty Π_{1}^{0} set. Moreover, e ∈ H^{X} for all X ∈ Q_{e+1}.

The construction insures that e ∈ H^{X} if and only if Case 2 holds at stage e + 1. It remains to verify that the entire construction is ≤_{T} H. It will then follow that H^{X} ≤_{T} H, i.e., X is low.

Here are the details of why the entire construction is ≤_{T} H.

We shall use our standard recursive enumeration P_{e}, e = 0, 1, 2, … of all Π_{1}^{0} subsets of 2^{ℕ}, namely

P_{e} = {X ∈ 2^{ℕ} | φ_{e}^{(1),X}(0) ↑} .

We say that e is an index of the Π_{1}^{0} set P_{e}. Note also that X′ = H^{X} = {e | φ_{e}^{(1),X}(0) ↓} = {e | X ∉ P_{e}}.

Recall that if P, Q are Π_{1}^{0} subsets of 2^{ℕ}, then P ∩ Q is of course Π_{1}^{0} and this holds uniformly with respect to the indices. In other words, we can find a recursive function f(i, j) such that P_{f(i,j)} = P_{i} ∩ P_{j} for all i, j. (This is proved by means of the Parametrization Theorem.)

To see that the construction is ≤_{T} H, we shall define a function g(e) and define Q_{e} as Q_{e} = P_{g(e)}. In other words, g(e) will be an index of Q_{e}. To define g, let B = {3^{i} · 5^{j} | P_{i} ∩ P_{j} ≠ ∅}. By Lemma 7.5.3, B is Π_{1}^{0}. See also Corollary 7.5.5. It follows that B ≤_{T} 0′.

Our construction may be described as follows. Start by letting g(0) be an index of P. If P_{g(e)} ∩ P_{e} ≠ ∅, let g(e + 1) = f(g(e), e), so that P_{g(e+1)} = P_{g} ∩ P_{e}. If P_{g(e)} ∩ P_{e} = ∅, let g(e + 1) = g(e), so that P_{g(e+1)} = P_{g(e)}. Note that g ≤_{T} B ≤_{T} 0′. Moreover, for all e, e ∈ H^{X} if and only if 3^{g(e)}5^{e} ∉ B. Hence X′ = H^{X} ≤_{T} B ≤_{T} 0′. Thus X is low. This completes the proof.

We now apply the Low Basis Theorem to draw a conclusion about randomness. First, note the following:

Theorem 9.2.4. Let

R = {X ∈ 2^{ℕ} | X is (Martin-Löf) random}.

Then R is Σ_{2}^{0}.

We present two proofs of this result.

First Proof. Our first proof is based on the existence of a universal test for randomness. Let V_{n}, n = 0, 1, 2, ⋯ be a universal test for randomness. Thus V_{n} is uniformly Σ_{1}^{0}, μ(V_{n}) ≤ 1/2^{n} (a test), and ∀X (X is random ⇔ X passes the test, i.e., X ∉ ⋂_{n=0}^{∞} V_{n}). Then

R = \underbrace{2^{ℕ} \ \underbrace{⋂_{n=0}^{∞} V_{n}}_{Π_{2}^{0}}}_{Σ_{2}^{0}} .

Second Proof. Our second proof is based on Schnorr’s Theorem. We have

X ∈ R ≡ K(X ↾ n) ≥ n − O(1)
≡ ∃c ∀n (K(X ↾ N) ≥ n − c)
≡ \underbrace{∃c \underbrace{∀n ∀σ [\underbrace{(U(σ) ≃ X ↾ n)}_{Σ_{1}^{0}} ⇒ |σ| ≥ n − c]}_{Π_{1}^{0}}}_{Σ_{2}^{0}} .

Corollary 9.2.5. We can find a nonempty Π_{1}^{0} set P ⊆ 2^{ℕ} such that ∀X (X ∈ P ⇒ X is random).

Proof. Since R is Σ_{2}^{0}, R is the union of a sequence of Π_{1}^{0} sets. Since R is nonempty, at least one of these is nonempty.

Remark 9.2.6.

1. We can actually find P as in the corollary such that μ(P) ≥ 1 − ϵ for any ϵ > 0. This follows easily from the fact that μ(R) = 1.

2. It can be shown that any P as in the corollary is Turing isomorphic to R (Kučera 1985).

Corollary 9.2.7. We can find X ∈ 2^{ℕ} such that X is random and low.

Proof. This follows from the previous corollary plus the Low Basis Theorem.

Similar to the Low Basis Theorem, there is the Hyperimmune-Free Basis Theorem:

Theorem 9.2.8 (Hyperimmune-Free Basis Theorem). If P ⊆ 2^{ℕ} is Π_{1}^{0}, nonempty then ∃X ∈ P (X is hyperimmune-free).

Proof. See Homework #11 Problem 4. The proof is by Π_{1}^{0} approximation as in the proof of the Low Basis Theorem. Starting with P define a descending sequence of nonempty Π_{1}^{0} sets

P = Q_{0} ⊇ Q_{1} ⊇ Q_{2} ⊇ ⋯ ⊇ Q_{e} ⊇ · · · .

By compactness, ⋂_{n=0}^{∞} Q_{n} is nonempty. Break the definition of hyperimmunefreeness into countably many requirements, and at stage e + 1 construct Q_{e+1} to satisfy requirement e. The details are left to the student.

Corollary 9.2.9. We can find X ∈ 2^{ℕ} which is random and hyperimmune-free.

Proof. This follows from the Hyperimmune-Free Basis Theorem, just as the previous corollary followed from the Low Basis Theorem.

Remark 9.2.10. We cannot combine these corollaries to get a random X which is both low and hyperimmune-free. In fact, the only Turing degree which is both low and hyperimmune-free is 0. See Homework #11, Problem 5.

9.3 Randomness relative to an oracle

Recall that X is random relative to f (i.e., f-random) if X ∉ ⋂_{n=0}^{∞} V_{n}^{f}, where V_{n}^{f} is any uniformly Σ_{1}^{0,f} sequence of sets with μ(V_{n}^{f}) ≤ 1/2^{n}.

Lemma 9.3.1. Assume that A ⊕ B is random. Then A is B-random and B is A-random. In particular, A and B are random, and A ≰_{T} B and B ≰_{T} A.

This lemma strengthens the result of Homework #9 Problem 4, which said that if X is random then X_{0} ≰_{T} X_{1} and X_{1} ≰_{T} X_{0}. Here X_{0} = the even part of X and X_{1} = the odd part of X, defined by X = X_{0} ⊕ X_{1}.

Corollary 9.3.2. There is an infinite descending sequence of Turing degrees.

Proof. Let X be random and consider X >_{T} X_{0} >_{T} X_{00} >_{T} X_{000} >_{T} · · ·.

Proof of Lemma. Suppose B is not A-random. Then B ∈ ⋂_{n=0}^{∞} V_{n}^{A} where V_{n}^{A} is uniformly Σ_{1}^{0,A} and μ(V_{n}^{A}) ≤ 1/2^{n}. Letting Φ(X, Y, n) be a partial recursive functional such that V_{n}^{A} = {Y | Φ(A, Y, n) ↓}, define V_{n}^{X} = {Y | Φ(X, Y, n)} and W_{n} = {X ⊕ Y | Y ∈ V_{n}^{X}[1/2^{n}]}. Here we are using the isomorphism 2^{ℕ} ≅ 2^{ℕ} × 2^{ℕ} given by X ⊕ Y ↦ (X, Y). Note that W_{n} is uniformly Σ_{1}^{0}, and by Fubini’s Theorem μ(W_{n}) ≤ 1/2^{n}. Also A ⊕ B ∈ W_{n} because B ∈ V_{n}^{A} = V_{n}^{A}[1/2^{n}] for all n. This contradicts the assumption that A ⊕ B is random.

Theorem 9.3.3 (Van Lambalgen’s Theorem). The following are pairwise equivalent:

1. A ⊕ B is random.

2. A is random and B is A-random.

3. B is random and A is B-random.

Proof. The previous lemma gives 1 ⇒ 2 and 1 ⇒ 3. We will prove 2 ⇒ 1, and the proof of 3 ⇒ 1 is similar. Assume 2 ∧ ¬ 1, i.e., A is random, B is A-random, and A⊕B is not random. Since A⊕B is not random, we have A⊕B ∈ ⋂_{n=0}^{∞} W_{n} where W_{n} is uniformly Σ_{1}^{0} and μ(W_{n}) ≤ 1/2^{n}. By passing to a subsequence, we may assume that μ(W_{n}) ≤ 1/2^{2n}. Let V_{n}^{X} = {Y | X ⊕ Y ∈ W_{n}} and U_{n} = {X | μ(V_{n}^{X}) > 1/2^{n}}. Note that U_{n} is uniformly Σ_{1}^{0}, because

X ∈ U_{n} ≡ μ(V_{n}^{X}) > 1/2^{n}
≡ ∃s \underbrace{μ(V_{n,s}^{X}) > 1/2^{n}}_{R(X,n,s) recursive}

Moreover, μ(U_{n}) ≤ 1/2^{n} because otherwise by Fubini’s Theorem we would have

μ(W_{n}) ≥ μ(U_{n}) · \frac{1}{2^{n}} > \frac{1}{2^{n}} · \frac{1}{2^{n}} = \frac{1}{2^{2n}}

a contradiction. Since A is random, Solovay’s Lemma tells us that A ∉ U_{n} for all but finitely many n. In other words, μ(V_{n}^{A}) ≤ 1/2^{n} for all but finitely many n. But V_{n}^{A} is uniformly Σ_{1}^{0,A}, and B ∈ V_{n}^{A} for all n (since A ⊕ B ∈ W_{n}). Thus B is not A-random. This completes the proof.

Theorem 9.3.4 (Miller/Yu 2004). Assume A is random and A ≤_{T} B where B is C-random. Then A is C-random.

Proof. We omit the proof.

9.4 Comments on Homework #11

In Homework #11, Problem 1 is to prove a generalization of the Magic Lemma 7.5.3, which states that the class of Π_{1}^{0} predicates is closed under ∃X. Recall that the proof of the Magic Lemma used the fact that 2^{ℕ} is compact. Note that 2^{ℕ} is a product space,

2^{ℕ} = {0, 1}^{ℕ} = ∏_{n=0}^{∞}{0, 1} = {0, 1} × {0, 1} × {0, 1} × · · · .

There is a theorem of general topology known as Tychonoff’s Theorem, which says that the product of any family of compact spaces is compact. Since {0, 1} is compact, we could have used Tychonoff’s Theorem to prove that 2^{ℕ} is compact. Similarly, for any fixed g ∈ ℕ^{ℕ} the product space

P_{g} = ∏_{n=0}^{∞}{0, 1, 2, …, g(n) − 1} = {f ∈ ℕ^{ℕ} | f is majorized by g}

is compact, by Tychonoff’s Theorem. Another way to see this is to note that the full g-tree

T_{g} = {σ ∈ ℕ^{<ℕ} | (∀n < |σ|) (σ(n) < g(n))}

is a finitely branching tree, so König’s Lemma applies. This remark will be useful in solving Problem 1.

Lemmas 7.5.1, 7.5.2, 7.5.3 are useful in Homework #11. Also useful is the fact that, in ℕ^{ℕ}, Π_{2}^{0} sets are Turing isomorphic to Π_{1}^{0} sets. This was proved in a previous homework.

Lecture 34: November 12, 2007

Reminder: Professor Cholak, an expert on Kolmogorov complexity and randomness, will give two talks tomorrow, Tuesday November 13.

• MASS Seminar, 10:10–12:05.

• Logic Seminar, 2:30–3:45.

Problem 2(b)

Recall

0^{(n)} = 0\underbrace{′′ ⋯ ′}_{n} = the nth Turing jump of 0.

By Post’s Theorem, the set 0^{(n)} ⊆ ℕ is Σ_{n}^{0} complete. Define

0^{(ω)} = ⨁_{n=0}^{∞} 0^{(n)} = {3^{m}5^{n} | m ∈ 0^{(n)}}.

Thus 0^{(ω)} is not Σ_{n}^{0} for any n, i.e., it is not in the arithmetical hierarchy. We have seen in an earlier homework problem that the singleton set {0^{(ω)}} is Π_{2}^{0}. Consider the predicates

P (X, n) ≡ X = 0^{(ω)} ∧ X(n) = 1

and

Q(n) ≡ ∃X P (X, n) ≡ n ∈ 0^{(ω)} .

Note that while P is Π_{2}^{0}, Q is not arithmetical.

Problem 3

Let P ⊆ 2^{ℕ} be Π_{1}^{0}. Let Φ(X, n) be a partial recursive functional such that Φ(X, n) ↓ for all X ∈ P and all n. We are asked to find a total recursive function g(n) exceeding Φ(X, n) for all X ∈ P and all n.

First we show that for each n, the values of Φ(X, n) for X ∈ P are bounded. We have

∀n (∀X ∈ P) ∃i (Φ(X, n) ≃ i)

hence

∀n ∀X ∃i (\underbrace{X ∉ P}_{Σ_{1}^{0}} ∨ \underbrace{Φ(X, n) ≃ i}_{Σ_{1}^{0}})

hence by Lemma 7.5.2 (the bounding principle)

∀n ∃j ∀X (∃i < j)(X ∉ P ∨ Φ(X, n) ≃ i)

hence

∀n ∃j (∀X ∈ P) (∃i < j) (Φ(X, n) ≃ i)

so the values are bounded. Now consider the predicate Q(n, j) saying that j is an appropriate bound, i.e.,

Q(n, j) ≡ (∀X ∈ P) (∃i < j) (Φ(X, n) ≃ i)
≡ ∀X (∃i < j) (X ∉ P ∨ Φ(X, n) ≃ i)
≡ \underbrace{∀X (\underbrace{X ∉ P ∨ Φ(X, n) ↓ < j}_{Σ_{1}^{0}})}_{Σ_{1}^{0}}

by Lemma 7.5.3. Thus the predicate Q(n, j) is Σ_{1}^{0}. Since ∀n ∃j Q(n, j) holds, we can find a recursive function g(n) such that ∀n Q(n, g(n)). This is the g as desired.

9.5 Homework #12, due November 26, 2007

Exercises 9.5.1.

1. (a) Let P be a Π_{1}^{0} subset of 2^{ℕ}. If P has only finitely many elements, prove that all of the elements of P are recursive. Hint: Use Lemma 7.5.3, a.k.a., the Magic Lemma.

(b) Does this hold with ℕ^{ℕ} instead of 2^{ℕ}?

2. (a) Let P ⊆ 2^{ℕ} be nonempty Π_{1}^{0} with no recursive elements. Prove that for all Y we can find X ∈ P such that X′ ≡_{T} X ⊕ 0′ ≡_{T} Y ⊕ 0′. Note: This result is a combination of the Low Basis Theorem and the Friedberg Jump Inversion Theorem. The proof is basically a combination of the two proofs.

(b) Deduce that for all Y we can find a random X such that X′ ≡_{T} X ⊕ 0′ ≡_{T} Y ⊕ 0′.

3. Recall that we have defined

K(n) = K(⟨\underbrace{1, …, 1}_{n}⟩)

for all n ∈ ℕ.

Assume that f(n) is a recursive function such that

∑_{n=0}^{∞} \frac{1}{2^{f(n)}} < ∞ .

Prove that K(n) ≤ f(n) + O(1) for all n.

4. Prove that

K(τ) ≤ C(τ) + K(C(τ)) + O(1)

for all bitstrings τ.

9.6 The Kučera/Gács Theorem

Continuing the line already pursued in Sections 9.1 and 9.2, we now present another theorem about deg_{T} (X) where X is random in the sense of Martin-Löf.

Theorem 9.6.1 (Kuc̆era 1985). For all Y ≥_{T} 0′ we can find X such that X ≡_{T} Y and X is random.

Corollary 9.6.2 (Gács). ∀Y ∃X (X random and Y ≤_{T} X). Corollary 9.6.3. ∃X (X random and 0′ ≤_{T} X).

To prove the theorem, we first prove some lemmas. Lemma 9.6.4. Let P ⊆ 2^{ℕ} be a measurable set. Let σ be a bitstring such that μ(P ∩N_{σ}) ≥ 1/2^{k} where k ≥ 1. Then we can find at least two distinct bitstrings τ of length 2k extending σ such that μ(P ∩ N_{τ}) ≥ 1/2^{4k}. Proof. Note first that

\frac{1}{2^{|σ|}} = μ(N_{σ}) ≥ μ(P ∩ N_{σ}) ≥ \frac{1}{2^{k}}

hence |σ| ≤ k < 2k since k ≥ 1. It follows that

P ∩ N_{σ} = ⋃_{τ⊃σ,|τ|=2k} P ∩ N_{τ} (disjoint union)

hence

μ(P ∩ N_{σ}) = ∑_{τ⊃σ,|τ|=2k} μ(P ∩ N_{τ}) .

Suppose for a contradiction that there is at most one τ as required. Then

∑_{τ⊃σ,|τ|=2k} μ(P ∩ N_{τ}) ≤ \frac{1}{2^{2k}} + (2^{2k−|σ|} − 1)\frac{1}{2^{4k}} < \frac{1}{2^{2k}} + 2^{2k} \frac{1}{2^{4k}} = \frac{2}{2^{2k}} ≤ \frac{1}{2^{k}}

so μ(P ∩ N_{σ}) < 1/2^{k} a contradiction.

Lecture 35: November 14, 2007

The following lemma has been implicit in previous results and homework, but we pause to make it explicit. Lemma 9.6.5. Let P ⊆ 2^{ℕ} be Π_{1}^{0}. Then, the 2-place number-theoretic predicate

Q(τ, k) ≡ μ(P ∩ N_{τ}) ≥ \frac{1}{2^{k}}

is Π_{1}^{0}. Proof. Let T ⊆ 2^{<ℕ} be a recursive tree such that P = {paths through T }. Then

μ(P ∩ N_{τ}) ≥ \frac{1}{2^{k}} ≡ (∀n ≥ |τ|) (\frac{|\{σ ∈ T | σ ⊇ τ, |σ| = n\}|}{2^{n}} ≥ \frac{1}{2^{k}} )

and this is clearly Π_{1}^{0}. Lemma 9.6.6. Let P ⊆ 2^{ℕ} be Π_{1}^{0} of positive measure. Then for all Y we can find X ∈ P such that Y ≤_{T} X. Moreover X ≤_{T} Y ⊕ 0′.

Proof. The idea of the proof is to apply Lemma 9.6.4 repeatedly to construct a mapping f : 2^{<ℕ} → 2^{<ℕ}. Namely, for each ρ ∈ 2^{<ℕ}, f(ρ^{⌢}⟨0⟩) and f(ρ^{⌢}⟨1⟩) will be incompatible bitstrings extending f(ρ) obtained from Lemma 9.6.4. For technical reasons, we choose f(ρ^{⌢}⟨0⟩) and f(ρ^{⌢}⟨1⟩) to be the leftmost and rightmost such strings. Here “leftmost” and “rightmost” are with respect to some fixed recursive linear ordering of all bitstrings.

Here are the details of the construction of f. Fix k ≥ 1 such that μ(P) ≥ 1/2^{k}. Define f(ρ) by induction on |ρ|. Begin with f(⟨⟩) = ⟨⟩. Note that N_{⟨⟩} = 2^{ℕ} hence μ(P ∩ N_{⟨⟩}) = μ(P) ≥ 1/2^{k}. Assume inductively that f(ρ) has already been defined and μ(P ∩ N_{f(ρ)}) ≥ 1/2^{4^{n}k} where n = |ρ|. By Lemma 9.6.4 there are at least two bitstrings τ extending f(ρ) of length 2 · 4^{n}k such that μ(P ∩ N_{τ}) ≥ 1/2^{4^{n+1}k}. Let f(ρ^{⌢}⟨0⟩) and f(ρ^{⌢}⟨1⟩) be the leftmost and rightmost such τ. Note that f(ρ^{⌢}⟨0⟩) and f(ρ^{⌢}⟨1⟩) are distinct bitstrings of length 2 · 4^{n}k.

By Lemma 9.6.5 we have f ≤_{T} 0′. Given Y ∈ 2^{ℕ}, let X = ⋃_{n=0}^{∞} f(Y ↾ n). Clearly X ≤_{T} Y ⊕ f ≤_{T} Y ⊕ 0′ and Y ≤_{T} X ⊕ f ≤_{T} X ⊕ 0′. Moreover, X ∈ N_{f(Y ↾n)} and P ∩ N_{f(Y ↾n)} ≠ ∅ for all n. Since P is a closed set, it follows that X ∈ P.

It remains to show that Y ≤_{T} X. Using X as an oracle, we describe how to compute Y. Suppose we have already computed Y ↾ n. We need to decide whether Y (n) = 0 or Y (n) = 1. We know what f(Y ↾ n + 1) is, namely it is X ↾ 2 · 4^{n}k. We also know what f(Y ↾ n) is, namely it is X ↾ 2 · 4^{n−1}k if n > 0, or ⟨⟩ if n = 0. Moreover, we know that Y (n) = 0 (respectively Y (n) = 1) if and only if μ(P ∩ N_{τ}) < 1/2^{4^{n+1}k} for all τ extending f(Y ↾ n) of length 2 · 4^{n}k lying to the left (respectively right) of X ↾ 2 · 4^{n}k. By Lemma 9.6.5 these predicates are Σ_{1}^{0}. Therefore, since one of these predicates holds, we can wait until we find out which one holds, and at that point we know whether Y (n) = 0 or Y (n) = 1. This completes the proof.

Lecture 36: November 15, 2007

Proof of Theorem 9.6.1. Let P be a Π_{1}^{0} set containing only random X. For any Turing degree a ≥ 0′, let Y be such that a = deg_{T} (Y). Then by the above lemma, we can find X ∈ P such that Y ≤_{T} X and X ≤ Y ⊕0′. But Y ⊕0′ ≡_{T} Y, so X ≡_{T} Y.

In addition Jonas Kibelbek presented solutions of Problems 7 and 8 in Homework #11. See Section 9.8 below.

Lecture 37: November 23, 2007

Jonas presented solutions of some problems in Homework #10. See Section 9.7 below.

Lecture 38: November 26, 2007

We have proved the following two theorems. Theorem 9.6.7 (van Lambalgen’s Theorem). The following are pairwise equivalent:

1. A ⊕ B is random

2. A is random, and B is random relative to A.

3. B is random, and A is random relative to B. Theorem 9.6.8 (Kučera/Gács Theorem). For all Y ≥_{T} 0′ we can find a random X ≡_{T} Y.

We now combine these two theorems to deduce the following corollaries. Corollary 9.6.9. Suppose A is random, A ≤_{T} B, B is random relative to C, and C ≥_{T} 0′. Then A is random relative to C. Proof. Since C ≥_{T} 0′, we can assume by Kuc̆era/Gács that C is random. Since B is random relative to C, it follows by van Lambalgen that B ⊕ C is random. Hence, by van Lambalgen again, C is random relative to B. Hence, since A ≤_{T} B, C is random relative to A. We are also assuming that A is random, so by van Lambalgen we get that A ⊕ C is random. Applying van Lambalgen again, we see that A is random relative to C. Remark 9.6.10. The previous corollary actually holds without the assumption C ≥_{T} 0′. This result is due to Miller/Yu 2004. Definition 9.6.11. We say that A is n-random if A is random relative to 0^{(n−1)}. We say that A is arithmetically random if A is n-random for all n.

For example,

1-random ≡ random,
2-random ≡ random relative to 0′,
3-random ≡ random relative to 0′′,

etc.

Thus we have a hierarchy which we can use to measure higher and higher amounts of randomness. Corollary 9.6.12. Assume that A is random and A ≤_{T} B.

1. If B is n-random, then so is A.

2. If B is arithmetically random, then so is A.

Proof. In the previous corollary, let C = 0^{(n−1)}.

Remark 9.6.13. The above corollaries can be paraphrased as follows:

If A is random, and if A ≤_{T} B for some B which is “highly random,”

then A itself is “highly random.”

Thus we see that Martin-Löf randomness is in a sense a threshhold amount of randomness. Beyond this threshhold, higher amounts of randomness behave nicely in that they propagate downward via Turing reducibility. This phenomenon may be viewed as further evidence for our belief that Martin-Löf’s concept of randomness is a very natural concept.

9.7 Some solutions for Homework #10

Problem 4

Let X ∈ 2^{ℕ}. We say that X is 2-random if X is random relative to 0′. Recall also that X is weakly 2-random if X ∉ any Π_{2}^{0} set of measure 0. Let a = deg_{T} (X) = the Turing degree of X.

Part (a): Show that if X is 2-random then X is weakly 2-random.

Solution. We need to show

(X ∉ any Martin-Löf test relative to 0′) ⇒ (X ∉ any Π_{2}^{0} null set)

We will do this by showing that every Π_{2}^{0} null set P corresponds to a Martin-Löf test relative to 0′.

Since P is Π_{2}^{0}, we have X ∈ P ≡ ∀m ∃n R(X, m, n) where R is recursive. If

we consider the Σ_{1}^{0} sets W_{m} = {X | ∃n R(X, m, n)}, we see that P = ⋂_{m=0}^{∞} W_{m}.

That is, a Π_{2}^{0} set is the intersection of a uniform sequence of Σ_{1}^{0} sets. We may assume that W_{0} ⊇ W_{1} ⊇ ⋯ ⊇ W_{m} ⊇ ⋯, since we could replace each W_{m} with

W̃_{m} = ⋂_{i=0}^{m} W_{i}, which is again a uniform sequence of Σ_{1}^{0} sets.

The difference between P and a Martin-Löf test is that we do not have a nice bound on how quickly μ(W_{m}) goes to 0. (Recall that a Martin-Löf test is T = ⋂_{n=0}^{∞} V_{n} where the V_{n} are uniformly Σ_{1}^{0} and μ(V_{n}) ≤ \frac{1}{2^{n}}.) We can use the Halting Problem to estimate the measures μ(W_{m}) and take a subsequence f(n) so that μ(W_{f(n)}) ≤ 1/2^{n}.

Recall that we have a standard way to index Σ_{1}^{0} sets; U_{e} = {X | φ_{e}^{(1),X}(0) ↓}. By the Parametrization Theorem, we can find a total function h(m) such that W_{m} = U_{h(m)}. It is helpful to define the sets W_{m,s} = U_{h(m),s} = {X | φ_{e,s}^{(1),X↾s}(0) ↓ }, which are “finite approximations of W_{m}.” For all s, W_{m,s} is recursive, and

W_{m,0} ⊆ W_{m,1} ⊆ ⋯ ⊆ W_{m,s} ⊆ ⋯ with ⋃_{s=0}^{∞} W_{m,s} = W_{m}.

Define the partial recursive function ψ(n, m) = the least s such that μ(W_{m,s}) > 1/2^{n}. Then ψ(n, m) halts if μ(W_{m}) > 1/2^{n} and it fails to halt if μ(W_{m}) ≤ 1/2^{n}. So, using the Halting Problem, we can compute an m sufficiently large that μ(W_{m}) ≤ 1/2^{n}.

Define f(n) = the least m such that ψ(n, m) fails to halt. Then W_{f(n)} is a uniform Σ_{1}^{0,0′} sequence with μ(W_{f(n)}) ≤ 1/2^{n}. Thus, P = ⋂_{m=0}^{∞} W_{m} = ⋂_{n=0}^{∞} W_{f(n)} is a Martin-Löf test relative to 0′; and so every 2-random X is also weakly 2-random.

Part (b): Show that if X is weakly 2-random then inf(a, 0′) = 0.

Solution. Suppose that inf(a, 0′) ≠ 0. Let Y be such that 0 <_{T} Y ≤_{T} 0′ and Y ≤_{T} X. Since Y ≤_{T} 0′, by Post’s Theorem, Y is Δ_{2}^{0}. Since Y ≤_{T} X, there is some e such that Y = φ_{e}^{(1),X}.

We will show that the set P = {X | Y = φ_{e}^{(1),X}} is Π_{2}^{0} of measure 0, so that X cannot be weakly 2-random. Since Y is nonrecursive, we know that P has measure 0. (See problem 7 of Homework #11.) We check that P is Π_{2}^{0}:

X ∈ P ≡ \underbrace{∀n \underbrace{(\underbrace{φ_{e}^{(1),X}(n) ↓}_{Σ_{1}^{0}} ∧ \underbrace{φ_{e}^{(1),X}(n) = Y (n)}_{Δ_{2}^{0}})}_{Δ_{2}^{0}}}_{Π_{2}^{0}}

Part (c): In Part (b), what if we assume only that X is random?

Solution. By the Low Basis Theorem, we can show that there exists X that are random and low. That is, X′ ≡_{T} 0′, which implies X <_{T} 0′. Then, letting a = deg_{T} (X), inf(a, 0′) = a ≠ 0. Thus, the result of part (b) does not hold if we only assume that X is random.

Part (d): If X is 2-random then X′ ≡_{T} X ⊕ 0′.

Solution. Recall that X′ = H^{X} = {e | φ_{e}^{(1),X}(0) ↓} = the Halting Problem relative to X.

Let U_{e} = {X | φ_{e}^{(1),X}(0) ↓}
= the eth Σ_{1}^{0} subset of 2^{ℕ} .
Let U_{e,s} = {X | φ_{e,s}^{(1),X↾s}(0) ↓}
= ⋃_{σ∈2^{ℕ},|σ|=s} N_{σ} where φ_{e,s}^{(1),σ}(0) ↓ .

Note that

1. U_{e,s} is a finite union of neighborhoods.

2. μ(U_{e,s}) is a recursive function of e, s.

3. U_{e} = ⋃_{s=0}^{∞} U_{e,s}, hence μ(U_{e}) = lim_{s→∞} μ(U_{e,s}).

Let f(e) = the least s such that μ(U_{e} \ U_{e,s}) ≤ 1/2^{e}. We claim that f ≤_{T} 0′. This is because f(e) = the least s such that (∀t ≥ s) (μ(U_{e,t} \ U_{e,s}) ≤ 1/2^{e}) which is a Π_{1}^{0} condition.

Let V_{e} = U_{e} \ U_{e,f(e)}. Note that μ(V_{e}) ≤ 1/2^{e}. Moreover V_{e} is Σ_{1}^{0,f}, hence Σ_{1}^{0,0′}. So the sets V_{e}, e = 0, 1, 2, … form a test for randomness relative to 0′.

By Solovay’s Lemma relative to 0′, since X is random relative to 0′, X ∉ V_{e} for all but finitely many e.

Therefore, for all sufficiently large e, e ∈ H^{X} ≡ X ∈ U_{e} ≡ X ∈ U_{e,f(e)}. It follows that H^{X} ≤_{T} X ⊕ f ≤_{T} X ⊕ 0′, which is what we need to show that X′ ≡_{T} X ⊕ 0′.

9.8 Some solutions for Homework #11

Problem 3

Let P ⊆ 2^{ℕ} be Π_{1}^{0}. Let Φ(X, n) be a partial recursive functional such that Φ(X, n) ↓ for all X ∈ P and all n. We are to find a recursive upper bound on these values, g(n), where g is a total recursive function.

One solution was given in class earlier, using several lemmas. A more handson approach is to note that for each X ∈ P and each n there is a finite amount of information from X that causes Φ(X, n) to halt. This finite amount of information is essentially a neighborhood, and P is covered by these neighborhoods. Since P is a Π_{1}^{0} subset of 2^{ℕ}, it is closed and hence compact, so we can find a finite subcovering. In fact, we can find such a finite subcovering recursively. Since we now have only finitely many neighborhoods to consider, we can take g(n) to be one more than the maximum value of Φ(X, n) on these neighborhoods.

Problem 4

Given P ⊆ 2^{ℕ} Π_{1}^{0} nonempty. To find a hyperimmune-free X ∈ P, we construct a descending sequence of nonempty Π_{1}^{0} sets

P = Q_{0} ⊇ Q_{1} ⊇ ⋯ ⊇ Q_{e} ⊇ Q_{e+1} ⊇ ⋯

and at the end of the construction we let X ∈ ⋂_{e=0}^{∞} Q_{e}.

Stage 0. Let Q_{0} = P.

Stage e + 1. Given Q_{e}, there are two cases.

Case 1: ∃n (∃X ∈ Q_{e}) (φ_{e}^{(1),X}(n) ↑). In this case, fix such an n and let Q_{e+1} = {X ∈ Q_{e} | φ_{e}^{(1),X}(n) ↑}. Note that Q_{e+1} is Π_{1}^{0} because we have chosen a particular n and defined

X ∈ Q_{e+1} ≡ X ∈ Q_{e} ∧ φ_{e}^{(1),X}(n) ↑ .

Moreover Q_{e+1} is nonempty by the choice of n. We have now satisfied the eth requirement for hyperimmune-freeness, because the partial X-recursive function φ_{e}^{(1),X}(n) is not total for any X ∈ Q_{e+1}.

Case 2: Not case 1. In this case let Q_{e+1} = Q_{e}. Then, the partial recursive functional Φ(X, n) ≃ φ_{e}^{(1),X}(n) is ↓ for all X ∈ Q_{e+1} and all n. Hence by Problem 3 we can find a total recursive function g_{e} such that φ_{e}^{(1),X}(n) ↓ < g_{e}(n) for all X ∈ Q_{e+1} and all n. Thus, in this case also, the eth requirement for hyperimmune-freeness has been satisfied.

Problem 7

Part (a): Prove that if Y is nonrecursive then μ({X | Y ≰_{T} X}) = 1.

Solution. This is equivalent to saying that μ({X | Y ≤_{T} X}) = 0. We can simplify by defining the sets C_{e} = {X | Y = φ_{e}^{(1),X}}. Then our problem is equivalent to showing that μ(C_{e}) = 0 for all e. (Since the union of countably many measure 0 sets has measure 0.)

Suppose for some e, C_{e} has positive measure. It would be nice if we could conclude that there is some neighborhood N_{σ} ⊆ C_{e}, so that Y can be computed, using the program given by e, using as an oracle any X ⊃ σ. This would make Y recursive, since it could be computed using just the finite amount of information in σ, which would give us the needed contradiction.

However, it is not true that if C_{e} has positive measure, it must contain some neighborhood. (For example, the set of random elements has measure 1, but contains no neighborhoods.) We can however approximate C_{e} closely by neighborhoods. By regularity of the measure μ, there is an open set U containing C_{e} with μ(U) < 4μ(C_{e})/3. Since U is an open set, it is a union of neighborhoods, U = ⋃_{σ∈S} N_{σ}. We can take a finite subset F of S so that V = ⋃_{σ∈F} N_{σ} has measure μ(V) > 3μ(U)/4. This gives us a very simple open set V that is a good approximation of C_{e}. The set V is a finite union of neighborhoods, and at least 2/3 of V by measure is in C_{e}. (The worst case is that all of U \ C_{e} is in V, but μ(U \ C_{e}) < μ(U)/4 < μ(V)/3.)

This means that, by measure, most of the X ∈ V are such that φ_{e}^{(1),X} = Y. So, to compute Y (n), find the least k such that for strings σ of length k with N_{σ} ⊆ V, the computation φ_{e}^{(1),σ}(n) halts with the same value for enough σ’s to account for at least 1/3 of the measure of V. Then the value at which they halt is Y (n). Thus, Y is computable, which contradicts our initial assumption.

Hence, C_{e} = {X ∈ 2^{ℕ} | Y = φ_{e}^{(1),X}} has measure 0 for all e.

Part (b): Deduce that for each nonrecursive Y we can find a random X such that Y ≰_{T} X.

Solution. Note that the set R = {X | X is random} has measure 1. So, for any nonrecursive Y, R ∩ {X ∈ 2^{ℕ} | Y ≰_{T} X} is the intersection of two measure 1 sets, and thus has measure 1 and is nonempty.

Part (c): More generally, given a sequence of nonrecursive oracles Y_{i}, i = 0, 1, 2, …, find an X which is n-random for all n such that Y_{i} ≰_{T} X for all i.

Solution. For each n, the set R_{n} = {X | X is n-random} has measure 1. (This is because the Martin-Löf tests relative to 0^{(n−1)} can be enumerated as T_{ni}, i = 0, 1, 2, …, i.e., there are only countably many of them. Then R_{n} = 2^{ℕ}\⋃_{i=0}^{∞} T_{n,i} where ⋃_{i=0}^{∞} T_{ni} has measure 0 because each T_{n,i} has measure 0.)

Thus ⋂_{n=0}^{∞} R_{n} has measure 1. Similarly, ⋂_{i=0}^{∞}{X ∈ 2^{ℕ} | Y_{i} ≰_{T} X} has measure 1. Their intersection still has measure 1 and so is nonempty.

Problem 8

Part (a): Assume that P ⊆ 2^{ℕ} is Π_{1}^{0} and ¬ ∃X (X ∈ P ∧ X is recursive). Find a nonrecursive Y such that ∄X (X ∈ P ∧ X ≤_{T} Y).

Solution. We will construct such a Y by finite approximation. We need to guarantee that Y is nonrecursive. (This step is straightforward; we have done it several times before.) The new step is to guarantee that Y cannot be used to compute any X ∈ P. That is, for each e, we need to guarantee that φ_{e}^{(1),Y} is not in P.

Stage 0. Let σ_{0} = ⟨⟩.

Stage 2e+1.

Case 1: If φ_{e}(|σ_{2e}|) = 0, let σ_{2e+1} = σ_{2e}^{⌢}⟨1⟩.

Case 2: If φ_{e}(|σ_{2e}|) ≠ 0, let σ_{2e+1} = σ_{2e}^{⌢}⟨0⟩.

The odd numbered steps guarantee that Y = ⋃_{n=0}^{∞} σ_{n} is nonrecursive.

Stage 2e+2. Find a σ ⊃ σ_{2e+1} such that there is no Y ⊃ σ with φ_{e}^{(1),Y} in P. Let σ_{2e+2} = σ.

Claim: There is no need for a case 2, since such a σ always exists.

It is clear that if the claim is true, then Y = ⋃_{n=0}^{∞} σ_{n} will not compute any X in P, and the problem will be complete.

Suppose there is no such σ. Then for all τ ⊇ σ_{2e+1}, there is some Y ⊃ τ with φ_{e}^{(1),Y} in P. We will show that some X in P is recursive.

To compute X, first find the least τ_{0} ⊇ σ_{2e+1} such that φ_{e}^{(1),τ_{0}}(0) halts; define X(0) = φ_{e}^{(1),τ_{0}}(0).

Given τ_{n}, let τ_{n+1} be the least extension of τ_{n} such that φ_{e}^{(1),τ_{n}}(n) halts; define X(n + 1) = φ_{e}^{(1),τ_{n+1}}(n + 1).

Note that we can always find such extensions, by the assumption that every string τ extending σ_{2e+1} is an initial segment for some Y where φ_{e}^{(1),Y} is total and in P. Clearly, X is recursive. Since P is Π_{1}^{0}, P is the set of infinite paths through a recursive tree T. Since each τ_{n} is an initial segment for some Y with φ_{e}^{(1),Y} in P, φ_{e}^{(1),τ_{n}} ↾ n = φ_{e}^{(1),Y} ↾ n is in the tree T. That is, X ↾ n = φ_{e}^{(1),τ_{n}} ↾ n is in T for all n; thus X ∈ P.

This gives us our contradiction, since we assume that P contains no recursive elements.

Part (b): Find a nonrecursive Y such that ¬ ∃X (X is random ∧ X ≤_{T} Y).

Solution. Recall that the set of random elements R = 2^{ℕ} \ ⋂_{n=0}^{∞} V_{n} where ⋂_{n=0}^{∞} V_{n} is a universal Martin-Löf test and each V_{n} is Σ_{1}^{0}. Then R = ⋃_{n=0}^{∞}(2^{ℕ} \ V_{n}) is the union of countably many Π_{1}^{0} sets.

So, to construct a Y that does not compute any random X, we modify the above finite approximation construction so that we take care of all the sets P_{n} = 2^{ℕ} \ V_{n}. (Note that these are Π_{1}^{0} sets without any recursive elements, as required.)

Stage 0: Let σ_{0} = ⟨⟩.

Stage 2k+1:

Case 1: If φ_{k}(|σ_{2k}|) = 0, let σ_{2k+1} = σ_{2k}^{⌢}⟨1⟩.

Case 2: If φ_{k}(|σ_{2k}|) ≠ 0, let σ_{2k+1} = σ_{2k}^{⌢}⟨0⟩.

The odd numbered steps guarantee that Y = ⋃_{n=0}^{∞} σ_{n} is nonrecursive.

Stage 2k+2: Let n = (k)_{0} and e = (k)_{1}. Find a σ ⊃ σ_{2k+1} such that there is no Y ⊃ σ with φ_{e}^{(1),Y} in P_{n}. Let σ_{2k+2} = σ.

Let Y = ⋃_{n=0}^{∞} σ_{n}. Then Y is nonrecursive and no X ∈ P_{n} for any n is Turing reducible to Y. Since each random X ∈ some P_{n}, no random X is Turing reducible to Y.

9.9 Comments on Homework #12

Problem 4

The problem should have been to prove K(τ) ≤ C(τ) + K(C(τ)) + O(1).

Solution. Define M by

M(ρ^{⌢}σ) ≃ τ ≡ U(ρ) = ⟨\underbrace{1, …, 1}_{|σ|}⟩ ∧ U^{∗}(σ) ≃ τ

where U is a universal prefix-free machine and U^{∗} is a universal machine. It is easy to verify that M is a prefix-free machine. Given τ, let σ be such that U^{∗}(σ) ≃ τ and |σ| = C(τ). Let ρ be such that U(ρ) = ⟨\underbrace{1, …, 1}_{|σ|}⟩ and |ρ| =

K(|σ|). Then M(ρ^{⌢}σ) ≃ τ, hence K(τ) ≤ |ρ| + |σ| + O(1) = C(τ) + K(C(τ)) + O(1).

9.10 Homework #13, due December 3, 2007

Exercises 9.10.1. Recall that A is said to be strongly random if A does not belong to any Π_{2}^{0} set of measure 0.

1. (a) Suppose A ⊕ B is strongly random. Prove that inf(a, b) = 0 where a = deg_{T} (A) and b = deg_{T} (B).

(b) What if we assume only that A ⊕ B is random?

2. Prove the following. If A is random and A ≤_{T} B and B is strongly random, then A is strongly random.

3. Prove that the following conditions are equivalent.

(a) A is random relative to 0^{(n)} for all n.

(b) A does not belong to any arithmetical set of measure 0.

In this case we say that A is arithmetically random.

9.11 LR- and LK-reducibility

We shall end the course by discussing two additional methods of classifying Turing oracles. These classification methods are motivated by the ideas of relative randomness and relative prefix-free complexity, respectively.

Definition 9.11.1 (LR-reducibility). Write A ≤_{LR} B to mean that

∀X (if X is B-random then X is A-random).

Write A ≡_{LR} B to mean that A ≤_{LR} B and B ≤_{LR} A. Write A <_{LR} B to mean that A ≤_{LR} B and B ≰_{LR} A.

Remark 9.11.2. The idea of LR-reducibility is that we are classifying oracles according to their ability to reveal nonrandom patterns.

Remark 9.11.3. Clearly A ≤_{T} B implies A ≤_{LR} B. Moreover, LR-reducibility is similar to Turing reducibility in that it is a transitive, reflexive relation on Turing oracles. Likewise A ≡_{T} B implies A ≡_{LR} B, and ≡_{LR} is similar to ≡_{T} in that it is an equivalence relation.

Exercise 9.11.4. Show that A <_{LR} A′ for all A. Here A is any Turing oracle, and A′ is the Turing jump of A.

Solution. We know that A <_{T} A′, hence A ≤_{LR} A′. It remains to show that A′ ≰_{LR} A. There are several ways to see this. First, by Corollary 9.2.7 relativized to A, let X be A-random and low relative to A, i.e., (X ⊕ A)′ ≡_{T} A′. Since X is low relative to A, it follows in particular that X ≤_{T} A′, hence X is not A′-random. Thus X witnesses that A′ ≰_{LR} A. Alternatively, by the Kučera/Gács Theorem 9.6.1 relativized to A, let X be A-random such that X ⊕ A ≡_{T} A′. Again it follows that X ≤_{T} A′, hence X is not A′-random, so again X witnesses A′ ≰_{LR} A.

The following theorem says that, not suprisingly, LR-reducibility does not coincide with Turing reducibility.

Theorem 9.11.5 (Kuc̆era/Terwijn 2002). For all A we can find B such that A <_{T} B and A ≡_{LR} B.

We also have the following characterization of LR-reducibility. This characterization is interesting in that it does not mention randomness.

Theorem 9.11.6 (Kjos-Hanssen 2005). A ≤_{LR} B if and only if every Π_{1}^{0,A} set of positive measure includes a Π_{1}^{0,B} set of positive measure.

The second classification method which we want to consider is as follows.

Definition 9.11.7 (LK-reducibility). Write A ≤_{LK} B to mean that

K^{B}(τ) ≤ K^{A}(τ) + O(1)

for all bitstrings τ. Write A ≡_{LK} B to mean that A ≤_{LK} B and B ≤_{LK} A. Write A <_{LK} B to mean that A ≤_{LK} B and B ≰_{LK} A.

Remark 9.11.8. Just as for LR-reducibility, we have similar properties for LK-reducibility. Namely, A ≤_{T} B implies A ≤_{LK} B, and LK-reducibility is reflexive and transitive, and ≡_{LK} is an equivalence relation.

The idea of LK-reducibility is that we are classifying oracles according to their ability to compress bitstrings. From this point of view, the following theorem is remarkable. It says that LR-reducibility and LK-reducibility coincide.

Theorem 9.11.9 (Kjos-Hanssen/Miller/Solomon 2006). A ≤_{LR} B if and only if A ≤_{LK} B.

In the time remaining, we shall try to prove as many of these theorems as possible.

Lecture 39: November 28, 2007

Remark 9.11.10 (Final Examinations). December 10, 12, 14. Each student will have an individual 1-hour oral final exam. The exam will consist of a question from the list handed out in class, a problem to solve similar to easier homework questions, and a presentation of the research project. Please turn in the paper for your research project by 8 am on Monday, December 10 so that it can be reviewed before the final examination, when grades are assigned.

We now begin the proofs of some of the theorems on LR-reducibility.

Let U, V be Σ_{1}^{0} subsets of 2^{ℕ}, with U = ⋃_{σ∈S} N_{σ}, V = ⋃_{τ∈T} N_{τ} where S, T are prefix-free Σ_{1}^{0} subsets of 2^{<ℕ}.

Define a product operation UV = ⋃_{σ∈S} ⋃_{τ∈T} N_{σ^{⌢}τ}.

Properties:

1. UV is Σ_{1}^{0}.

2. Given indices of U, V (qua Σ_{1}^{0} sets), we can compute an index of UV (qua Σ_{1}^{0} set). Namely,

( \begin{matrix}U_{e} = \{X | φ_{e}^{(1),X}(0) ↓\} = ⋃_{σ∈S_{e}} N_{σ} \\ where S_{e} = \{σ | φ_{e,|σ|}^{(1),σ}(0) ↓ ∧(∀ρ ⊂ σ)φ_{e,|ρ|}^{(1),ρ}(0) ↑\} \end{matrix} )

3. UV ⊆ U (because N_{σ^{⌢}τ} ⊆ N_{σ}).

4. μ(UV) = μ(U)μ(V).

(because each N_{σ} is a copy of the entire Cantor space, N_{σ}V has measure μ(V)/2^{|σ|}, and UV = ⋃_{σ∈S} N_{σ}V.)

5. The product is associative. (UV)W = U(V W).

Define U^{n} = \underbrace{U ⋯ U}_{n}. Then μ(U^{n}) = μ(U)^{n}. If μ(U) < 1, then lim_{n→∞} μ(U^{n}) =

0. Let k be such that μ(U^{k}) ≤ 1/2, then μ(U^{nk}) = (μ(U^{k})^{n}) ≤ 1/2^{n}, hence U^{nk}, n = 0, 1, 2, … is a Martin-Löf test.

Lecture 40: November 29, 2007

Review:

Definition 9.11.11. Let U, V ⊆ 2^{ℕ} be open with U = ⋃_{σ∈S} N_{σ} and V = ⋃_{τ∈T} N_{τ} where S, T are prefix-free sets of bitstrings. Define UV = ⋃_{σ∈S,τ∈T} N_{σ^{⌢}τ}. This is again an open set.

Remark 9.11.12. This product operation UV is not really an operation on open sets. Rather, it is an operation on the prefix-free sets of bitstrings which define these open sets. To be absolutely correct we should write ST = {σ^{⌢}τ | σ ∈ S, τ ∈ T } and note that this is again a prefix-free set of bitstrings. However, we shall instead continue to abuse notation by writing UV as if it were an operation on open sets.

Remark 9.11.13. Our product operation UV has the following properties:

1. UV ⊆ U.

2. (UV)W = U(V W).

3. μ(UV) = μ(U)μ(V).

4. If U and V are Σ_{1}^{0}, then UV is Σ_{1}^{0}. Moreover, this holds uniformly.

Letting U^{n} = \underbrace{U ⋯ U}_{n} we see that μ(U^{n}) = μ(U)^{n}. Therefore, if μ(U) < 1

we have lim_{n→∞} μ(U^{n}) = lim_{n→∞} μ(U)^{n} = 0 geometrically. If in addition U is Σ_{1}^{0}, then the sequence of sets U^{n}, n = 0, 1, 2, … is uniformly Σ_{1}^{0}, so we have a Martin-Löf test. Thus, for any random X ∈ 2^{ℕ} we have X ∉ U^{n} for some n. Consider the least such n. Then X ∈ U^{n−1} and X ∉ U^{n} = U^{n−1}U. Thus X = σ_{1}^{⌢} ⋯^{⌢}σ_{n−1}^{⌢}Y for some σ_{1}, …, σ_{n−1} ∈ S and some Y ∉ U.

We have now essentially proved the following lemma:

Lemma 9.11.14 (Kuc̆era, 1985). Let P ⊆ 2^{ℕ} be Π_{1}^{0} of positive measure. Then for all random X ∈ 2^{ℕ} there exist σ and Y such that X = σ^{⌢}Y and Y ∈ P.

Proof. Let U = 2^{ℕ} \ P and reason as above. Let σ = σ_{1}^{⌢} ⋯^{⌢}σ_{n−1} and let Y be such that X = σ^{⌢}Y.

Remark 9.11.15. The above lemma is a refinement of the 0 − 1 Law in probability theory.

Corollary 9.11.16. Let P, Q be nonempty Π_{1}^{0} sets consisting of random elements. Then P, Q are Turing isomorphic.

Proof. For each X ∈ P we have X = σ^{⌢}Y for some bitstring σ and some Y ∈ Q, hence X ≡_{T} Y. And vice versa.

The following lemma is implicit in earlier work.

Lemma 9.11.17. Let P ⊆ 2^{ℕ} be Π_{1}^{0}. The following are pairwise equivalent:

1. μ(P) > 0

2. P includes a nonempty Π_{1}^{0} set consisting of random element.

3. P contains a random element.

Proof. (1 ⇒ 2) Recall that R = {X | X is random } is Σ_{2}^{0}, hence R = ⋃_{n=0}^{∞} P_{n} where P_{n} is Π_{1}^{0}. Also, μ(R) = 1. If μ(P) > 0, then μ(P ∩ P_{n}) > 0 for some n. Thus P ∩ P_{n} is a Π_{1}^{0} set which is included in P and consists entirely of random elements.

(2 ⇒ 3) is trivial.

(3 ⇒ 1) because random ⇒ weakly random.

Our goal now is to prove the following result giving several characterizations of LR-reducibility.

Theorem 9.11.18 (Kjos-Hanssen 2005). The following are pairwise equivalent.

1. A ≤_{LR} B. (This means that B-random implies A-random.)

2. Every Π_{1}^{0,A} set of positive measure includes a Π_{1}^{0,B} set of positive measure.

3. There exists a Π_{1}^{0,A} set consisting of A-random elements which includes a Π_{1}^{0,B} set of positive measure.

4. There exists a Π_{1}^{0,B} set of positive measure consisting of A-random elements.

Toward the proof of this theorem, note that 2 ⇒ 3 follows from the previous lemma relativized to A. Also, 3 ⇒ 4 is trivial. To prove 4 ⇒ 1, assume 4 and let Q be a Π_{1}^{0,B} set of positive measure consisting of A-random elements. If X is B-random, then by Lemma 9.11.14 relativized to B we have X = σ^{⌢}Y for some Y ∈ Q. It follows that Y is A-random, hence X is A-random, and this proves 1.

It remains to prove 1 ⇒ 2. In order to prove 1 ⇒ 2 we make the following definition.

Definition 9.11.19. Let U ⊆ 2^{ℕ} be open. We say that U is fat if U intersects every Π_{1}^{0} set of positive measure. We say that U is B-fat if U intersects every Π_{1}^{0,B} set of positive measure.

Note that 2 amounts to saying that every B-fat Σ_{1}^{0,A} set is of measure 1.

Lemma 9.11.20. If U and V are fat, then UV is fat.

Proof. Let U and V be fat. Let Q be a Π_{1}^{0} set of positive measure. By the previous lemma, we may assume every element of Q is random. Write U = ⋃_{σ∈S} N_{σ} where S is prefix-free. Then UV = ⋃_{σ∈S} σ^{⌢}V. Since U is fat, U ∩Q ≠ ∅, hence N_{σ} ∩Q ≠ ∅ for some σ ∈ S. Hence N_{σ} ∩Q is Π_{1}^{0} of positive measure (by previous lemma). Since V is fat, σ^{⌢}V is fat within N_{σ}. Hence Q ∩ (σ^{⌢}V) ≠ ∅, hence Q ∩ UV ≠ ∅, Q.E.D.

Note also that Lemma 9.11.20 relativizes as follows:

For all B, if U and V are B-fat then UV is B-fat.

Lecture 41: November 30, 2007

Review:

Lemma 9.11.21. Every Π_{1}^{0} set of positive measure includes a Π_{1}^{0} set of positive measure consisting of random elements.

Proof. This is because {X | X is random} is Σ_{2}^{0} of measure 1.

Lemma 9.11.22. Every Π_{1}^{0} set which contains a random element is of positive measure.

Proof. This is because randomness implies weak randomness.

Lemma 9.11.23 (Kuc̆era’s Lemma). If P is Π_{1}^{0} of positive measure then ∀X (X random ⇒ P contains a “tail” of X), i.e., X = σ^{⌢}Y where Y ∈ P.

We have also proved:

Lemma 9.11.24. If U and V are open and fat, then UV is open and fat. (Recall fat = “intersects every Π_{1}^{0} set of positive measure.”)

We shall now prove Theorem 9.11.18.

Proof. (2 ⇒ 3) is immediate from Lemma 9.11.21 relativized to A.

(3 ⇒ 4) is trivial.

(4 ⇒ 1) follows from Kuc̆era’s Lemma 9.11.23. Namely, let P be as in 4, i.e., P is Π_{1}^{0,B} of positive measure consisting of A-random elements. Let X be B-random. By Kuc̆era’s Lemma relative to B, X has a tail in P, i.e., X = σ^{⌢}Y where Y ∈ P. By our assumption about P, Y is A-random. Therefore X is A-random.

It remains to prove (1 ⇒ 2).

Assume 1, i.e., A ≤_{LR} B. To prove 2, it suffices to prove that every B-fat Σ_{1}^{0,A} set is of measure 1.

Let U be a Σ_{1}^{0,A} set which is B-fat. Assume for a contradiction that μ(U) < 1. As we have seen, the sets U^{n}, n = 1, 2, 3, … form a test for A-randomness. Hence every X ∈ ⋂_{n=1}^{∞} U^{n} is not A-random. Also, since U is B-fat, by Lemma 9.11.24 relativized to B we see that U^{n} is B-fat for all n.

By Lemma 9.11.21 relativized to B, let P be nonempty Π_{1}^{0,B} consisting of B-random elements. By Lemma 9.11.22 we have μ(P) > 0. More generally, by Lemma 9.11.22 we have

∀σ (P ∩ N_{σ} ≠ ∅ ⇒ μ(P ∩ N_{σ}) > 0).

Since U^{n} is B-fat, it follows that

∀n ∀σ (P ∩ N_{σ} ≠ ∅ ⇒ (P ∩ N_{σ} ∩ U^{n} ≠ ∅)).

Since U^{n} is open, it follows that

∀n ∀σ (P ∩ N_{σ} ≠ ∅ ⇒ (∃τ ⊃ σ) (P ∩ N_{τ} ≠ ∅ ∧ N_{τ} ⊆ U^{n})).

Apply this repeatedly starting with σ_{0} = ⟨⟩ to get an increasing sequence of bitstrings σ_{0} ⊂ σ_{1} ⊂ ⋯ ⊂ σ_{n} ⊂ σ_{n+1} ⊂ ⋯ such that for all n,

P ∩ N_{σ_{n}} ≠ ∅ and N_{σ_{n}} ⊆ U^{n}.

Finally let X = ⋃_{n=1}^{∞} σ_{n}. Then X ∈ P (because P is closed), hence X is Brandom. On the other hand X ∈ ⋂_{n=1}^{∞} U^{n}, hence X is not A-random. This contradicts our assumption A ≤_{LR} B.

This completes the proof of Theorem 9.11.18.

We now prove one more important theorem concerning LR-reducibility.

Theorem 9.11.25 (Kučera/Terwijn, 2002). We can find a simple r.e. set A such that A ≤_{LR} 0.

Corollary 9.11.26. ∃A (A ≤_{LR} 0 and A not recursive).

Corollary 9.11.27. ≤_{LR} does not coincide with ≤_{T}.

Corollary 9.11.28. ∃A (A is recursively enumerable ∧ 0 <_{T} A < T 0′).

Proof. Let A be as in Theorem 9.11.25. Since A is simple, A is recursively enumerable and 0 <_{T} A ≤_{T} 0′. It remains to prove that A <_{T} 0′. For this it suffices to prove that 0′ ≰_{LR} 0, but this is immediate from Exercise 9.11.4.

Remark 9.11.29. We have seen in Theorem 5.5.7 (due to Kleene and Post in 19454) that there exist Turing degrees a such that 0 < a < 0′. Corollary 9.11.28 improves this by obtaining a recursively enumerable Turing degree with these properties. This result is originally due to Friedberg 1957 and Muchnik 1956, independently and by a different method.

Some other known results are as follows:

Theorem 9.11.30. A ≤_{LR} 0 ⇒ A is low i.e., A′ ≡_{T} 0′.

Corollary 9.11.31. A ≤_{LR} 0 ⇒ A ≤_{T} 0′, hence A is Δ_{2}^{0}.

Corollary 9.11.32. There are only countably many A such that A ≤_{LR} 0.

Theorem 9.11.33. If A ≤_{LR} 0 and B ≤_{LR} 0 then A ⊕ B ≤_{LR} 0.

Theorem 9.11.34. A ≤_{LR} 0 ⇒ ∃B (B r.e., A ≤_{T} B, B ≤_{LR} 0).

Theorem 9.11.35. A ≤_{LR} B ⇔ A ≤_{LK} B.

Remark 9.11.36. Let A be a Turing oracle. If A ≤_{LR} 0 we say that A is lowfor-random. If A ≤_{LK} 0 we say that A is low-for-K. The above results have been proved in the past few years and give much insight concerning oracles which are low-for-random. In particular, A is low-for-random if and only if A is low-for-K.

We now begin the proof of the Kučera/Terwijn Theorem 9.11.25.

Proof of Theorem 9.11.25. We know we can find a Π_{1}^{0} set P with μ(P) > 1/2 such that ∀X (X ∈ P ⇒ X is random).

Let us uniformly relativize this to an arbitrary oracle C. Thus P^{C} is uniformly Π_{1}^{0,C}, μ(P^{C}) > 1/2 and ∀X (X ∈ P^{C} ⇒ X is C-random).

Let U^{C} = 2^{ℕ} \ P^{C}. Note that U^{C} is Σ_{1}^{0,C} (uniformly) and μ(U^{C}) < 1/2. To prove the theorem, it will suffice to build a simple r.e. set A and a Σ_{1}^{0} set

V such that U^{A} ⊆ V and μ(V) < 1.

Then, letting Q = 2^{ℕ} \ V, it follows that Q is Π_{1}^{0} and μ(Q) > 0 and all elements of Q are A-random. Hence by Theorem 9.11.18 every random X is A-random, i.e., A ≤_{LR} 0, Q.E.D.

The proof will be completed next class.

Lecture 42: December 3, 2007

We have reduced Theorem 9.11.25 to a lemma:

Lemma 9.11.37. For all oracles C let U^{C} be uniformly Σ_{1}^{0,C} of measure < 1/2. Then we can find a simple r.e. set A and a Σ_{1}^{0} set V such that U^{A} ⊆ V and μ(V) < 1.

We shall now prove this lemma and thereby complete the proof of Theorem 9.11.25.

Proof. We shall build A as A = ⋃_{s=0}^{∞} A_{s} where A_{0} ⊆ A_{1} ⊆ ⋯ ⊆ A_{s} ⊆ A_{s+1} ⊆ ⋯ is an increasing recursive sequence of finite sets. The entire construction will be recursive. Consequently A will be r.e. We will let V = ⋃_{s=0}^{∞} U_{s}^{A_{s}↾s} so that V is Σ_{1}^{0}. We need to insure (1) A is simple, (2) μ(V) < 1.

Explanation: U^{C} is uniformly Σ_{1}^{0,C}, so let e be such that U^{C} = {X | φ_{e}^{(1),C⊕X}(0) ↓} for all oracles C. Define U_{s}^{C↾s} = {X | φ_{e,s}^{(1),C↾s⊕X↾s}(0) ↓} and note that this is a finite union of neighborhoods, namely

U_{s}^{C↾s} = ⋃_{\begin{matrix}|σ| = s \\ φ_{e,s}^{(1),C↾s⊕σ}(0) ↓ \end{matrix}} N_{σ} .

In particular, U_{s}^{A_{s}↾s}, s = 0, 1, 2, … is the union of a finite set of neighborhoods, and this finite set of neighborhoods can be computed recursively from s.

To control μ(V), let us write V as a disjoint union, V = ⋃_{t=0}^{∞} V_{t} where V_{t} = U_{t}^{A_{t}↾t} \ ⋃_{s<t} U_{s}^{A_{s}↾s}. Note that V_{t}, t = 0, 1, 2, … is again a recursive sequence of finite unions of neighborhoods. Moreover, the sets V_{t}, t = 0, 1, 2, … are pairwise disjoint.

Recall that an r.e. set A is said to be simple if (1) ℕ \ A is infinite, (2) ∀e (W_{e} infinite ⇒ A∩W_{e} ≠ ∅). To insure that A is simple, we use our standard enumeration of all r.e. sets, W_{e} = {n | φ_{e}^{(1)}(n) ↓}. We use the approximations W_{e,s} = {n < s | φ_{e,s}^{(1)}(n) ↓}. Clearly W_{e,s} is a recursive sequence of finite sets, and W_{e} = ⋃_{s=0}^{∞} W_{e,s}.

The construction of A is as follows.

Stage 0: Let A_{0} = ∅.

Assume that A_{t}, t = 0, 1, …, s have already been defined. Hence U_{t}^{A_{t}↾t} and V_{t}, t = 0, 1, …, s have already been defined. In preparation for stage s + 1, define c(n, s) = μ(⋃_{n<t≤s} V_{t}) = ∑_{n<t≤s} μ(V_{t}). The recursive function c(n, s) is known as a “cost function.” Its purpose is to measure the potential “cost” of putting n into A at stage s + 1.

Stage s + 1: For each e < s such that A_{s} ∩ W_{e,s} = ∅, look for n ∈ W_{e,s} such that n ≥ 2e and c(n, s) ≤ 1/2^{e+2} and, if such an n is found, put the least such n into A_{s+1}.

This completes our description of the construction. Note that the entire construction is recursive.

We claim that ℕ \ A is infinite.

To see this, note that for each e at most one n was put into A for the sake of intersecting W_{e}. Therefore, our restriction n ≥ 2e insures that |A ∩ {0, 1, 2, …, 2e−1}| ≤ e for all e. It follows that the complement of A is infinite.

We claim that W_{e} infinite ⇒ A ∩ W_{e} ≠ ∅.

To see this, note that ∑_{t} μ(V_{t}) = μ(⋃_{t} V_{t}) ≤ 1, hence ∑_{n<t} μ(V_{t}) → 0 as n → ∞. Since W_{e} is infinite, let n ∈ W_{e} be so large that n ≥ 2e and ∑_{n<t} μ(V_{t}) ≤ 1/2^{e+1}. It follows that c(n, s) = ∑_{n<t≤s} μ(V_{t}) ≤ 1/2^{e+1} for all s. Let s be so large that n ∈ W_{e,s}. Then by construction A_{s+1} ∩ W_{e,s} ≠ ∅.

We claim that μ(V) < 1.

To see this, it suffices to prove that μ(V \ U^{A}) ≤ 1/2 (because we already know that μ(U^{A}) < 1/2). Given X ∈ V \ U^{A}, consider the unique t such that X ∈ V_{t}. Then X ∈ U_{t}^{A_{t}↾t} \ U^{A}. It follows that A ↾ t ≠ A_{t} ↾ t. Therefore, at some stage s + 1 > t, some n < t must have been put into A_{s+1} for the sake of W_{e} for some e < s. For this particular e, the set of all such X’s is included in ⋃_{n<t≤s} V_{t} and is therefore of measure ≤ ∑_{n<t≤s} μ(V_{t}) = c(n, s) ≤ 1/2^{e+1}. Hence, the set of all such X’s for all e is of measure ≤ ∑_{e=0}^{∞} 1/2^{e+1} = 1/2. This proves the claim.

The proof of Lemma 9.11.37 and Theorem 9.11.25 is now complete.

Chapter 10

Solutions to all of the exercises

In this chapter we present solutions to all of the exercises in the previous chapters.