Download Strong Induction - Discrete Mathematics and Probability Theory - Lecture Notes and more Study notes Discrete Structures and Graph Theory in PDF only on Docsity!
This lecture covers further variants of induction, including strong induction and the closely related well- ordering axiom. We then apply these techniques to prove properties of simple recursive programs.
Strong induction
Axiom 3.1 (Strong Induction) : For any property P , if P ( 0 ) and ∀ n ∈ N ( P ( 0 ) ∧ P ( 1 ) ∧... ∧ P ( n ) =⇒ P ( n + 1 )), then ∀ n ∈ N P ( n ).
This says that if all the following sentences are true:
P ( 0 ) P ( 0 ) =⇒ P ( 1 ) P ( 0 ) ∧ P ( 1 ) =⇒ P ( 2 ) P ( 0 ) ∧ P ( 1 ) ∧ P ( 2 ) =⇒ P ( 3 ) P ( 0 ) ∧ P ( 1 ) ∧ P ( 2 ) ∧ P ( 3 ) =⇒ P ( 4 )
and so on, then P ( n ) must be true for all n. Intuitively, this seems quite reasonable. If the truth of P all the way up to n always implies the truth of P ( n + 1 ), then we immediately obtain the truth of P all the way up to n + 1, which implies the truth of P ( n + 2 ), and so on ad infinitum.
If we compare the Strong Induction axiom to the original Induction axiom from Lecture 2, we see that Strong Induction appears to make it easier to prove things. With simple induction, one must prove P ( n + 1 ) given the inductive hypothesis P ( n ); with strong induction one gets to assume the inductive hypothesis P ( 0 ) ∧ P ( 1 ) ∧... ∧ P ( n ), which is much stronger.
Consider the following example, which is one half of the Fundamental Theorem of Arithmetic. (The other half says that the product is unique.)
Theorem 3.1 : Any natural number n > 1 can be written as a product of primes.
To prove this, of course, we need to define prime numbers:
Definition 3.1 (Prime) : A natural number n > 1 is prime iff it has exactly two factors (1 and n ). 1 itself is not prime.
Let’s see first what happens when we try a simple induction:
Proof : (Attempt 1) The proof is by induction over the natural numbers n > 1.
- Base case: prove P ( 2 ). P ( 2 ) is the proposition that 2 can be written as a product of primes. This is true, since 2 can be written as the product of one prime, itself. (Remember that 1 is not prime!)
- Inductive step: prove P ( n ) =⇒ P ( n + 1 ) for all natural numbers n > 1.
CS 70, Fall 2004, Lecture 3 1
- The inductive hypothesis states that n can be written as a product of primes.
- To prove: n + 1 can be written as a product of primes.
- We’re stuck: given P ( n ), we could easily establish P ( 2 n ) or P ( 7 n ), but P ( n + 1 ) is unconnected to P ( n ).
2
With a strong induction, we can make the connection between P ( n + 1 ) and earlier facts in the sequence that are relevant. For example, if n + 1 =72, then P ( 36 ) and P ( 24 ) are useful facts.
Proof : The proof is by strong induction over the natural numbers n > 1.
- Base case: prove P ( 2 ), as above.
- Inductive step: prove P ( 2 ) ∧... ∧ P ( n ) =⇒ P ( n + 1 ) for all natural numbers n > 1.
- The inductive hypothesis states that, for all natural numbers m from 2 to n , m can be written as a product of primes.
- To prove: n + 1 can be written as a product of primes.
- Proof by cases: - n + 1 is prime: then n + 1 can be written as the product of one prime, itself. - n + 1 is not prime: then by the definition of prime numbers, there exist integers a , b such that 2 ≤ a , b < n + 1 and n + 1 = a · b. By the inductive hypothesis, both a and b can be written as a product of primes. Hence n + 1 can be written as a product of primes.
2
Consider the following example, which is of immense interest to post offices and their customers:
Theorem 3.2 : Any integer amount of postage from 8 ¢ upwards can be composed from 3 ¢ and 5 ¢ stamps.
With a strong induction, we can make the connection between P ( n + 1 ) and earlier facts in the sequence. In particular, P ( n − 2 ) is relevant because n + 1 can be composed from the solution for n − 2 plus one 3¢ stamp. So the inductive step works if P ( n − 2 ) is known already. This will not be the case when n + 1 is 9 or 10, so we will need to handle these separately.
Proof : The proof is by strong induction over the natural numbers n ≥ 8.
- Base case: prove P ( 8 ). P ( 8 ) is the proposition that 8¢ of postage can be composed from 3¢ and 5¢ stamps. This is true, requiring 1 of each.
- Inductive step: prove P ( 8 ) ∧... ∧ P ( n ) =⇒ P ( n + 1 ) for all natural numbers n ≥ 8.
- The inductive hypothesis states that, for all natural numbers m from 8 to n , m ¢ of postage can be composed from 3¢ and 5¢ stamps.
- To prove: ( n + 1 )¢ of postage can be composed from 3¢ and 5¢ stamps.
- The cases where n + 1 is 9 or 10 must be proved separately. 9¢ can be composed from three 3¢ stamps. 10¢ can be composed from two 5¢ stamps.
- For all natural numbers n + 1 > 10, the inductive hypothesis entails the proposition P ( n − 2 ). If ( n − 2 )¢ can be composed from 3¢ and 5¢ stamps, then ( n + 1 )¢ can be composed from 3¢ and 5¢ stamps simply by adding one more 3¢ stamp.
CS 70, Fall 2004, Lecture 3 2
Theorem 3.3 : The strong induction axiom and the simple induction axiom are logically equivalent.
Why have two different forms of induction then? The point is that strong induction reminds its user of the opportunity to use P ( 0 ) ∧... ∧ P ( n ) in the inductive step when P ( n ) is defined the “natural” way from the statement of the theorem to be proved.
The well-ordering principle
If one thinks about why induction works, one might ask the question “How could the induction axiom fail to be true?” To violate the induction axiom, we would need to satisfy its antecedent (so P ( 0 ) is true and P ( n ) =⇒ P ( n + 1 ) for all n ) while violating its consequent (so ∃ n ¬ P ( n )). Let us consider the first n for which P ( n ) is false. By definition, we know that P ( n − 1 ) is true; and by assumption we know that P ( n − 1 ) =⇒ P ( n ); therefore we have a straightforward contradiction!
Have we proved the induction axiom? Actually, no; we have proved that the induction axiom follows from another axiom, which was used implicitly in defining “the first n for which P ( n ) is false.”
Axiom 3.2 (Well-Ordering) : Every nonempty set of natural numbers has a smallest element.
Duh. Doesn’t every nonempty set of orderable elements have a smallest element? No! Every finite set has a smallest element, but not every infinite set. For example, neither the integers nor even the positive rationals have a smallest element.
The well-ordering principle not only underlies the induction axioms, but also has direct uses in its own right. A particularly elegant example concerns the existence of cycles in tournaments.
Definition 3.2 (Round-Robin) : A round-robin tournament is one in which each player p plays each other player q exactly once and either wins ( p q ) or loses ( q p ).
Definition 3.3 (Cycle) : A cycle in a tournament is a set of players { p 1... pk } such that p 1 p 2 ... pk − 1 pk p 1.
Theorem 3.4 : In every round-robin tournament, if there is a cycle, then there is a cycle of length 3.
Proof : The proof is by contradiction.
- Assume the theorem is false. Consider the set of cycle lengths of the tournament. By assumption, this must be nonempty.
- By the well-ordering principle, it must have a smallest element k. By assumption, k > 3.
- Let the first three elements in this cycle be p 1 , p 2 , p 3 , and consider the result of the match between p 1 and p 3.
- Case 1: p 1 p 3. Then we have p 1 p 3 ... pk − 1 pk p 1 , i.e., a cycle of length k − 1, contradicting our assumption that the smallest cycle has length k > 3.
- Case 2: p 3 p 1. Then we have p 1 p 2 p 3 p 1 , i.e., a cycle of length 3, contradicting our assumption that the smallest cycle has length k > 3.
- By the definition of round-robins, either p 1 p 3 or p 3 p 1. Therefore, a contradiction exists.
- Hence, it must be the case that any tournament with a cycle has a cycle of length 3.
2
CS 70, Fall 2004, Lecture 3 4
This proof illustrates a common way to use well-ordering combined with proof by contradiction. The well- ordering principle allows one to focus on a concrete counterexample with the property that every smaller example satisfies some property. For certain proofs, this can be an easier thought process than induction.
Induction and recursion
There is an intimate connection between induction and recursion. Essentially every recursive function relies for its correctness on an inductive proof. Remember that a recursive function applies itself to a “smaller” argument. The inductive proof says that if the recursive function works on all smaller arguments it will work on the current argument.
We’ll begin with that old favourite, the factorial function. Let’s give a recursive definition for a function f ( n ) and show it’s identical to n !. For any n ∈ N ,
f ( n ) = 1 if n = 0 f ( n ) = n f ( n − 1 ) otherwise
Theorem 3.5 : For all natural numbers n, f ( n ) = n!.
Proof : The proof is by induction over the natural numbers. Let P ( n ) be the proposition that f ( n ) = n !.
- Base case: prove P ( 0 ). P ( 0 ) is the proposition that f ( 0 ) = 0!. By the definition above, f ( 0 ) = 1 = 0!, hence P ( 0 ) is true.
- Inductive step: prove P ( n ) =⇒ P ( n + 1 ) for all n ∈ N.
- The inductive hypothesis is f ( n ) = n !.
- To prove: f ( n + 1 ) = ( n + 1 )!.
- By the definition above, f ( n + 1 ) = ( n + 1 ) · f ( n ) because n ∈ N so ( n + 1 ) 6 = 0 = ( n + 1 ) · n! by the inductive hypothesis = ( n + 1 ) · n · ( n − 1 ) · · · 1 = ( n + 1 )!
Hence, by the induction principle, ∀ n ∈ N f ( n ) = n !. 2
Mathematical functions and real programs
The above discussion applies to a purely mathematical definition of f ( n ). If we wanted to reason about a real program, first we have to write it in a real language, such as Scheme:
(define (factorial n) (if (= n 0) 1 (* n (factorial (- n 1)))))
CS 70, Fall 2004, Lecture 3 5
For the most part, we will use mathematical rather than Scheme definitions because it makes the proofs typographically cleaner and the theorems true.
Induction over things besides numbers
Persons other than pure mathematicians often write programs that manipulate objects other than natural numbers—for example, strings, lists, trees, arrays, hash tables, programs, airline schedules, and so on. So far, the examples of induction we have seen deal with induction over the natural numbers. How does this help with these other domains? One answer is that we can do inductive proofs over natural numbers that correspond to the size of the STRINGS objects under consideration. Suppose we want to prove that ∀ s P ( s ) for the domain of strings. Then define a proposition on natural numbers as follows:
Q ( n ) is the property that every string s of length n satisfies P ( s ).
Then a proof that ∀ n Q ( n ) by induction on n establishes that ∀ s P ( s ). Similarly, we can prove things about trees by induction on the depth of the tree, or about programs by induction on the number of symbols in the program. These inductions can become quite cumbersome and unnatural. Let’s suppose we had never heard of the natural numbers; could we still do anything with strings and trees and programs? It turns out that we can define very natural induction principles for these sorts of objects without mentioning numbers at all.
An induction principle for strings
Let’s write a recursive algorithm for reversing a string and show that it works correctly.
SYMBOLS First, we will need to say what strings are. The elements of a string are symbols drawn from a set of ALPHABET symbols called an alphabet , which is usually denoted Σ. For example, if Σ ={ a , b }, then strings can consist of sequences of a s and b s. Σ∗^ denotes the set of all possible strings on the alphabet Σ, and always includes the empty string, which is denoted λ. Every symbol of Σ is also a string of length 1. (Note: this property in particular distinguishes strings from lists; but in general reasoning about strings is quite similar to reasoning about lists.)
CONCATENATION The basic way to construct strings is by concatenation. If s 1 and s 2 are strings, then their concatenation is also a string and is written s 1 s 2 or s 1 · s 2 if punctuation is needed for clarity. Concatenation is defined as follows: Axiom 3.3 (Concatenation) :
∀ s ∈ Σ∗ λ · s = s · λ = s ∀ a ∈ Σ ∀ s 1 , s 2 ∈ Σ∗^ ( a · s 1 ) · s 2 = a · ( s 1 · s 2 )
Just as Peano did for the natural numbers, we now provide axioms concerning what strings are, then we state an induction principle that allows proofs for all strings. Strings satisfy the following axioms: Axiom 3.4 (Strings) :
The empty string is a string: λ ∈ Σ∗ Joining any symbol to a string gives a string: ∀ a ∈ Σ ∀ s ∈ Σ∗^ a · s ∈ Σ∗
CS 70, Fall 2004, Lecture 3 7
Because these axioms do not strictly define strings, we need an induction principle to construct proofs over all strings: Axiom 3.5 (String Induction) :
For any property P , if P ( λ ) and ∀ a ∈ Σ ∀ s ∈ Σ∗^ ( P ( s ) =⇒ P ( a · s )), then ∀ s ∈ Σ∗^ P ( s ).
STRUCTURALINDUCTION This is a simple instance of structural induction , where a set of axioms defines the way in which objects
in a set are constructed and an induction principle uses the construction step repeatedly to cover the entire CONSTRUCTOR domain. Here, “·” is the constructor for the domain of strings, just as “+1” is the constructor for the natural numbers. Notice that numbers appear nowhere in these axioms. We can do proofs thinking only about the objects in question. Let’s define a function that reverses a string and prove that it works. Axiom 3.6 (Reverse) :
r ( λ ) = λ ∀ a ∈ Σ ∀ s ∈ Σ∗ r ( a · s ) = r ( s ) · a
We would like to say something like “for every string s , r ( s ) reverses it.” To make this a precise theorem, we’ll need some independent, non-recursive way to say what we mean by reversing! There are several ways to do this, of which the easiest is to take advantage of “dot dot dot” notation: Theorem 3.7 : ∀ s ∈ Σ∗ , let s = a 1 a 2... an; then r ( s ) = an... a 2 a 1 Proof : The proof is by induction over the strings on the alphabet Σ. Let P ( s ) be the proposition that if s = a 1 a 2... an , then r ( s ) = an... a 2 a 1.
- Base case: prove P ( λ ). P ( λ ) is the proposition that r ( λ ) = λ , which is true by definition.
- Inductive step: prove P ( s ) =⇒ P ( a · s ) for all a ∈ Σ, s ∈ Σ∗.
- The inductive hypothesis states that, for some arbitrary string s , if s = a 1 a 2... an , then r ( s ) = an... a 2 a 1.
- To prove: for every symbol a , r ( a · s ) = an... a 2 a 1 a.
- By the axiom for reverse, r ( a · s ) = r ( s ) · a by the reverse axiom = an... a 2 a 1 a by the inductive hypothesis
Hence, by the string induction principle, for every string s , r ( s ) reverses it. 2 We could alternatively have proven this theorem by induction over the length of the input string. It is an excellent exercise to work out the details of how to do this, and compare to the above method.
Induction over binary trees
Trees are a fundamental data structure in computer science, underlying efficient implementations in many areas including databases, graphics, compilers, editors, optimization, game-playing, and so on. Trees are
CS 70, Fall 2004, Lecture 3 8
The function maxleaf is “correct” if it satisfies two properties: first, maxleaf ( t ) has to be greater than or equal to every leaf of t ; second (and often forgotten ), maxleaf ( t ) has to be a leaf of t! Let’s prove the second property first: Theorem 3.8 : For every tree, t, maxleaf ( t ) is a leaf of t. Proof : The proof is by induction over the binary trees on the atoms A. Let P ( t ) be the proposition leaf ( maxleaf ( t ), t ).
- Base case: prove ∀ a ∈ A P ( a ). P ( a ) is the proposition that leaf ( maxleaf ( a ), a ), which is equivalent by substitution to the proposition leaf ( a , a ), which is true by definition.
- Inductive step: prove P ( t 1 ) ∧ P ( t 2 ) =⇒ P ( t 1 • t 2 ) for all t 1 , t 2 ∈ T.
- The inductive hypothesis states that leaf ( maxleaf ( t 1 ), t 1 ) ∧ leaf ( maxleaf ( t 2 ), t 2 ).
- To prove: leaf ( maxleaf ( t 1 • t 2 ), t 1 • t 2 ).
- By the definition above, maxleaf ( t 1 • t 2 ) = max ( maxleaf ( t 1 ), maxleaf ( t 2 )).
- Since ∀ x , y [( max ( x , y ) = x ) ∨ ( max ( x , y ) = y )], we have ( maxleaf ( t 1 • t 2 ) = maxleaf ( t 1 )) ∨ ( maxleaf ( t 1 • t 2 ) = maxleaf ( t 2 )).
- Substituting in the induction hypothesis, we obtain leaf ( maxleaf ( t 1 • t 2 ), t 1 ) ∨ leaf ( maxleaf ( t 1 • t 2 ), t 2 ).
- Hence, by the definition of leaf , leaf ( maxleaf ( t 1 • t 2 ), t 1 • t 2 ).
Hence, by the binary induction principle, for every tree t , maxleaf ( t ) is a leaf of t. 2 The other part of the verification is the following (the proof is left as an exercise): Theorem 3.9 : For every tree, t, maxleaf ( t ) is greater than or equal to every leaf of t. Tree induction seems very natural. Could we do a similar proof using natural number induction? Certainly we can prove facts about trees by induction over the depth of the tree. P ( n ) would state that all trees of depth n satisfy some property Q. Unfortunately, the inductive step for a simple induction would look like this:
Given: all trees t of depth n satisfy Q ( t ) Prove: all trees t of depth n + 1 satisfy Q ( t )
This is usually impossible: for a tree of depth n + 1, one subtree has depth n , but not necessarily the other. Strong induction over the depth of the tree does work; in fact it can always be used instead of tree induction.
Induction over pairs of natural numbers
CARTESIAN PRODUCT Often we need to prove properties over the Cartesian product of some given sets. The Cartesian product PAIRS of sets A and B is written A × B. It is the set of all pairs ( a , b ) where a ∈ A and b ∈ B. For example, the set N × N is the set of all pairs of natural numbers. Such sets arise when we prove properties of functions with two arguments, when we prove facts about all points on a grid, etc. Let’s look at an example: the knight’s tour. We will prove that a knight starting at (0,0) can visit every square on the unbounded nonnegative quadrant. Figure 1 shows (part of) the infinite board and illustrates the moves a knight can make.
CS 70, Fall 2004, Lecture 3 10
Figure 1: The knight’s tour, showing the “base case” squares, the possible legal moves for a knight, and the “inductive step.”
To prove this result, we’ll need some facts about knight’s moves. In particular, we’ll need the following:
Axiom 3.11 (Knight’s Move) :
If square ( x ± 1 , y ± 2 ) or ( x ± 2 , y ± 1 ) is reachable by a knight, then square ( x , y ) is reachable by a knight.
We’ll also need an induction principle for pairs of natural numbers. The idea for the knight’s move proof is to establish a region that is reachable and then to show that any square adjacent to that region is reachable; hence the region grows to fill the unbounded quadrant. There are many ways to define the shape of this region; we’ll use the triangular region shown in Figure 1.
Our induction principle is, informally, that if the truth of P for every pair ( x ′, y ′) in the region “just below” ( x , y ) implies the truth of P for ( x , y ), then P is true for all ( x , y ). Notice that this is a strong induction principle.
Axiom 3.12 (Strong Induction (Pairs)) :
For any property P , if ∀ x , y ∈ N [∀ x ′, y ′^ ∈ N ( x ′^ + y ′) < ( x + y ) =⇒ P ( x ′, y ′)] =⇒ P ( x , y ) then ∀ x , y ∈ N P ( x , y ).
But where is the base case? Actually, it’s there but hidden. When ( x , y ) = ( 0 , 0 ), the condition [∀ x ′, y ′^ ∈ N ( x ′^ + y ′) < ( x + y ) =⇒ P ( x ′, y ′)] is vacuously true because there are no such pairs. Hence P ( 0 , 0 ) is part of the premise to be proved. More generally, the “base case” is the set of ( x , y ) pairs for which the inductive hypothesis does not suffice to provide a proof.
CS 70, Fall 2004, Lecture 3 11
For any property P , and any wellfounded relation ≺ on X , if ∀ x ∈ X [[∀ y ∈ X y ≺ x =⇒ P ( y )] =⇒ P ( x )] then ∀ x ∈ X P ( x ).
As with induction over pairs, the well-founded induction principle includes the requirement for establishing the “base case”—that is, proving P ( x ) independently for all those x where the inductive hypothesis does not suffice. The property of well-foundedness is easy to see for all the cases we have covered. There is also a generalized equivalent of well-ordering:
WELL-ORDERED Definition 3.5 (Well-ordering) : A set X is well-ordered by the relation ≺ iff every nonempty subset of X has at least one minimal element with respect to ≺. The following very general theorem can be proved: Theorem 3.11 : A relation ≺ on X is well-founded iff X is well-ordered by ≺. Although this seems very abstract and useless, it is in fact used all the time by programmers who write recursive functions that do complex things to their arguments. Consider the following recursive skeleton:
f ( x ) = if B ( x ) then k else f ( g ( x ))
This will terminate iff g ( x ) ≺ x for some well-ordering of X with minimal element(s) satisfying B ( x ). Thus, the programmer must be sure that repeated application of g cannot generate an infinite sequence of values that do not satisfy B. Sometimes, “smaller” can be surprisingly nonobvious. Consider the following function on the natural num- bers:
f ( 0 ) = 1; f ( 1 ) = 1 if n > 1 is even then f ( n ) = f ( n / 2 ), else f ( n ) = f ( 3 n + 1 ).
COLLATZ CONJECTURE The Collatz conjecture states that ∀ n ∈ N f ( n ) = 1. You may wish to check this out for various values of n. No proof is known.
CS 70, Fall 2004, Lecture 3 13
