Download 01B - Inductive Reasoning Notes and more Lecture notes Computer science in PDF only on Docsity!
Mathematical Induction
Victor Adamchik
Fall of 2005
Lecture 1 (out of three)
Plan
- The Principle of Mathematical Induction
- Induction Examples
The Principle of Mathematical Induction
Suppose we have some statement Pn and we want to demonstrate that Pn is true for all n . Even if we can provide proofs for P 0 , P 1 , ..., Pk, where k is some large number, we have accomplished very little. However, there is a general method, the Princi- ple of Mathematical Induction.
Induction is a defining difference between discrete and continuous mathematics.
Principle of Induction. In order to show that n, Pn holds, it suffices to establish the following two properties:
(I1) Base case: Show that P 0 holds.
(I2) Induction step: Assume that Pn holds, and show that Pn 1 also holds.
In the induction step, the assumption that Pn holds is called the Induction Hypothesis (IH). In more formal notation, this proof technique can be stated as P 0 k Pk Pk 1 n Pn
You can think of the proof by (mathematical) induction as a kind of recursive proof:
Instead of attacking the problem directly, we only explain how to get a proof for Pn 1 out of a proof for Pn.
How would you prove that the proof by induction indeed works??
Proof (by contradiction) Assume that for some values of n, Pn is false. Let n 0 be the least such n that Pn 0 is false. n 0 cannot be 0, because P 0 is true. Thus, n 0 must be in the form n 0 1 n 1. Since n 1 n 0 then by Pn 1 is true. Therefore, by inductive hypothesis Pn 1 1 must be true. It follows then that Pn 0 is true. Contradiction.
The crucial part of this proof is that we are able to get a least element. This is called the Least Element Principle. We have been using this principal a few times when we cover the divisibility.
Comments
The base case can start with any nonnegative number. If that number is n 0 then you would prove that assertion Pn holds for all n n 0. Pn 0 k n 0 Pk Pk 1 n Pn The induction step not necessarily should start with n. You can change the step from n 1 to n, where n 0. Sometimes this yields slightly shorter expressions. However, you cannot make a step from n 1 to n 1.
Induction Examples
First Example Prove for n 1 1 1 2 2 3 3 ... n n n 1 1 This could be also written by using notation
Claim. The ATM machine can generate any output amount n 4.
Proof: By induction on n. Base case: n 4. 2 two's, done.
Induction step: suppose the machine can already handle n 4 dollars. To produce n 1 dollars, we proceed as follows.
Case 1: The n dollar output contains a five. Then we can replace the five by 3 two's to get n 1 dollars.
Case 2: The n dollar output contains only two's. Since n 4, there must be at least 2 two's. Remove 2, and replace them by 1 five.
Tower of Hanoi
You have three pegs and a collection of disks of different sizes. Initially all of the disks are stacked on top on each other according to size on the first peg -- the largest disk being on the bottom, and the smallest on the top. A move in this game consists of moving a disk from one peg to another, subject to the condition that a larger disk may never rest on a smaller one. The objective of the game is to find a number of permissable moves that will transfer all of the disks from the first peg to the third peg, making sure that the disks are assembled on the third peg according to size. The second peg is used as an intermediate peg.
Claim. It takes 2n^ 1 moves to move n disks from the first peg to the third peg.
Proof: Base case: n 1. It takes exactly one move. Indeed, 21 1 1.
Inductive hypothesis: suppose it takes 2n^ 1 moves to move n disks.
We need to prove that the number of moves for n 1 disks is
2 n^1
We proceed as follows. First, we move top n disks to the second peg (using the third peg as an intermediate peg.) It takes 2n^ 1 moves. Then, we move the last disk to the third peg - it takes one move. Finally, we move n disks from the second peg to the third peg. Here is the total number of moves
2 n^1 1 2 n^1 2 2 n^1 2 n^1
All horses are the same color
Claim. All horses are the same color.
Proof. The proof by induction on the number of horses.
Base: if there is one horse, then it is trivially the same color as itself.
Inductive hypothesis: assume that horses, numbered from 1 to n have the same color.
Inductive step: There are n 1 horses. By the induction hypothesis, horses from 1 to n have the same color, let's say white. In particular, horse 2 is white. Also by induction hypothesis 2 through n 1 have the same color - white, since 2 is white. Therefore, all of the horses have the same color.
Explanation. We deal with two sets of horses
- from 1 to n. Denote this by G 1.
- from 2 to n 1 Denote this by G 2.
Both of them have n horses and thus we conclude that horses in the union G 1 G 2 have the same color.
It would be true if their intersection G 1 G 2 is NOT empty. And this is a flaw! If G 1 and G 2 are disjoint, then there is no reason why horses from both sets have the same color. When these two sets are disjoint? When n 1. Each set has only one element. P 1 does not imply P 2 .
Mathematical Induction
Victor Adamchik
Fall of 2005
Lecture 2 (out of three)
Plan
- Strong Induction
- Faulty Inductions
- Induction and the Least Element Principal
Strong Induction
Fibonacci Numbers Fibonacci number Fn is defined as the sum of two previous Fibonacci numbers Fn Fn 1 Fn 2 F 1 1, F 0 0 Claim. Fibonacci numbers are growing exponentially Fn n^2 , n 2 where is the golden ratio
1 5 2
Proof: Base case: n 2 F 2 ^2 ^2 1
Inductive hypothesis: assume the following
Fn n^2 Fn 1 n^3
Note, we need two assumptions. Prove that
Fn 1 n^1
We start with the definition
Fn 1 Fn Fn 1
Next we use the inductive hypothesis to obtain
Fn 1 Fn Fn 1 n^2 n^3 n^3 1
Now we use the property of the golden ratio (prove this!)
^2 1
Substituting this into the previous formula, we get
Fn 1 n^3 1 n^1
Question. Where would be the proof failed if you attempted to prove Fn n^2.
Formal Definition
The weak form induction is stated as
Pn 0 n n 0 Pn Pn 1
Here Pn 0 is a base case and Pn is the inductive hypothesis. To prove that Pn is true for n n 0 , we have to
- show that Pn 0 is true
- show that Pn Pn 1 is true for n n 0
How to justify the proof by strong induction?
The proof is by contradiction.
Suppose that some statements in the list P 1 , ..., Pn were actually false. We choose the first false statement, say Pm, where m 0. Now we know that P 0 , P 1 , ..., Pm 1 are true. Then by inductive hypothesis, Pm logically follows from P 0 , P 1 , ..., Pm 1 . Therefore, Pm is true. Contradiction.
Binary Search
The number of comparisons used during binary search in a table of size n in the worst case described by the recurrence
an a (^) n 2 1, a 1 1
with the solution
an log 2 n 1
Proof. Base case: a 1 log 2 1 1 0 1 1
Inductive hypothesis: Assume ak log 2 k 1 for k 2 , ..., n 1.
Inductive step: prove for k n:
an log 2 n 1
We start with
an a (^) n 2 1
and make a use of inductive hypothesis
a (^) n 2 log 2 ^ n 2 1
to obtain
an log 2 ^ n 2 2
Let n is even n 2 p, then a 2 p log 2 p 2 log 2 2 p log 2 2 2 log 2 2 p 1
Let n is odd n 2 p 1 , then
a 2 p 1 log 2 ^2 p 2 ^1 2 log 2 2 p 1 1
Faulty Inductions
Example 1 Claim: Every positive integer n 2 has a unique prime factorization
Proof. Base step: P 2 is true
Induction step: Assume that Pk is true for 2 k n
Prove that Pn 1 is true
There are two possibilities: Case 1 : n 1 is prime. Then we are done. Case 2 : n 1 is composite. Let n 1 p q where 1 p, q n 1. By inductive hypothesis, p and q have unique factorizations. Since the product of two unique factorizations is again unique, we conclude the proof. QED
Explanation. n 1 p q is NOT unique.
Example 2 Claim. 6 n 0 for all n 0.
Base step: Clearly 6*0 = 0.
Induction and the Least Element Principal
Weak vs. Strong These two forms of induction are equivalent. They only differ from each other from the point of view of writing a proof. It is always possible to convert a proof using one form of induction into the other.
The conversion from weak to strong form is trivial, because a weak form is already a strong form.
The conversion from a strong form into a weak form is more interesting. Here are two forms Pn 0 n n 0 Pn Pn 1 Pn 0 n n 0 Pn 0 Pn 0 1 ... Pn Pn 1 We introduce a new hypothesis Qn defined by Qn : Pn 0 Pn 0 1 ... Pn The base step is identical in both cases, namely Qn 0 .
The inductive step is Qn Pn 1 Since Qn implies itself, we rewrite the above statement as Qn Qn Pn 1 which is equivalent (by definition of Q) Qn Qn 1 Therefore, the strong induction in P can be written as a weak induction in Q Qn 0 n n 0 Qn Qn 1
Inductions vs. the Least Principal Element
Least Element Principal or Least Number Principal or Well-Ordering Principle:
Each non-empty subset of has a least element.
In this section we prove the following
Theorem. Induction and the Least Element Principal are logically equivalent.
Proof. We prove that strong induction implies the least element principal. The proof is by contrapositive.
Negate
|S| 0 min. element
to get
min. element |S| = 0
Suppose S is a subset of with no minimal element. Define proposition P by
Pn : n S
We will show that P satisfies all conditions of strong induction. Clearly P 0 is true. Assume that P 0 , P 1 , ..., Pn are all true. This means that none of 1, 2, ..., n is in S. What about Pn 1 ? It is true as well, because othewise, we would have that n 1 S and, therefore, S would have a minimal element. So, now all conditions of strong induction are satisfied. It follows then that Pn is true for all n 0, hence S is empty.
Exercise 1.
Prove that every positive integer has a unique representation of the form
n Fp Fp 1 ... Fq 1 Fq
where Fk are Fibonacci numbers.
Mathematical Induction
Victor Adamchik
Fall of 2005
Lecture 3 (out of three)
Plan
- Recursive Definitions
- Recursively Defined Sets
- Program Correctness
Recursive Definitions
Sometimes it is easier to define an object using a self-reference. For example,
A linked list is either empty list or a node followed by a linked list.
A binary tree is either empty tree or a node containing left and right binary trees.
Sometimes self-referencing leads to paradoxes:
The set of all sets which aren't elements of themselves.
The process of defining an object using a self-reference is called recursion. A primitive recursion is a restricted way of defining f n 1 in terms of f n . A recursive function is a function that calls itself in order to return an answer. A recursive definition of a function is given in two parts
- a set of base values (or initial values)
- a rule for calculating f n in terms of previous values
This is a typical example of recursive definition (or inductive definition) f 0 5 f n 1 f n 1 Here is another example, GCD0, b b GCD a , b GCD b , a , if a b GCD a , b GCD b mod a , a A palindrome is an expression that reads the same backwards and forwards. For example,
Rats live on no evil star
Madam I'm Adam
Let us define a palindrome over a , b , c , d alphabet recursively.
Initial values: P 0 P 1 a , b , c , d General rule:
Pn 1 a a , b b , c c , d d Pn 1 , n 1
Recursively Defined Sets
We start with an example, 2 S if a S b S a b S
Claim. The above set is a set of positive even integers.
Proof. Let E be a set of ALL positive even integers. We have to prove
The second rule says that new strings are generated by concatenation. The length of a string L is defined by
- L empty 0
- L x L 1 , x
Based on the above two definition we prove
L 1 2 L 1 L 2 , 1 , 2
Proof (by induction on 2 )
Basis step: 2 empty. By the definition of the length of a string,
L 1 2 L 1 L 1 L 2 L 1 0
Inductive step: we assume that
L 1 2 L 1 L 2 , 1 , 2
for all 1 L 2 n. We have to prove the above formula for L 2 n 1. Note that by recursive definition,
2 ^ ^ x , ^ ^ , x
Therefore,
L 1 2 L 1 ^ ^ x L 1 ^ 1 by^ I H^ L 1 L ^ 1 L 1 L ^ ^ x
which concludes the proof.
Program Correctness
How can we be sure that a particular algorithm implementation is correct?
int prod = 1; for ( int k=1; k<=n; k++) prod *= k; return prod;
The idea is to use a loop invariant - an assertion that is true before and after each execution of the body of the loop.
[precondition] while (guard) { loop-body } [postcondition]
In the above example, a loop invariant is the following proposition P : prod k 1 k n A loop invariant should serve two purposes: to state what the loop is supposed to accom- plish and to help in proving the algorithm correctness.
To prove that P is a loop invariant we use a mathematical induction. First we note that P is true before the loop is entered, since prod 1 . Next we assume that P is true for 1 k n , namely after n 1 loop executions. In the next execution, k is incremented by 1 (thus, it becomes n ) and prod k. Since by inductive hypothesis the previous value of prod is k 1 , we conclude that prod n . Therefore, P remains true. Finally, we need to show that the program terminates, which is trivial in our case.
