Download Algorithms and Efficiency: Understanding Selection Sort and Big O Notation and more Study notes Discrete Structures and Graph Theory in PDF only on Docsity!
What’s left??
- Algorithms: Unambiguous instructions to accomplish some task.
To find my name in the phone book, scan trough the names in order until you find mine.
- Recursion: Induction for algorithms.
To find my name in part of the the phone book, divide it in half, figure out which half contains my name, and find my name in that half.
- Counting: How many X’s are there in Y?
How many four-letter words can you make with the letters in ABRACADABRA?
- Big-O notation: How to talk about the running time of algorithms with- out knowing anything about the CPU speed, programming language, compiler, operating system, etc.
Linear search takes O(n) time; binary search takes O(log n) time.
- Solving recurrences: How fast is that recursive algorithm?
If T (n) ≤ T (n/2) + O(1), then T (n) = O(log n).
Algorithms
An algorithm is a set of unambiguous instructions describing how to transform some input into some output.
- The input and output are from some specified domains.
- Generality: The algorithm applies to many different inputs.
- Each step is precisely described.
- Each step can be performed in finite time.
- For any input, the algorithm finishes after a finite amount of time.
Richard Feynman’s problem-solving “algorithm”:
- Write down the problem.
- Think very hard.
- Write down the answer.
What we need to know about algorithms:
- Correctness: Does it do what we want?
- Efficiency: How fast is it?
Is it correct?
SELECTIONSORT(A[1 .. n]): for i ← 1 to n for j ← i + 1 to n if A[j] < A[i] tmp ← A[i] A[i] ← A[j] A[j] ← tmp
How do we prove that an algorithm is correct?
What does “correct” actually mean?
Theorem: SELECTIONSORT reorders the elements of any input array A[1 .. n] into increasing order:
A[1] ≤ A[2] ≤ A[3] ≤ · · · ≤ A[n]
In other words, SELECTIONSORT sorts its input.
Lemma: For any input array A[1 .. n] and any integer i, after the ith itera- tion of the outer loop, A[i] is the smallest element of A[i .. n].
SELECTIONSORT(A[1 .. n]): for i ← 1 to n for j ← i + 1 to n if A[j] < A[i] swap A[i] ↔ A[j]
Lemma: For all input arrays A[1 .. n], for all i and j such that 1 ≤ i < j ≤ n, during the ith iteration of the outer loop, after the jth iteration of the inner loop, A[i] is the smallest element of A[i .. j].
Proof: Let A[1 .. n] be an arbitrary input array. Let i and j be arbitrary integers such that 1 ≤ i < j ≤ n. Consider the ith iteration of the outer loop.
Assume that for all k < j, after the kth iteration of the inner loop, A[i] is the smallest element of A[i .. k].
There are two cases to consider: j = i + 1 and j > i + 1.
- Suppose j = i + 1. If A[i] > A[i + 1] before the if statement, then the algorithm swaps A[i] and A[i + 1], so A[i] < A[i + 1] afterwards. If A[i] ≤ A[i + 1] before the if statement, then the algorithm does not swap A[i] and A[i + 1], so A[i] ≤ A[i + 1] afterwards.
- Suppose j ≥ i + 2. The inductive hypothesis implies that just before the jth iteration of the inner loop, A[i] = min A[i .. j − 1]. So in the jth iteration, the algorithm compares min A[i .. j −1] with A[j] and swaps the smaller of the two values into A[i]. So the new value of A[i] is min{min A[i .. j − 1], A[j]} = min A[i .. j].
In both cases, we conclude that after the jth iteration of the inner loop, A[i] is the smallest element of A[i .. j].
Loop invariants
SELECTIONSORT(A[1 .. n]): for i ← 1 to n 〈〈A[1 .. i − 1] is sorted〉〉 for j ← i + 1 to n 〈〈A[i] is smallest in A[i .. j − 1]〉〉 if A[j] < A[i] swap A[i] ↔ A[j] 〈〈A[i] is smallest in A[i .. j]〉〉 〈〈A[i] is smallest in A[i .. n]〉〉 〈〈A[1 .. i] is sorted〉〉 〈〈A[1 .. n] is sorted〉〉
How long does it take?
SELECTIONSORT(A[1 .. n]): for i ← 1 to n for j ← i + 1 to n if A[j] < A[i] swap A[i] ↔ A[j]
- What computer? What language? What compiler? What OS?
- How long does it take to compare A[i] and A[j]? To swap A[i] and A[j]? To maintain i and j?
- How many times do we swap?
- What numbers are in the array A[1 .. n]?
- What is n?
How long does it take?
SELECTIONSORT(A[1 .. n]): for i ← 1 to n for j ← i + 1 to n if A[j] < A[i] swap A[i] ↔ A[j]
The computer science answer:
SELECTIONSORT runs in Θ(n^2 ) time.
Next time we’ll see what this actually means.
