Download Lecture 8: Algorithm Analysis and Sorting in CPSC 223 and more Slides Data Structures and Algorithms in PDF only on Docsity!
CPSC 223
Algorithms & Data Abstract Structures
Lecture 8:
Algorithm Analysis and Sorting
Today …
• Homework
- Homework 3 Due
- Homework 4 Assigned
• More on complexity analysis [Sect. 9.1]
• Binary search
• Bubble, selection, and insertion sort [Sect. 9.2]
Next week:
- Midterm (Thurs.) CPSC 223 -‐-‐ Fall 2010 2
Order-of-Magnitude Analysis
Big O notation focuses algorithm growth rates
- an upper bound (running time will not be worse)
- independent of the particular implementation or computer used to execute the algorithm
Given algorithm A, we say A requires time proportional
to a function f ( n ) …
- n is the size of the input
- we say that “A is order f ( n )”
- … which we write as O ( f ( n )) CPSC 223, 2009 3
Order-of-Magnitude Analysis
f ( n ) represents the algorithm’s growth rate as a
function over input size n
Examples:
- If A requires time directly proportional to n , then f ( n ) = n … which we write O ( n )
- If A requires time proportional to n^2 , then f ( n ) = n^2 … which we write O ( n^2 ) CPSC 223, 2009 4
Order-of-Magnitude Analysis
Definition of the Order of an Algorithm (textbook):
Algorithm A is O ( f ( n )) if constants k and n 0 exist such
that A requires no more than k · f ( n ) time units to
solve a problem of size n ≥ n 0
Alternative definition of big O analysis [Cormen et al.]
For a given function g ( n ), we denote by O ( g ( n )) the set of functions O ( g ( n )) = { f ( n ) | there exist positive constants k and n 0 such that 0 ≤ f ( n ) ≤ k · g ( n ) for all n ≥ n 0 }
- Thus, big O gives an upper bound on a function f ( n ) to
within a constant factor
CPSC 223, 2009 7
Growth Rates
Fairly typical growth-
rate functions
to most expensive
CPSC 223, 2009 8 [Carrano, 2007]
Asymptotic Upper Bounds
Big O notation can sometimes be misleading
- We are considering asymptotic upper bounds Any algorithm that is O ( n ) is also O( n^2 ), since every function f ( n ) in O ( n ) is also in O ( n^2 ) … ( n ≤ k 1 · n ≤ k 2 · n^2 )
- Are we analyzing the algorithm or the problem? The algorithm is a particular computational approach for solving the problem … e.g., selection sort is O ( n^2 ) The problem is the general task … e.g., what is the smallest upper bound for the complexity of sorting CPSC 223, 2009 9
Binary Search
CPSC 223 -‐-‐ Fall 2010 10
Binary Search
- Given a sorted list, binary search finds an element in O
(log n ) time
- Recursively search for an element:
- Pick middle element of the list
- If middle element == key, then found match
- If middle element > key, search left half of list
- If middle element < key, search right half of the list CPSC 223, 2009 13
Binary Search (Review)
CPSC 223, 2009 14 Input: A B C D E F G Pick middle: A B C D E F G New input: A B C Pick middle: A B C New input: C Pick middle: C C < D Is C in the List? B < C Found It!
Average-Case Time Complexity
- Many sorting algorithms have O ( n^2 ) worst-case and
O ( n log n ) or O ( n ) best-case costs
- worst-case is the most “ pessimistic ” view
- … the algorithm on any problem won’t be slower
- Average-case
- An algorithm is O ( f ( n )) if the average amount of time it requires to solve a problem of size n is no more than k * f ( n ) for n > n 0
- Often hard to determine … e.g., what’s the average case? CPSC 223, 2009 15
Three Sorting Algorithms:
Selection, Bubble, and Insertion Sort
CPSC 223 -‐-‐ Fall 2010 16
Evaluating Sorting Algorithms
Number of comparisons
- How many pairs of items are compared within a pass
Number of moves
- How many times do we move list items
- Within a list or to a temporary variable
- We often do “ swaps ” void swap(int& x, int& y) // here just integers { int tmp = x; x = y; y = tmp; } CPSC 223, 2009 19 How many moves in a swap? 3
Selection Sort
What is the best and worst case?
- They are the same for selection sort!
What is the cost?
- The outer loop executes n – 1 times (passes)
- Each pass p requires n – p comparisons Σ p = 1 .. n – 1 ( n – p ) = ( n – 1) + ( n – 2) + … + 1 = n *( n – 1)/
- Thus, insertion sort is O ( n^2 )! CPSC 223, 2009 20 Check that we would get the same result if we analyzed moves instead of comparisons
Comparison of Sorting Algorithms
CPSC 223, 2009 21 SelecRon Sort Bubble Sort Best Case Average Case Worst Case O ( n^2 ) O ( n^2 ) O ( n^2 ) InserRon Sort Mergesort Quicksort Heapsort Treesort
Bubble Sort
The basic idea:
- Compare adjacent items …
- Exchange items if they are out of order
- Repeat for n – 1 passes
- Each pass p requires n – p comparisons CPSC 223, 2009 22 29 10 14 13 10 29 14 13 10 14 29 13 10 14 13 29 10 14 13 29 10 14 13 29 10 13 14 29 10 13 14 29 10 13 14 29 IniRal list: ( n =4) Pass 1 Pass 2^ Pass 3
Comparison of Sorting Algorithms
CPSC 223, 2009 25 SelecRon Sort Bubble Sort Best Case Average Case Worst Case O ( n^2 ) O ( n^2 ) O ( n^2 ) O ( n ) O ( n^2 ) O ( n^2 ) InserRon Sort Quicksort Heapsort Treesort Mergesort
Insertion Sort
The basic idea:
- Partition list into sorted and unsorted regions
- Select first item in unsorted region
- Insert item into the right location of sorted region
- Shift larger items one location forward in the list CPSC 223, 2009 26 29 10 14 13 29 10 14 13 10 29 14 13 10 29 14 13 10 29 14 13 10 14 29 13 10 14 29 13 10 14 29 13 IniRal list: ( n =4) Pass 1 Pass 2^ Pass 3 10 13 14 29
Insertion Sort (based on textbook)
void insertionSort(Entry theArray[], int n)
for(int i = 1; i < n; i++) {
Entry next = theArray[i];
int j = i; // insertion index
while(j > 0 && theArray[j-1] > next) {
theArray[j] = theArray[j-1]; // shift
j = j – 1;
} // end inner for
theArray[j] = next;
} // end outer for
CPSC 223, 2009 27
Insertion Sort
What is the best and worst case?
- Worst case is when inner loop has to shift ( n – 1) times
- … which happens when list is in reverse order
- Best case is when list is already in order
- … outer loop executes n – 1 times doing nothing in inner loop
What is the worst-case cost?
- The outer loop executes n – 1 times (passes)
- Each pass p requires p comparisons Σ p = 1 .. n – 1 p = 1 + 2 + … + ( n – 1) = n *( n – 1)/
- Each pass p also requires p data-item shifts
- Thus, insertion sort is O ( n^2 )! CPSC 223, 2009 28
