Download Pseudocode Algorithms: Understanding Complexity and Comparisons in Data Structures - Prof. and more Study notes Discrete Structures and Graph Theory in PDF only on Docsity!
EECS 210 Fall 2007 Algorithm analysis notes
Most of the material below was discussed in the class lectures. I’ve expanded a bit on some of
the examples.
An algorithm is a finite set of precise instructions for performing a computation or solving a
problem. An algorithm should have the following characteristics:
input-- Input values from a specified set
output— output values produced by an input set. The values are the solution to the problem
definiteness —steps are precisely defined
correctness —will return the correct output for each set of input values
finiteness —output must be produced in a finite number of steps
generality —will handle any legitimate data set without error
The main thing to remember about a pseudocode algorithm is that it must be general enough
to be translated into an arbitrary programming language. Put another way, the steps are
English-type statements rather than “code.” An algorithm should contain little, if any, language
specific syntax.
Indentation and/or begin-end blocks are used to indicate nesting or grouping. Loops will be
terminated with an end statement if necessary for clarity.
comments—enclosed in braces ( { } ) or begin with //
comparison operators: =, <, , >, , (Notice the use of mathematical notation e.g. not!
=))
assignment operator: (You’ll often see :=) in other versions of pseudocode.)
loops
while ( Boolean condition or statement ) do —test at top
examples: while (there are vertices left to examine) do
while (A < 10) do
for index start to finish do (There may be an optional step (increment size))
examples: for i 1 to 10 do
for i 1 to 30 step 3 do
repeat
statements to execute
until ( Boolean condition ) always executes at least once
example: repeat
examine next list item
until no items remain
if statements
if ( Boolean condition ) then
statements to execute
endif
if ( Boolean condition ) then
statements to execute
elseif ( Boolean condition )
statements to execute
endif
array or list elements: A[i, j] or aij
Complexity of an algorithm
The complexity of an algorithm is expressed in terms of the input size (usually n). For
example, sorting a list of 100 integers (n =) 100) will take much less time than sorting a list of
1,000,000 integers (n =) 1,000,000). We want an expression in terms of n that will describe the
amount of time required to run the algorithm on an input of that size. That way, given a value
for n, we can “predict” how long it will take to produce an answer.
The complexity of an algorithm is usually expressed by one of three measures—the best case,
the worst case, or the average case. The best case is the least amount of time required for
any input of size n. (The best case is NOT when n =) 1.) Typically this occurs when the input is
a “special case”. For example, an already sorted list is often the “best” input for a sorting
algorithm. Similarly, the worst case complexity is an expression that describes the amount of
time needed for the “worst” possible input of size n. For instance, for some sorting algorithms
the “worst” case is a list that is sorted in reverse order. You can think of the worst case
complexity as a “guarantee” in that you know no matter what the data set looks like, the
algorithm will never perform worse than the stated worst case complexity. The average case
complexity is just what the name implies—if the algorithm were run on many different data
sets, averaging the times taken for all the data sets would give you a good idea of how long
the algorithm would usually take for a data set of size n.
Now, let’s look at some algorithms and discuss how we might determine their best, worst or
average case complexities.
Example 1: Finding the largest element of a finite sequence
max(a 1 , a 2 , …, an distinct integers)
max a 1 // initialize max to the first element of the list
for i 2 to n
if max < ai then max ai
return(max)
If you also want to find the location of the largest value, the algorithm below should be used.
max(a 1 , a 2 , …, an: integers)
max a 1 // initialize max to the first element of the list
location 1 // remember the location of max
for i 2 to n
if max < ai then list item comparison
max ai
location i
return(location)
max1 ai there is another comparison in the else
else if (^) ai > max 2 then max 2 ai statement
return(max1, max2)
Thus, in the worst case we have 2n – 3 comparisons and 2n – 2 assignments. Notice that
even though big2 and big1 both have complexity O(n), big2 will run faster because the
coefficient is smaller.
(Note: it is not possible to have both the worst case number of assignments and the worst
case number of comparisons for the same list. Just to keep things simple, think of calculating
the two quantities separately, and taking the worst case value for each.)
We now turn to the searching problem—given a list of n numbers (a 1 , a 2 , …, an) and a value x,
return the location of x if it is on the list and otherwise return 0. This is a linear or sequential
search. Notice that the algorithm below could describe either searching an array or searching
a linked list. If the data is in a linked list a pointer to the node contain x is returned, and a nil
pointer is returned if x is not on the list.
Example 3: linear search
linearsearch(x : int, a 1 , a 2 , …, an distinct int)
i 1 1 assignment
while (i n and x a i
) 2 comparisons each time
i i + 1 1 assign each time
if i n then location i 1 comparison and 1 assignment
else location 0
Best Case: x is in the first position. Two comparisons are needed in the while loop and one in
the if statement for a total of three comparisons.
Worst Case: If x is not on the list, 2n + 2 comparisons are required:
2n for the n executions of the while loop
1 to drop out of the while loop
1 in the if statement
The worst case for a successful search is 2n + 1 comparisons.
A question that naturally arises at this point is whether it is possible to do better than this. The
answer is yes, if the list we are given is sorted. In that case we employ the following
approach: compare x (the value we’re looking for) with the element in the middle of the list. If
it is greater than the value in the middle, search the last half of the list. Otherwise, search the
first half of the list. This is known as a binary search. This is the first example we have of a
divide and conquer algorithm.
Example 4: Binary search
binarysearch( x: int, a 1 , a 2 , …, an, increasing int.)
i 1 \ left endpoint of search interval 1 assignment
j n \ right endpoint of search interval 1 assignment
while i < j 1 comparison
m (i+j)/2
if x > am then i m + 1 1 comparison and 1 assignment
else j m
end
if x = ai then location i 1 comparison and 1 assignment
else location 0
You should go through the algorithm to convince yourself that it will terminate. Also, notice
that we do not explicitly check if am =) x. Instead, when the size of the list is reduced to one
item, then we check if the item being sought is in that position.
Now, let’s look at the complexity of binary search. We're going to count comparisons between
x and elements of the list. Let n =) 2
k
. (Note how this is not a problem because we're doing a
worst case analysis and if the estimate is a little too high that's not a problem.)
|—————————|—————————| There are k iterations of the while loop since
i m j we can only divide the list in half k times. Thus,
2k comparisons are needed.
|————|—————| 1 more comparison is needed to drop out of
i m j the loop and another is needed in the if test
at the bottom. This gives us a total of 2k + 2
|—————| comparisons.
i j
Notice there is no best or worst case since
- every list of length n will take the same number
- of comparisons
2
1 If n =) 2
k then k =) lg n. Thus the complexity is:
2
0 =) 1 element 2 lg n + 2 =) O(lg n)
We now look at two different methods that can be used to find the largest and smallest
elements of a list. The second method is another divide and conquer algorithm. In this case
we’ll count the total number of steps
Example 5: Largest and smallest list elements (version 1)
minmax(a 1 , a 2 , …, an)
// Find the largest and smallest elements of a list.
min a (^1) 1 assignment
max a 1 1 assignment
for i 2 to n do n - 1 iterations
if min >a i
then min (^) a i
2(n -1) =) 2n – 2 steps
Let’s solve this by back substitution.
f(n) =) 2f(n/2) + 2 First note that f(n/2) =) 2f(n/4) + 2 so substituting in we get
f(n) =) 2[2f(n/4) + 2] + 2 =) 2
2 f(n/
2 ) + 2
2
k
2 [2f(n/
3 ) + 2]+ 2
2
- 2 Then, the basis case is n/(
k- ) =) 2
k /
k- =) 2
=) 2
3 f(n/
3 ) + 2
3
2
- 2 we divide the list k – 1 times until the list has
… length 2
=) 2
k- f(2) + 2
k-
k-
2
Now, using the fact that f(2) =) 1, we get
k-
f(n) =) 2
k-
i =) 2
k-
k
- 2 (Remember the sum starts at i =) 0 not i =) 1) i=)
k-
k-
k-
This method is faster than the first version above, and in fact it can be shown that this
algorithm is optimal in that no other algorithm for the problem uses fewer comparisons.
Sorting
Another operation that we often look at is sorting a list into numerical or alphabetical order.
There are many different ways this can be done, some of which are much more efficient than
others. In sorting algorithms, both comparisons and swaps of list items are counted in the
analysis. The algorithms we’ll look at sort the list into nondecreasing order (i.e. increasing
order, except there may be duplicates.) Let’s begin with selection sort.
Example 7: Selection Sort
select_sort (A : data_array; n : integer)
\ Sort the list into increasing order using the following method:
\ Select the smallest element among ai, ..., an, and
\ swap with ai, continuing until the list is sorted.
(1) for i 1 to n - 1 do Loop executes n – 1 times
(2) low i;
(3) for j i + 1 to n do Number of comparisons:
(4) if aj < alow then n-1, then n-2, …, 1
(5) low j
(6) swap(ai, alow) n-1 swaps, one for each value of i
Let’s begin with the complexity of the swap. A swap (line 6) requires three assignments. Line
4 contains comparisons of list items. Note that the number of comparisons remains the same
no matter what the order of the list is originally. The number of comparisons can be expressed
by the sum
n-
i =) (n-1)n/2 =) O(n
2 )
i=)
Looking at the best, worst and average case complexities we have:
swaps list item comparisons
Best case n – 1 n(n – 1)/
Worst case n – 1 n(n – 1)/
Average case n – 1 n(n – 1)/
This is O(n
2 ) and also (n
2 )
The only real difference between the best and worst cases is the number of times the
assignment in line 5 is made. Since we’re only looking at the number of swaps and the
number of list item comparisons, all three cases are exactly the same.
We now look at the insertion sort.
Example 8: Insertion Sort
insert_sort (A : data; n : integer)
// A[0] contains a sentinel that is less than all list elements
// At the beginning of the for loop the first i – 1 list items are in sorted order.
// Then, the item in position i is moved forward until its proper place is found
(1) for i 2 to n do n-1 iterations
(2) j i; tmp A[i] 2 assignments
(3) while (tmp < A[j-1]) do up to i list item comparisons
(4) A[j] A[j-1] 1 assignment
(5) j j - 1 1 assignment
(6) A[j] tmp 1 assignment
How the algorithm works—The initial part of the list is sorted. Beginning with the first element
in the unsorted part of the array, move that element forward until it’s proper position is found.
In line 3, for each value of i, at most i comparisons are made giving the total number of
comparisons:
n
i =) n(n+1)/2 – 1 (The 1 is subtracted because the sum starts with i =) 2, not i =) 1.) i=)
Best case—list is sorted in correct order. In this case the test in the line 3 always fails. Thus,
the total number of comparisions is n - 1 and there is no movement of data.
Worst case—list is sorted in reverse order. In this case, as noted above, there are n(n+1)/2 – 1
comparisons. In line 4, i – 1 list items will be moved for each value of i, so there are O(n
2 )
assignments as well.
The bound is tight since there exists an input that requires this many comparisons and
movements.
