Introduction to Algorithms and Algorithmic Efficiency, Lecture notes of Algorithms and Programming

Download Introduction to Algorithms and Algorithmic Efficiency and more Lecture notes Algorithms and Programming in PDF only on Docsity!

1 http://xkcd.com/1185/

Introduction to Object-Oriented Programming

Algorithms

Christopher Simpkins

[email protected]

CLRS

The Sorting Problem

Sorting means rearranging the elements of an array (or permuting the

array) so that the elements are in a well-defined order. For example, we can sort an array in non-decreasing order (ascending) so that each

element A[i] ≤ A[i + 1 ] for i = 1... n − 1.

Note that we don’t define a separate output, only a result, or effect.

In today’s discussion we’ll assume that our input is an array whose elements are integers. We’ve already seen how to

generalize to arbitray user-defined types.

Insertion Sort

The insertion sort algorithm in pseudocode (from CLRS Chapter 2): 1 for j = 2 to A.length // A[1 .. A.length] is an array 2 key = A[j] 3 // Insert A[j] into the sorted sequence A[1 .. j - 1]. 4 i = j - 1 5 while i > 0 and A[i] > key 6 A[i + 1] = A[i] 7 i = i - 1 8 A[i + 1] = key

Note the following conventions we use when describing algorithms

abstractly:

we use pseudocode instead of code in a particular programming language,

the array indices start at 1 instead of 0, and

the array has a property length so that we don’t have to specify

the array’s size separately.

A Loop Invariant for Insertion Sort

1 for j = 2 to A.length 2 key = A[j] 3 // Insert A[j] into the sorted sequence A[1 .. j - 1]. 4 i = j - 1 5 while i > 0 and A[i] > key 6 A[i + 1] = A[i] 7 i = i - 1 8 A[i + 1] = key

At the start of each iteration of the for loop of lines 1-8, the

subarray A[1 .. j - 1] consists of the elements originally in A[

.. j - 1], but in sorted order.

Expresing Loop Invariants as assertions

Insertion sort in Java (note translation to 0-based indexes):

for (int j = 1; j < a.length; ++j) { assert isSorted(a, 0, j - 1); int key = a[j]; int i = j - 1; while(i >= 0 && a[i] > key) { a[i + 1] = a[i]; i = i - 1; } a[i + 1] = key; }

Note that we didn’t express the entire invariant in Java. We could, but

you must trade off implementation effort and benefit. Run the program with the -ea switch to enable assertions:

$ java -ea InsertionSort

See InsertionSort.java.

Linear Search

If we can make no assumptions about the order of the array, our only

option is linear search:

// A is an array, and v is the value we’re searching for LINEAR-SEARCH(A, v): for i = 1 to A.length if A[i] = v then return i return -

We’ll be dealing with search algorithms abstractly for the purpose of discussing running time analysis, but example implementations are

available in Search.java.

Algorithmic Efficiency

We can characterize algorithmic efficiency in terms of space

complexity (how much storage an algorithm requires) or time complexity (how “fast” an algorithm runs). Almost always primarily

concerned with time complexity.

Note that we want to eliminate platform-specific factors, like speed

of the particular computer an algorithm runs on.

So we characterize algorithm performance in terms of

input size, n, and

order of growth as a function of input size.

An efficient algorithm beats an inefficient one even if the inefficient

algortithm is run on a far superior computer.

Binary Search

If the array is sorted you can use a binary search: BINARY-SEARCH(A, v): p := 1, r := A.length while p ≤ r q := b(p + r )/ 2 c if A[q] = v then return q if A[q] > v then r := q - 1 else p = q + 1 return -

Intuitively: We check the midpoint of the array (q).

If the array is empty (p > r), the query value was not found.

If the midpoint holds the value, return the midpoint. If the midpoint holds a value greater than our search value, repeat

the process with the lower half of the array. If the midpoint holds a value less than our search value, repeat

the process with the upper half of the array.

Efficiency of Binary Search

The key to analyzing the efficiency of BINARY-SEARCH is realizing that the array is halved in each iteration of the while loop.

In the worst case BINARY-SEARCH runs until the size of the array (r − p) goes from n to 1 by successive halving.

This is equivalent to going from 1 to n by successive doubling.

Counting the number of times x we need to double to get from 1 to n is

2

x = n

so

x = lg n

and the worst-case running time of BINARY-SEARCH is

O(lg n)

Run-time Analysis Summary: Linear

Sample pseudocode:

for i := 1 to n // ...

Java implementation:

for (int i = 0; i < n; i++) { // ... }

Intuition:

“For each of the n elements,” or “for n times.”

Big-O:

O(n)

Run-time Analysis Summary: Quadratic

Sample pseudocode:

for i := 1 to n for j := 1 to n

Java implementation:

for (int i = 0; i < n; i++) { for (int j = 0; j < n; j++) { // ... } }

Intuition:

“n times for each n”

Big-O:

O(n

)

Run-time Analysis Summary: Logarithmic (1 of 2)

Sample pseudocode: p := 1, r := n while p ≤ r // cut difference between p and r in half until it’s 0...

Java implementation: int lo = 0, hi = array.length - 1; while (lo <= hi) { int middle = (lo + hi)/2; // either lo becomes middle + 1, or hi becomes middle - 1 }

Intuition: Each iteration of the loop cuts the remaining input in half. We stop when we’ve cut the input size to 1.

Mathematically, we’re multiplying the input size by

2 each time. Run-time is the number of times we multiply by (^12)

Mathematically ...

Run-time Analysis Summary: Logarithmic (2 of 2)

p := 1, r := n while p ≤ r // cut difference between p and r in half until it’s 0...

Run-time is the number of times x we multiply by 12 :

1

2 x^

n = 1

1

2 x^

=

1

n

2

x = n

x = log 2 n

So Big-O is: O(log n)