Algorithm Implementation - Introduction to Computer Science - Lecture Slides, Slides of Computer Science

Download Algorithm Implementation - Introduction to Computer Science - Lecture Slides and more Slides Computer Science in PDF only on Docsity!

Algorithm Implementation

• These notes are intended for use by students in CS1501 at the University of Pittsburgh and no one else
• These notes are provided free of charge and may not be sold in any shape or form
• These notes are NOT a substitute for material covered during course lectures. If you miss a lecture, you should definitely obtain both these notes and notes written by a student who attended the lecture.
• Material from these notes is obtained from various sources, including, but not limited to, the following: - Algorithms in C++ by Robert Sedgewick - Introduction to Algorithms, by Cormen, Leiserson and Rivest - Various Java and C++ textbooks

Goals of Course

• Goals of this course:

1) To learn how to convert (nontrivial) algorithms

into programs

 Often what seems like a fairly simple algorithm is not

so simple when converted into a program

 Other algorithms are complex to begin with, and the

conversion must be carefully considered

 Many issues must be dealt with during the

implementation process

 Let's hear some

Goals of Course

2) To see and understand differences in

algorithms and how they affect the run-times

of the associated programs

• Many problems can be solved in more than one way
• Different solutions can be compared using many

factors

• One important factor is program run-time » Sometimes a better run-time makes one algorithm more desirable than another » Sometimes a better run-time makes a problem solution feasible where it was not feasible before
• However, there are other factors for comparing algorithms » Let's hear some

Algorithm Analysis

• Big O
  • Upper bound on the asymptotic performance
• Big Omega
  • Lower bound on the asymptotic performance
• Theta
  • Upper and lower bound on the asymptotic performance – Exact bound
• Compare on the board
• So why would we ever use Big O?
• Theta is harder to show in some cases
• Lower bounds are typically harder to show than upper

bounds

• So sometimes we can just determine the upper bound

Algorithm Analysis

• So is algorithm analysis really important?

– Yes! Different algorithms can have considerably

different run-times

• Sometimes the differences are subtle but sometimes

they are extreme

• Let's look at a table of growth rates on the board
  • Note how drastic some of the differences are for large problem sizes

Exhaustive Search

• Idea:

– We find a solution to a problem by considering

(possibly) all potential solutions and selecting

the correct one

• Run-time:

– The run-time is bounded by the number of

possible solutions to the problem

– If the number of potential solutions is

exponential, then the run-time will be

exponential

Exhaustive Search

• Example: Hamiltonian Cycle

• A Hamiltonian Cycle (HC) in a graph is a cycle that visits each node in the graph exactly one time - See example on board and on next slide
• Note that an HC is a permutation of the nodes in the graph (with a final edge back to the starting vertex) - Thus a fairly simple exhaustive search algorithm could

be created to try all permutations of the vertices,

checking each with the actual edges in the graph

• See text Chapter 44 for more details

Exhaustive Search

• Unfortunately, for N vertices, there are N!

permutations, so the run-time here is poor

• Pruning and Branch and Bound

– How can we improve our exhaustive search

algorithms?

• Think of the execution of our algorithm as a tree
  • Each path from the root to a leaf is an attempt at a solution
  • The exhaustive search algorithm may try every path
• We'd like to eliminate some (many) of these

execution paths if possible

• If we can prune an entire subtree of solutions, we can greatly improve our algorithm runtime

Pruning and Branch and Bound

• For example on board:
  • Since edge (C, D) does not exist we know that no paths with (C, D) in them should be tried » If we start from A, this prunes a large subtree from the tree and improves our runtime
  • Same for edge (A, E) and others too

– Important note:

• Pruning / Branch and Bound does NOT improve the

algorithm asymptotically

• The worst case is still exponential in its run-time
• However, it makes the algorithm practically solvable for

much larger values of N

Recursion and Backtracking

• Idea of backtracking:

– Proceed forward to a solution until it becomes

apparent that no solution can be achieved

along the current path

• At that point UNDO the solution (backtrack) to a

point where we can again proceed forward

– Example: 8 Queens Problem

• How can I place 8 queens on a chessboard such that

no queen can take any other in the next move?

• Recall that queens can move horizontally, vertically or diagonally for multiple spaces
• See on board

8 Queens Problem

– 8 Queens Exhaustive Search Solution:

• Try placing the queens on the board in every

combination of 8 spots until we have found a solution

• This solution will have an incredibly bad run-time
• 64 C 8 = (64!)/[(8!)(64-8)!] = (64636261605958*57)/ = 4,426,165,368 combinations (multiply by 8 for total queen placements)
• However, we can improve this solution by realizing that

many possibilities should not even be tried, since no

solution is possible

• Ex: Any solution has exactly one queen in each column

8 Queens Problem

• Ex: No queens on the same diagonal
• See example on board
• Using this approach we come up with the solution as

shown in 8-Queens handout

• JRQueens.java
• Idea of solution:
• Each recursive call attempts to place a queen in a specific column » A loop is used, since there are 8 squares in the column
• For a given call, the state of the board from previous placements is known (i.e. where are the other queens?)
• If a placement within the column does not lead to a solution, the queen is removed and moved "down" the column

8 Queens Problem

• When all rows in a column have been tried, the call terminates and backtracks to the previous call (in the previous column)
• If a queen cannot be placed into column i, do not even try to place one onto column i+1 – rather, backtrack to column i- and move the queen that had been placed there
• Using this approach we can reduce the number of potential solutions even more
• See handout results for details