Download Backtracking Algorithms: Solving Complex Problems through Recursive Search and more Study notes Computer Science in PDF only on Docsity!
COP 3503 – Computer Science II – CLASS NOTES - DAY # Backtracking Algorithms Backtracking algorithms use recursion to try all possible solutions and pick the optimal solution from the set of all solutions. This is a common strategy employed in AI games, parsers, etc. Since trying all possible solutions is essentially an exhaustive searching technique there must be a technique which will allow a backtracking algorithm to “know” that some solution paths are not optimal and therefore need not be explored. This technique is known as search tree pruning and we will examine it later. Many problems can be viewed in terms of abstract graphs. For example, the nodes in a graph can represent the positions in a chess game and the edges represent the legal moves. Often the original problem translates to searching for a specific node, path, or pattern in the associated graph. If the graph contains a large number of nodes, and particularly if it is infinite, it may be wasteful or infeasible to explicitly build it in memory before applying a search technique to the graph. In such cases an implicit graph is required. An implicit graph (recall that a tree is just a special case of a graph) is one for which a description of its nodes and edges is available, so that only the relevant parts of the graph are constructed as the search progresses. Computing time is reduced whenever the search succeeds before the entire graph has been constructed. A savings in terms of memory is also achieved with implict graphs and can be further enhanced whenever nodes that have been previously searched can be discarded thereby making room for nodes to be subsequently explored (this is not always possible however). If the graph is infinite, then the implicit graph technique offers the only hope of exploring all of the graph. In its basic form, backtracking resembles a depth-first search in a digraph (directed graph). The digraph concerned is typically a tree (or at least it contains no cycles). The digraph is assumed to be implicit regardless of its structure. The aim of the search is to find solutions to the problem represented by the digraph. This is done by building partial solutions as the search proceeds; such partial solutions limit the regions in which a complete solution may be found. Generally speaking, when a search begins, nothing is known about the solutions to the problem. Each move along an edge of the implicit graph corresponds to adding a new element to a partial solution, i.e., it narrows down the remaining possibilities for a complete solution. The search is successful if, proceeding in this way, a solution can be completely defined. In this case the algorithm may stop (if there only one solution to the problem is required) or continue looking for alternative solutions (if you would like to see all of the solutions). On the other hand, the search is unsuccessful if at some stage the partial solution constructed so far cannot be completed. In this case, the search backs-up (exactly like a depth-first tree search), removing as it goes, the elements that were added to the partial solution at each stage. When it returns
to a node which has one or more unexplored options (neighboring nodes), the search for a solution resumes. Example 1 - The Knapsack Problem A classic problem in computer science is the knapsack problem which is stated as: Suppose you are given a knapsack and n different types of objects (assume that an adequate number of objects of each type are available). For i = 1, 2, …, n, and object of type i has a positive weight wi and a positive value vi. The knapsack can carry a weight not exceeding W. The object is to fill the knapsack in a way that maximizes the value of the included objects, while respecting the capacity constraint. An object may be either selected or left behind but you cannot take a fraction of an object. As a specific instance, suppose we have the following knapsack problem. There are four types of objects, whose weights are respectively 2, 3, 4, and 5 units, and whose values are 3, 5, 6, and 10. The knapsack can carry a maximum of 8 units. Numbers to left of sem-colon are the weights of the selected objects. Number to the right is the current value of the knapsack. Implicit Tree for Knapsack Problem Moving down from a node in the tree to one of its children corresponds to deciding which kind of object to put into the knapsack next. Without loss of generality – you can decide 2,2,2,2: 2,2,2: : 2:3 3:5 4:6 5: 2,2:6 2,3:8 2,4:9 2,5:13 3,3:10 3,4:11 3,5:15 4,4: 2,2,3: 1 2,2,4:12 2,3,3:
another piece that lies on the same row, the same column, or the same diagonal (in either direction) as the queen. A chessboard is a matrix of eight rows and eight columns. If you attempt to do this problem, it is by no means a trivial undertaking. The physicist/mathematican C.F. Gauss attempted to produce a complete solution to this problem in the 1850s as was unable to do so. Solutions to this problem do in fact exist, it is not a problem which has no solution. Two solutions to this problem are shown below. Two solutions to the Eight Queens Problem In order to see how the solution to such a problem can be accomplished, let’s look at a smaller, simpler version of the same problem, the Four Queens Problem. The Four Queens Problem The problem is basically the same as for the Eight Queens problem except now we need to place four queens onto a smaller 4 x 4 matrix (board) with the same requirements about their inability to capture each other. The nature of the problem and the number of pieces involved means that at most one queen can be placed somewhere in each row and in each column (the rules of chess would prevent any two queens from being in either the same row or column – or immediate capture would result). The figures below will illustrate the various steps that our algorithm will take in solving this problem. Starting with the topmost row of the board, we will place a queen in the leftmost position. [Using a row major notation, this queen is placed in board position (1,1).] Since we know that this queen must occupy one of the column positions in this row, we have simply made a choice and there remain three other possible choices for where the queen in row 1 might be placed (i.e., it could still wind up in positions (1,2), (1,3), or (1,4)).
The remaining possible choices are marked with “?” in the figure below. Before these other possible choices are tried, we move to the next row an attempt to place the second queen somewhere in this row without violating the rules of the game. Notice that the placement of the first queen piece into position (1,1) has narrowed the choices where the second queen might be positioned. Positions (2,1) and (2,2) are guarded by the queen at (1,1) and are therefore not available as positions for the second queen. These unavailable positions are indicated by the “x” in the position. Positions (2,3) and (2,4) are both available, so we will place the second queen into position (2,3) in keeping with our policy of placing the queen as far to the left in the row as possible. Once, again before position (2,4) is tried for the second queen piece, we move on to the third queen piece and attempt to place it. However, when attempting to place the third queen onto the matrix, we discover that every position in the third row (positions (3,1), (3,2), (3,3), and (3,4)) are guarded by one of the two queens already placed onto the matrix. Thus, we have reached a dead-end in attempting to find a solution to the problem. This scenario is shown in the following diagram. Figure showing positioning of queens when attempting to place third queen onto the board When the dead-end is reached while attempting to position the third queen, the algorithm must backtrack to the point at which the last decision was made (i.e., the point at which the last queen was positioned onto the board and there existed an option as to where to place the queen) and try another alternative. This is shown in the next figure, in which the first queen’s position remains unchanged, but the second queen’s position moves from position (2,3) to position (2,4). This represents the last point at which there was a choice in the possible positions in which a queen could be placed. We initially chose the leftmost available position within the row, but that choice has lead to a dead-end and we must now try the next alternative that is available. In this case, there is only one more possibility for the position of the second queen, so this position is now attempted (position (2,4)). Notice that the position that was first attempted for the second queen is now marked with an “x” since it has proven to lead to a dead-end and no longer represents a possible solution to the problem. Now when considering the possible positions in which the third queen can be placed, we discover that position (3,1) is guarded by the first queen, and positions (3,3) and (3,4) are guarded by the second queen. This narrows the possible positions in which the third queen can be positioned to only the
first queen into position (1,3) and would result in the board shown in the figure below, which is also a solution to the problem. Figure showing a second solution to the Four Queens Problem Finally, the exhaustive search for possible solutions will bring us to the point of positioning the first queen into position (1,4). As was the case when the first queen was positioned in position (1,1), there will be no solution. In fact, the solutions with the first queen positioned in either (1,3) or (1,4) are just the mirror images of those when the first queen was in position (1,1) and (1,2) respectively. If you do a left-right reflection of the board when the first queen was in position (1,2), you will obtain the board shown above when the first queen is in position (1,3). The same will hold true of the left-right reflection of the board when the first queen was in position (1,1) and when the first queen was in position (1,4). This is illustrated in the figure below. Place a mirror on the right hand side of the board on the left side of the figure and looking into the mirror you will see the board which is on the right hand side of the figure. Try it! Place mirror here
Figure illustrating the mirror image nature of the solutions to the Four Queens Problem Regardless of the search technique employed, it remains that for games such as chess a complete search of the associated graph is not feasible. In situations such as this, you must be content with a partial search around the current position. This is the basic principle which underlies an important heuristic called the minimax principle. Although this heuristic does not allow one to be certain of winning whenever it is possible to do so, it finds a move that may reasonably be expected to be among the best moves available. More importantly does so while exploring only part of the graph starting from some given position. Exploration of the graph is normally stopped before the terminal positions are reached, using one of several possible criteria, and the positions where the exploration stopped are evaluated heuristically. In a sense, this is merely a systematic version of the method used by some human players that consists of looking ahead a small number of moves. player rule A max 10 B min -3 10 - A max B eval -7 5 -3 10 -20 0 –5 10 -15 20 1 6 -8 14 -30 0 -8 - The Minimax Principle Consider the game of tic-tac-toe. This is an adversarial game of two players. Using the minimax strategy with the additional assumption that both players utilize optimal play. The state tree shown below illustrates this best. Day 10 - 8
max mi n
evaluation leads to a draw refutation In the tree of the previous page- notice the subtree with the blue circles is unevaluated – it has been pruned. The reason that it can be pruned away is the basis of the minimax tree. Notice that the node with the? is a max node. Its parent (the yellow node) is a min node and its max node parent has already found a sequence that guarantees a value of 44. There is nothing that evaluating the subtree with root? will do toward changing this value – so why evaluate it? We don’t, and this is called alpha pruning this node is unimportant In practice alpha-beta pruning limits the searching to only O(√N) nodes where N is the number of nodes in the full tree. [Note that this means in a practical sense that alpha-beta pruned trees can search twice as deep for the same cost.] ???
= 44 44
X X
X O
O O
X X X
X O
O O
X X
X X O
O O
X X
X O
O X O
X O X
X X O
O O
X X
X X O
O O O
X O X
X O
O X O
X X
O X O
O X O
X O X
X X O
O X O
X O X
X X O
O X O
X X X
O X O
O X O
X wins O wins draw
LEVEL
X moves LEVEL 7 O moves LEVEL 8 X moves
-1 = X loses 0 = draw 1 = X wins LEVEL 9 Game Ends
LEVEL 0
Game Begins Tic-Tac-Toe game Tree (Minimax Tree) – Partial Solution
