The Algorithm Design Manual, Study notes of Design

Download The Algorithm Design Manual and more Study notes Design in PDF only on Docsity!

Steven S. Skiena

The Algorithm Design Manual

Second Edition

Contents

• I Practical Algorithm Design
• 1 Introduction to Algorithm Design - 1.1 Robot Tour Optimization - 1.2 Selecting the Right Jobs - 1.3 Reasoning about Correctness - 1.4 Modeling the Problem - 1.5 About the War Stories - 1.6 War Story: Psychic Modeling - 1.7 Exercises
• 2 Algorithm Analysis - 2.1 The RAM Model of Computation - 2.2 The Big Oh Notation - 2.3 Growth Rates and Dominance Relations - 2.4 Working with the Big Oh - 2.5 Reasoning About Efficiency - 2.6 Logarithms and Their Applications - 2.7 Properties of Logarithms - 2.8 War Story: Mystery of the Pyramids - 2.9 Advanced Analysis (*) - 2.10 Exercises
• 3 Data Structures - 3.1 Contiguous vs. Linked Data Structures
  • 3.2 Stacks and Queues xii C O N T E N T S
  • 3.3 Dictionaries
  • 3.4 Binary Search Trees
  • 3.5 Priority Queues
  • 3.6 War Story: Stripping Triangulations
  • 3.7 Hashing and Strings
  • 3.8 Specialized Data Structures
  • 3.9 War Story: String ’em Up
  • 3.10 Exercises
• 4 Sorting and Searching
  • 4.1 Applications of Sorting
  • 4.2 Pragmatics of Sorting
  • 4.3 Heapsort: Fast Sorting via Data Structures
  • 4.4 War Story: Give me a Ticket on an Airplane
  • 4.5 Mergesort: Sorting by Divide-and-Conquer
  • 4.6 Quicksort: Sorting by Randomization
  • 4.7 Distribution Sort: Sorting via Bucketing
  • 4.8 War Story: Skiena for the Defense
  • 4.9 Binary Search and Related Algorithms
  • 4.10 Divide-and-Conquer
  • 4.11 Exercises
• 5 Graph Traversal
  • 5.1 Flavors of Graphs
  • 5.2 Data Structures for Graphs
  • 5.3 War Story: I was a Victim of Moore’s Law
  • 5.4 War Story: Getting the Graph
  • 5.5 Traversing a Graph
  • 5.6 Breadth-First Search
  • 5.7 Applications of Breadth-First Search
  • 5.8 Depth-First Search
  • 5.9 Applications of Depth-First Search
  • 5.10 Depth-First Search on Directed Graphs
  • 5.11 Exercises
• 6 Weighted Graph Algorithms
  • 6.1 Minimum Spanning Trees
  • 6.2 War Story: Nothing but Nets
  • 6.3 Shortest Paths
  • 6.4 War Story: Dialing for Documents
  • 6.5 Network Flows and Bipartite Matching
  • 6.6 Design Graphs, Not Algorithms
  • 6.7 Exercises
• 7 Combinatorial Search and Heuristic Methods C O N T E N T S xiii - 7.1 Backtracking - 7.2 Search Pruning - 7.3 Sudoku - 7.4 War Story: Covering Chessboards - 7.5 Heuristic Search Methods - 7.6 War Story: Only it is Not a Radio - 7.7 War Story: Annealing Arrays - 7.8 Other Heuristic Search Methods - 7.9 Parallel Algorithms - 7.10 War Story: Going Nowhere Fast - 7.11 Exercises
• 8 Dynamic Programming - 8.1 Caching vs. Computation - 8.2 Approximate String Matching - 8.3 Longest Increasing Sequence - 8.4 War Story: Evolution of the Lobster - 8.5 The Partition Problem - 8.6 Parsing Context-Free Grammars - 8.7 Limitations of Dynamic Programming: TSP - 8.8 War Story: What’s Past is Prolog - 8.9 War Story: Text Compression for Bar Codes - 8.10 Exercises
• 9 Intractable Problems and Approximation Algorithms - 9.1 Problems and Reductions - 9.2 Reductions for Algorithms - 9.3 Elementary Hardness Reductions - 9.4 Satisfiability - 9.5 Creative Reductions - 9.6 The Art of Proving Hardness - 9.7 War Story: Hard Against the Clock - 9.8 War Story: And Then I Failed - 9.9 P vs. NP - 9.10 Dealing with NP-complete Problems - 9.11 Exercises
• 10 How to Design Algorithms
  • II The Hitchhiker’s Guide to Algorithms
• 11 A Catalog of Algorithmic Problems
• 12 Data Structures xiv C O N T E N T S
  • 12.1 Dictionaries
  • 12.2 Priority Queues
  • 12.3 Suffix Trees and Arrays
  • 12.4 Graph Data Structures
  • 12.5 Set Data Structures
  • 12.6 Kd-Trees
• 13 Numerical Problems
  • 13.1 Solving Linear Equations
  • 13.2 Bandwidth Reduction
  • 13.3 Matrix Multiplication
  • 13.4 Determinants and Permanents
  • 13.5 Constrained and Unconstrained Optimization
  • 13.6 Linear Programming
  • 13.7 Random Number Generation
  • 13.8 Factoring and Primality Testing
  • 13.9 Arbitrary-Precision Arithmetic
  • 13.10 Knapsack Problem
  • 13.11 Discrete Fourier Transform
• 14 Combinatorial Problems
  • 14.1 Sorting
  • 14.2 Searching
  • 14.3 Median and Selection
  • 14.4 Generating Permutations
  • 14.5 Generating Subsets
  • 14.6 Generating Partitions
  • 14.7 Generating Graphs
  • 14.8 Calendrical Calculations
  • 14.9 Job Scheduling
  • 14.10 Satisfiability
• 15 Graph Problems: Polynomial-Time
  • 15.1 Connected Components
  • 15.2 Topological Sorting
  • 15.3 Minimum Spanning Tree
  • 15.4 Shortest Path
  • 15.5 Transitive Closure and Reduction
  • 15.6 Matching
  • 15.7 Eulerian Cycle/Chinese Postman
  • 15.8 Edge and Vertex Connectivity
  • 15.9 Network Flow
  • 15.10 Drawing Graphs Nicely
  • 15.11 Drawing Trees C O N T E N T S xv
  • 15.12 Planarity Detection and Embedding
• 16 Graph Problems: Hard Problems
  • 16.1 Clique
  • 16.2 Independent Set
  • 16.3 Vertex Cover
  • 16.4 Traveling Salesman Problem
  • 16.5 Hamiltonian Cycle
  • 16.6 Graph Partition
  • 16.7 Vertex Coloring
  • 16.8 Edge Coloring
  • 16.9 Graph Isomorphism
  • 16.10 Steiner Tree
  • 16.11 Feedback Edge/Vertex Set
• 17 Computational Geometry
  • 17.1 Robust Geometric Primitives
  • 17.2 Convex Hull
  • 17.3 Triangulation
  • 17.4 Voronoi Diagrams
  • 17.5 Nearest Neighbor Search
  • 17.6 Range Search
  • 17.7 Point Location
  • 17.8 Intersection Detection
  • 17.9 Bin Packing
  • 17.10 Medial-Axis Transform
  • 17.11 Polygon Partitioning
  • 17.12 Simplifying Polygons
  • 17.13 Shape Similarity
  • 17.14 Motion Planning
  • 17.15 Maintaining Line Arrangements
  • 17.16 Minkowski Sum
• 18 Set and String Problems
  • 18.1 Set Cover
  • 18.2 Set Packing
  • 18.3 String Matching
  • 18.4 Approximate String Matching
  • 18.5 Text Compression
  • 18.6 Cryptography
  • 18.7 Finite State Machine Minimization
  • 18.8 Longest Common Substring/Subsequence
  • 18.9 Shortest Common Superstring
• 19 Algorithmic Resources xvi C O N T E N T S - 19.1 Software Systems - 19.2 Data Sources - 19.3 Online Bibliographic Resources - 19.4 Professional Consulting Services
  • Bibliography
  • Index

1

Introduction to Algorithm Design

What is an algorithm? An algorithm is a procedure to accomplish a specific task. An algorithm is the idea behind any reasonable computer program. To be interesting, an algorithm must solve a general, well-specified problem. An algorithmic problem is specified by describing the complete set of instances it must work on and of its output after running on one of these instances. This distinction, between a problem and an instance of a problem, is fundamental. For example, the algorithmic problem known as sorting is defined as follows:

Problem: Sorting Input: A sequence of n keys a 1 ,... , an. Output: The permutation (reordering) of the input sequence such that a′ 1 ≤ a′ 2 ≤ · · · ≤ a′ n− 1 ≤ a′ n.

An instance of sorting might be an array of names, like {Mike, Bob, Sally, Jill, Jan}, or a list of numbers like {154, 245, 568, 324, 654, 324}. Determining that you are dealing with a general problem is your first step towards solving it. An algorithm is a procedure that takes any of the possible input instances and transforms it to the desired output. There are many different algorithms for solving the problem of sorting. For example, insertion sort is a method for sorting that starts with a single element (thus forming a trivially sorted list) and then incrementally inserts the remaining elements so that the list stays sorted. This algorithm, implemented in C, is described below:

S.S. Skiena, The Algorithm Design Manual, 2nd ed., DOI: 10.1007/978-1-84800-070-4 1, ©c Springer-Verlag London Limited 2008

1. 1 R O B O T T O U R O P T I M I Z A T I O N 5

0 0 1

2

3 (^54)

6

7

8

Figure 1.2: A good instance for the nearest-neighbor heuristic

algorithm correctly solves a given problem. Correct algorithms usually come with a proof of correctness, which is an explanation of why we know that the algorithm must take every instance of the problem to the desired result. However, before we go further we demonstrate why “it’s obvious” never suffices as a proof of correctness, and is usually flat-out wrong.

1.1 Robot Tour Optimization

Let’s consider a problem that arises often in manufacturing, transportation, and testing applications. Suppose we are given a robot arm equipped with a tool, say a soldering iron. In manufacturing circuit boards, all the chips and other components must be fastened onto the substrate. More specifically, each chip has a set of contact points (or wires) that must be soldered to the board. To program the robot arm for this job, we must first construct an ordering of the contact points so the robot visits (and solders) the first contact point, then the second point, third, and so forth until the job is done. The robot arm then proceeds back to the first contact point to prepare for the next board, thus turning the tool-path into a closed tour, or cycle. Robots are expensive devices, so we want the tour that minimizes the time it takes to assemble the circuit board. A reasonable assumption is that the robot arm moves with fixed speed, so the time to travel between two points is proportional to their distance. In short, we must solve the following algorithm problem:

Problem: Robot Tour Optimization Input: A set S of n points in the plane. Output: What is the shortest cycle tour that visits each point in the set S?

You are given the job of programming the robot arm. Stop right now and think up an algorithm to solve this problem. I’ll be happy to wait until you find one...

6 1. I N T R O D U C T I O N T O A L G O R I T H M D E S I G N

Several algorithms might come to mind to solve this problem. Perhaps the most popular idea is the nearest-neighbor heuristic. Starting from some point p 0 , we walk first to its nearest neighbor p 1. From p 1 , we walk to its nearest unvisited neighbor, thus excluding only p 0 as a candidate. We now repeat this process until we run out of unvisited points, after which we return to p 0 to close off the tour. Written in pseudo-code, the nearest-neighbor heuristic looks like this: NearestNeighbor(P ) Pick and visit an initial point p 0 from P p = p 0 i = 0 While there are still unvisited points i = i + 1 Select pi to be the closest unvisited point to pi− 1 Visit pi Return to p 0 from pn− 1 This algorithm has a lot to recommend it. It is simple to understand and imple- ment. It makes sense to visit nearby points before we visit faraway points to reduce the total travel time. The algorithm works perfectly on the example in Figure 1.2. The nearest-neighbor rule is reasonably efficient, for it looks at each pair of points (pi, pj ) at most twice: once when adding pi to the tour, the other when adding pj. Against all these positives there is only one problem. This algorithm is completely wrong. Wrong? How can it be wrong? The algorithm always finds a tour, but it doesn’t necessarily find the shortest possible tour. It doesn’t necessarily even come close. Consider the set of points in Figure 1.3, all of which lie spaced along a line. The numbers describe the distance that each point lies to the left or right of the point labeled ‘0’. When we start from the point ‘0’ and repeatedly walk to the nearest unvisited neighbor, we might keep jumping left-right-left-right over ‘0’ as the algo- rithm offers no advice on how to break ties. A much better (indeed optimal) tour for these points starts from the leftmost point and visits each point as we walk right before returning at the rightmost point. Try now to imagine your boss’s delight as she watches a demo of your robot arm hopscotching left-right-left-right during the assembly of such a simple board. “But wait,” you might be saying. “The problem was in starting at point ‘0’. Instead, why don’t we start the nearest-neighbor rule using the leftmost point as the initial point p 0? By doing this, we will find the optimal solution on this instance.” That is 100% true, at least until we rotate our example 90 degrees. Now all points are equally leftmost. If the point ‘0’ were moved just slightly to the left, it would be picked as the starting point. Now the robot arm will hopscotch up-down- up-down instead of left-right-left-right, but the travel time will be just as bad as before. No matter what you do to pick the first point, the nearest-neighbor rule is doomed to work incorrectly on certain point sets.

8 1. I N T R O D U C T I O N T O A L G O R I T H M D E S I G N

1−e

1+e

1+e

1−e

(l) 1+e

1−e

1+e

1−e

(r)

Figure 1.4: A bad instance for the closest-pair heuristic, with the optimal solution

remaining pairs will connect these pairs alternately around the boundary. The total path length of the closest-pair tour is 3(1 − e) + 2(1 + e) +

(1 − e)^2 + (2 + 2e)^2. Compared to the tour shown in Figure 1.4(r), we travel over 20% farther than necessary when e ≈ 0. Examples exist where the penalty is considerably worse than this. Thus this second algorithm is also wrong. Which one of these algorithms per- forms better? You can’t tell just by looking at them. Clearly, both heuristics can end up with very bad tours on very innocent-looking input. At this point, you might wonder what a correct algorithm for our problem looks like. Well, we could try enumerating all possible orderings of the set of points, and then select the ordering that minimizes the total length:

OptimalTSP(P) d = ∞ For each of the n! permutations Pi of point set P If (cost(Pi) ≤ d) then d = cost(Pi) and Pmin = Pi Return Pmin

Since all possible orderings are considered, we are guaranteed to end up with the shortest possible tour. This algorithm is correct, since we pick the best of all the possibilities. But it is also extremely slow. The fastest computer in the world couldn’t hope to enumerate all the 20! =2,432,902,008,176,640,000 orderings of 20 points within a day. For real circuit boards, where n ≈ 1 , 000, forget about it. All of the world’s computers working full time wouldn’t come close to finishing the problem before the end of the universe, at which point it presumably becomes moot. The quest for an efficient algorithm to solve this problem, called the traveling salesman problem (TSP), will take us through much of this book. If you need to know how the story ends, check out the catalog entry for the traveling salesman problem in Section 16.4 (page 533).

1. 2 S E L E C T I N G T H E R I G H T J O B S 9

The President’s Algorist Halting State "Discrete" Mathematics (^) Calculated Bets

Programming Challenges

Steiner’s Tree Process Terminated

Tarjan of the Jungle The Four Volume Problem

Figure 1.5: An instance of the non-overlapping movie scheduling problem

Take-Home Lesson: There is a fundamental difference between algorithms, which always produce a correct result, and heuristics, which may usually do a good job but without providing any guarantee.

1.2 Selecting the Right Jobs

Now consider the following scheduling problem. Imagine you are a highly-in- demand actor, who has been presented with offers to star in n different movie projects under development. Each offer comes specified with the first and last day of filming. To take the job, you must commit to being available throughout this entire period. Thus you cannot simultaneously accept two jobs whose intervals overlap. For an artist such as yourself, the criteria for job acceptance is clear: you want to make as much money as possible. Because each of these films pays the same fee per film, this implies you seek the largest possible set of jobs (intervals) such that no two of them conflict with each other. For example, consider the available projects in Figure 1.5. We can star in at most four films, namely “Discrete” Mathematics, Programming Challenges, Calculated Bets, and one of either Halting State or Steiner’s Tree. You (or your agent) must solve the following algorithmic scheduling problem:

Problem: Movie Scheduling Problem Input: A set I of n intervals on the line. Output: What is the largest subset of mutually non-overlapping intervals which can be selected from I?

You are given the job of developing a scheduling algorithm for this task. Stop right now and try to find one. Again, I’ll be happy to wait.

There are several ideas that may come to mind. One is based on the notion that it is best to work whenever work is available. This implies that you should start with the job with the earliest start date – after all, there is no other job you can work on, then at least during the begining of this period.

1. 3 R E A S O N I N G A B O U T C O R R E C T N E S S 11

of n things, as proposed for the robot tour optimization problem. There are only about one million subsets when n = 20, which could be exhaustively counted within seconds on a decent computer. However, when fed n = 100 movies, 2^100 is much much greater than the 20! which made our robot cry “uncle” in the previous problem. The difference between our scheduling and robotics problems are that there is an algorithm which solves movie scheduling both correctly and efficiently. Think about the first job to terminate—i.e. the interval x which contains the rightmost point which is leftmost among all intervals. This role is played by “Discrete” Mathematics in Figure 1.5. Other jobs may well have started before x, but all of these must at least partially overlap each other, so we can select at most one from the group. The first of these jobs to terminate is x, so any of the overlapping jobs potentially block out other opportunities to the right of it. Clearly we can never lose by picking x. This suggests the following correct, efficient algorithm:

OptimalScheduling(I) While (I = ∅) do Accept the job j from I with the earliest completion date. Delete j, and any interval which intersects j from I.

Ensuring the optimal answer over all possible inputs is a difficult but often achievable goal. Seeking counterexamples that break pretender algorithms is an important part of the algorithm design process. Efficient algorithms are often lurk- ing out there; this book seeks to develop your skills to help you find them.

Take-Home Lesson: Reasonable-looking algorithms can easily be incorrect. Al- gorithm correctness is a property that must be carefully demonstrated.

1.3 Reasoning about Correctness

Hopefully, the previous examples have opened your eyes to the subtleties of algo- rithm correctness. We need tools to distinguish correct algorithms from incorrect ones, the primary one of which is called a proof. A proper mathematical proof consists of several parts. First, there is a clear, precise statement of what you are trying to prove. Second, there is a set of assump- tions of things which are taken to be true and hence used as part of the proof. Third, there is a chain of reasoning which takes you from these assumptions to the statement you are trying to prove. Finally, there is a little square ( ) or QED at the bottom to denote that you have finished, representing the Latin phrase for “thus it is demonstrated.” This book is not going to emphasize formal proofs of correctness, because they are very difficult to do right and quite misleading when you do them wrong. A proof is indeed a demonstration. Proofs are useful only when they are honest; crisp arguments explaining why an algorithm satisfies a nontrivial correctness property.

12 1. I N T R O D U C T I O N T O A L G O R I T H M D E S I G N

Correct algorithms require careful exposition, and efforts to show both cor- rectness and not incorrectness. We develop tools for doing so in the subsections below.

1.3.1 Expressing Algorithms

Reasoning about an algorithm is impossible without a careful description of the sequence of steps to be performed. The three most common forms of algorithmic notation are (1) English, (2) pseudocode, or (3) a real programming language. We will use all three in this book. Pseudocode is perhaps the most mysterious of the bunch, but it is best defined as a programming language that never complains about syntax errors. All three methods are useful because there is a natural tradeoff between greater ease of expression and precision. English is the most natural but least precise programming language, while Java and C/C++ are precise but diffi- cult to write and understand. Pseudocode is generally useful because it represents a happy medium. The choice of which notation is best depends upon which method you are most comfortable with. I usually prefer to describe the ideas of an algorithm in English, moving to a more formal, programming-language-like pseudocode or even real code to clarify sufficiently tricky details. A common mistake my students make is to use pseudocode to dress up an ill- defined idea so that it looks more formal. Clarity should be the goal. For example, the ExhaustiveScheduling algorithm on page 10 could have been better written in English as:

ExhaustiveScheduling(I) Test all 2n^ subsets of intervals from I, and return the largest subset consisting of mutually non-overlapping intervals.

Take-Home Lesson: The heart of any algorithm is an idea. If your idea is not clearly revealed when you express an algorithm, then you are using too low-level a notation to describe it.

1.3.2 Problems and Properties

We need more than just an algorithm description in order to demonstrate cor- rectness. We also need a careful description of the problem that it is intended to solve. Problem specifications have two parts: (1) the set of allowed input instances, and (2) the required properties of the algorithm’s output. It is impossible to prove the correctness of an algorithm for a fuzzily-stated problem. Put another way, ask the wrong problem and you will get the wrong answer. Some problem specifications allow too broad a class of input instances. Suppose we had allowed film projects in our movie scheduling problem to have gaps in

14 1. I N T R O D U C T I O N T O A L G O R I T H M D E S I G N

• Simplicity – Good counter-examples have all unnecessary details boiled away. They make clear exactly why the proposed algorithm fails. Once a counter- example has been found, it is worth simplifying it down to its essence. For example, the counter-example of Figure 1.6(l) could be made simpler and better by reducing the number of overlapped segments from four to two.

Hunting for counter-examples is a skill worth developing. It bears some simi- larity to the task of developing test sets for computer programs, but relies more on inspiration than exhaustion. Here are some techniques to aid your quest:

• Think small – Note that the robot tour counter-examples I presented boiled down to six points or less, and the scheduling counter-examples to only three intervals. This is indicative of the fact that when algorithms fail, there is usually a very simple example on which they fail. Amateur algorists tend to draw a big messy instance and then stare at it helplessly. The pros look carefully at several small examples, because they are easier to verify and reason about.
• Think exhaustively – There are only a small number of possibilities for the smallest nontrivial value of n. For example, there are only three interesting ways two intervals on the line can occur: (1) as disjoint intervals, (2) as overlapping intervals, and (3) as properly nesting intervals, one within the other. All cases of three intervals (including counter-examples to both movie heuristics) can be systematically constructed by adding a third segment in each possible way to these three instances.
• Hunt for the weakness – If a proposed algorithm is of the form “always take the biggest” (better known as the greedy algorithm), think about why that might prove to be the wrong thing to do. In particular,...
• Go for a tie – A devious way to break a greedy heuristic is to provide instances where everything is the same size. Suddenly the heuristic has nothing to base its decision on, and perhaps has the freedom to return something suboptimal as the answer.
• Seek extremes – Many counter-examples are mixtures of huge and tiny, left and right, few and many, near and far. It is usually easier to verify or rea- son about extreme examples than more muddled ones. Consider two tightly bunched clouds of points separated by a much larger distance d. The optimal TSP tour will be essentially 2d regardless of the number of points, because what happens within each cloud doesn’t really matter.

Take-Home Lesson: Searching for counterexamples is the best way to disprove the correctness of a heuristic.

1. 3 R E A S O N I N G A B O U T C O R R E C T N E S S 15

1.3.4 Induction and Recursion

Failure to find a counterexample to a given algorithm does not mean “it is obvious” that the algorithm is correct. A proof or demonstration of correctness is needed. Often mathematical induction is the method of choice. When I first learned about mathematical induction it seemed like complete magic. You proved a formula like

∑n i=1 i^ =^ n(n^ + 1)/2 for some basis case like 1 or 2, then assumed it was true all the way to n − 1 before proving it was true for general n using the assumption. That was a proof? Ridiculous! When I first learned the programming technique of recursion it also seemed like complete magic. The program tested whether the input argument was some basis case like 1 or 2. If not, you solved the bigger case by breaking it into pieces and calling the subprogram itself to solve these pieces. That was a program? Ridiculous! The reason both seemed like magic is because recursion is mathematical induc- tion. In both, we have general and boundary conditions, with the general condition breaking the problem into smaller and smaller pieces. The initial or boundary con- dition terminates the recursion. Once you understand either recursion or induction, you should be able to see why the other one also works. I’ve heard it said that a computer scientist is a mathematician who only knows how to prove things by induction. This is partially true because computer scientists are lousy at proving things, but primarily because so many of the algorithms we study are either recursive or incremental. Consider the correctness of insertion sort, which we introduced at the beginning of this chapter. The reason it is correct can be shown inductively:

• The basis case consists of a single element, and by definition a one-element array is completely sorted.
• In general, we can assume that the first n − 1 elements of array A are com- pletely sorted after n − 1 iterations of insertion sort.
• To insert one last element x to A, we find where it goes, namely the unique spot between the biggest element less than or equal to x and the smallest element greater than x. This is done by moving all the greater elements back by one position, creating room for x in the desired location.

One must be suspicious of inductive proofs, however, because very subtle rea- soning errors can creep in. The first are boundary errors. For example, our insertion sort correctness proof above boldly stated that there was a unique place to insert x between two elements, when our basis case was a single-element array. Greater care is needed to properly deal with the special cases of inserting the minimum or maximum elements. The second and more common class of inductive proof errors concerns cavallier extension claims. Adding one extra item to a given problem instance might cause the entire optimal solution to change. This was the case in our scheduling problem (see Figure 1.7). The optimal schedule after inserting a new segment may contain