Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
A collection of exercises and concepts related to algorithms and data structures, covering topics such as time complexity analysis, stack implementations, binary search trees, leftist heaps, and topological sorting. It includes pseudocode examples and problems for finding minimum subsequence sums, determining prime numbers, and bin-packing strategies. Valuable for students studying computer science and algorithm design, providing practical problems and theoretical insights into fundamental data structures and algorithms. It also touches on advanced topics like splay trees and binomial queues, offering a comprehensive overview of essential concepts in the field. Useful for university students.
Typology: Study Guides, Projects, Research
1 / 422
This page cannot be seen from the preview
Don't miss anything!
This book describes data structures , methods of organizing large amounts of data, and algorithm analysis, the estimation of the running time of algorithms. As computers become faster and faster, the need for programs that can handle large amounts of input becomes more acute. Paradoxically, this requires more careful attention to efficiency, since inefficiencies in programs become most obvious when input sizes are large. By analyzing an algorithm before it is actually coded, students can decide if a particular solution will be feasible. For example, in this text students look at specific problems and see how careful implementations can reduce the time constraint for large amounts of data from 16 years to less than a second. Therefore, no algorithm or data structure is presented without an explanation of its running time. In some cases, minute details that affect the running time of the implementation are explored. Once a solution method is determined, a program must still be written. As computers have become more powerful, the problems they solve have become larger and more complex, thus requiring development of more intricate programs to solve the problems. The goal of this text is to teach students good programming and algorithm analysis skills simultaneously so that they can develop such programs with the maximum amount of efficiency. This book is suitable for either an advanced data structures (CS7) course or a first-year graduate course in algorithm analysis. Students should have some knowledge of intermediate programming, including such topics as pointers and recursion, and some background in discrete math.
I believe it is important for students to learn how to program for themselves, not how to copy programs from a book. On the other hand, it is virtually impossible to discuss realistic programming issues without including sample code. For this reason, the book usually provides about half to three-quarters of an implementation, and the student is encouraged to supply the rest. The algorithms in this book are presented in ANSI C, which, despite some flaws, is arguably the most popular systems programming language. The use of C instead of Pascal allows the use of dynamically allocated arrays (see for instance rehashing in Ch. 5). It also produces simplified code in several places, usually because the and (&&) operation is short- circuited. Most criticisms of C center on the fact that it is easy to write code that is barely readable. Some of the more standard tricks, such as the simultaneous assignment and testing against 0 via if (x=y) are generally not used in the text, since the loss of clarity is compensated by only a few keystrokes and no increased speed. I believe that this book demonstrates that unreadable code can be avoided by exercising reasonable care.
Chapter 1 contains review material on discrete math and recursion. I believe the only way to be comfortable with recursion is to see good uses over and over. Therefore, recursion is prevalent in this text, with examples in every chapter except Chapter 5. Chapter 2 deals with algorithm analysis. This chapter explains asymptotic analysis and its major weaknesses. Many
A solutions manual containing solutions to almost all the exercises is available separately from The Benjamin/Cummings Publishing Company.
References are placed at the end of each chapter. Generally the references either are historical, representing the original source of the material, or they represent extensions and improvements to the results given in the text. Some references represent solutions to exercises.
I would like to thank the many people who helped me in the preparation of this and previous versions of the book. The professionals at Benjamin/Cummings made my book a considerably less harrowing experience than I had been led to expect. I'd like to thank my previous editors, Alan Apt and John Thompson, as well as Carter Shanklin, who has edited this version, and Carter's assistant, Vivian McDougal, for answering all my questions and putting up with my delays. Gail Carrigan at Benjamin/Cummings and Melissa G. Madsen and Laura Snyder at Publication Services did a wonderful job with production. The C version was handled by Joe Heathward and his outstanding staff, who were able to meet the production schedule despite the delays caused by Hurricane Andrew. I would like to thank the reviewers, who provided valuable comments, many of which have been incorporated into the text. Alphabetically, they are Vicki Allan (Utah State University), Henry Bauer (University of Wyoming), Alex Biliris (Boston University), Jan Carroll (University of North Texas), Dan Hirschberg (University of California, Irvine), Julia Hodges (Mississippi State University), Bill Kraynek (Florida International University), Rayno D. Niemi (Rochester Institute of Technology), Robert O. Pettus (University of South Carolina), Robert Probasco (University of Idaho), Charles Williams (Georgia State University), and Chris Wilson (University of Oregon). I would particularly like to thank Vicki Allan, who carefully read every draft and provided very detailed suggestions for improvement. At FIU, many people helped with this project. Xinwei Cui and John Tso provided me with their class notes. I'd like to thank Bill Kraynek, Wes Mackey, Jai Navlakha, and Wei Sun for using drafts in their courses, and the many students who suffered through the sketchy early drafts. Maria Fiorenza, Eduardo Gonzalez, Ancin Peter, Tim Riley, Jefre Riser, and Magaly Sotolongo reported several errors, and Mike Hall checked through an early draft for programming errors. A special thanks goes to Yuzheng Ding, who compiled and tested every program in the original book, including the conversion of pseudocode to Pascal. I'd be remiss to forget Carlos Ibarra and Steve Luis, who kept the printers and the computer system working and sent out tapes on a minute's notice. This book is a product of a love for data structures and algorithms that can be obtained only from top educators. I'd like to take the time to thank Bob Hopkins, E. C. Horvath, and Rich Mendez, who taught me at Cooper Union, and Bob Sedgewick, Ken Steiglitz, and Bob Tarjan from Princeton. Finally, I'd like to thank all my friends who provided encouragement during the project. In particular, I'd like to thank Michele Dorchak, Arvin Park, and Tim Snyder for listening to my stories; Bill Kraynek, Alex Pelin, and Norman Pestaina for being civil next-door (office) neighbors, even when I wasn't; Lynn and Toby Berk for shelter during Andrew, and the HTMC for work relief. Any mistakes in this book are, of course, my own. I would appreciate reports of any errors you find; my e-mail address is [email protected]. M.A.W. Miami, Florida September 1992
In this chapter, we discuss the aims and goals of this text and briefly review programming concepts and discrete mathematics. We will See that how a program performs for reasonably large input is just as important as its performance on moderate amounts of input. Review good programming style. Summarize the basic mathematical background needed for the rest of the book. Briefly review recursion.
Suppose you have a group of n numbers and would like to determine the k th largest. This is known as the selection problem. Most students who have had a programming course or two would have no difficulty writing a program to solve this problem. There are quite a few "obvious" solutions. One way to solve this problem would be to read the n numbers into an array, sort the array in decreasing order by some simple algorithm such as bubblesort, and then return the element in position k. A somewhat better algorithm might be to read the first k elements into an array and sort them (in decreasing order). Next, each remaining element is read one by one. As a new element arrives, it is ignored if it is smaller than the k th element in the array. Otherwise, it is placed in its correct spot in the array, bumping one element out of the array. When the algorithm ends, the element in the k th position is returned as the answer. Both algorithms are simple to code, and you are encouraged to do so. The natural questions, then, are which algorithm is better and, more importantly, is either algorithm good enough? A simulation using a random file of 1 million elements and k = 500,000 will show that neither algorithm finishes in a reasonable amount of time--each requires several days of computer processing to terminate (albeit eventually with a correct answer). An alternative method, discussed in Chapter 7, gives a solution in about a second. Thus, although our proposed algorithms work, they cannot be considered good algorithms, because they are entirely impractical for input sizes that a third algorithm can handle in a reasonable amount of time. A second problem is to solve a popular word puzzle. The input consists of a two-dimensional array of letters and a list of words. The object is to find the words in the puzzle. These words may be horizontal, vertical, or diagonal in any direction. As an example, the puzzle shown in Figure 1.1 contains the words this, two, fat , and that. The word this begins at row 1, column 1 (1,1) and extends to (1, 4); two goes from (1, 1) to (3, 1); fat goes from (4, 1) to (2, 3); and that goes from (4, 4) to (1, 1). Again, there are at least two straightforward algorithms that solve the problem. For each word in the word list, we check each ordered triple ( row, column, orientation ) for the presence of the word. This amounts to lots of nested for loops but is basically straightforward. Alternatively, for each ordered quadruple ( row, column, orientation, number of characters ) that doesn't run off an end of the puzzle, we can test whether the word indicated is in the word list. Again, this amounts to lots of nested for loops. It is possible to save some time if the maximum number of characters in any word is known. It is relatively easy to code up either solution and solve many of the real-life puzzles commonly published in magazines. These typically have 16 rows, 16 columns, and 40 or so words. Suppose, however, we consider the
Let x = log c b, y = log c a , and z = log a b. Then, by the definition of logarithms, cx^ = b, cy^ = a , and az^ = b. Combining these three equalities yields ( cy ) z^ = cx^ = b. Therefore, x = yz , which implies z = x/y , proving the theorem. THEOREM 1.2. log ab = log a + log b PROOF: Let x = log a, y = log b, z = log ab. Then, assuming the default base of 2, 2 x = a , 2 y^ = b , 2 z^ = ab. Combining the last three equalities yields 2 x 2 y^ = 2 z^ = ab. Therefore, x + y = z , which proves the theorem. Some other useful formulas, which can all be derived in a similar manner, follow. log a/b = log a - log b log (ab )^ =^ b^ log^ a log x < x for all x > 0 log 1 = 0, log 2 = 1, log 1,024 = 10, log 1,048,576 = 20 1.2.3. Series The easiest formulas to remember are and the companion, In the latter formula, if 0 < a < 1, then and as n tends to , the sum approaches 1/(1 - a ). These are the "geometric series" formulas. We can derive the last formula for in the following manner. Let S be the sum. Then S = 1 + a + a^2 + a^3 + a^4 + a^5 +...
Then aS = a + a^2 + a^3 + a^4 + a^5 +... If we subtract these two equations (which is permissible only for a convergent series), virtually all the terms on the right side cancel, leaving S - aS = 1 which implies that We can use this same technique to compute , a sum that occurs frequently. We write and multiply by 2, obtaining Subtracting these two equations yields Thus, S = 2. Another type of common series in analysis is the arithmetic series. Any such series can be evaluated from the basic formula. For instance, to find the sum 2 + 5 + 8 +...^ + (3 k - 1), rewrite it as 3(1 + 2+ 3 +...^ + k ) - (1 + 1 + 1 +...^ + 1), which is clearly 3 k ( k + 1)/2 - k. Another way to remember this is to add the first and last terms (total 3 k + 1), the second and next to last terms (total 3 k + 1), and so on. Since there are k /2 of these pairs, the total sum is k (3 k + 1)/2, which is the same answer as before. The next two formulas pop up now and then but are fairly infrequent. When k = -1, the latter formula is not valid. We then need the following formula, which is used far more in computer science than in other mathematical disciplines. The numbers, H N, are known as the harmonic numbers, and the sum is
< (3/5)(5/3) k +1 + (3/5)^2 (5/3) k + < (3/5)(5/3) k +1^ +^ (9/25)(5/3) k + which simplifies to Fk +1 < (3/5 + 9/25)(5/3) k + < (24/25)(5/3) k + < (5/3) k + proving the theorem. As a second example, we establish the following theorem. THEOREM 1.3. PROOF: The proof is by induction. For the basis, it is readily seen that the theorem is true when n = 1. For the inductive hypothesis, assume that the theorem is true for 1 k n. We will establish that, under this assumption, the theorem is true for n + 1. We have Applying the inductive hypothesis, we obtain Thus, proving the theorem.
The statement Fk k^2 is false. The easiest way to prove this is to compute F 11 = 144 > 11^2.
Proof by contradiction proceeds by assuming that the theorem is false and showing that this assumption implies that some known property is false, and hence the original assumption was erroneous. A classic example is the proof that there is an infinite number of primes. To prove this, we assume that the theorem is false, so that there is some largest prime pk. Let p 1 , p 2 ,... , pk be all the primes in order and consider N = p 1 p 2 p 3.^.^.^ pk + 1 Clearly, N is larger than p k, so by assumption N is not prime. However, none of p 1 , p 2 ,... , p k divide N exactly, because there will always be a remainder of 1. This is a contradiction, because every number is either prime or a product of primes. Hence, the original assumption, that p k is the largest prime, is false, which implies that the theorem is true. int f( int x ) { /1/ if ( x = 0 ) /2/ return 0; else /3/ return( 2f(x-1) + xx ); } Figure 1.2 A recursive function
Most mathematical functions that we are familiar with are described by a simple formula. For instance, we can convert temperatures from Fahrenheit to Celsius by applying the formula C = 5( F - 32)/ Given this formula, it is trivial to write a C function; with declarations and braces removed, the one-line formula translates to one line of C. Mathematical functions are sometimes defined in a less standard form. As an example, we can define a function f , valid on nonnegative integers, that satisfies f (0) = 0 and f ( x ) = 2 f ( x - 1) + x^2. From this definition we see that f (1) = 1, f (2) = 6, f (3) = 21, and f (4) = 58. A function that is defined in terms of itself is called recursive. C allows functions to be recursive.* It is important to remember that what C provides is merely an attempt to follow the recursive spirit. Not all mathematically recursive functions are efficiently (or correctly) implemented by C's simulation of recursion. The idea is that the recursive function f ought to be expressible in only a few lines, just like a non-recursive function. Figure 1.2 shows the recursive implementation of f. *Using recursion for numerical calculations is usually a bad idea. We have done so to illustrate the basic points.
/3/ return( bad (n/3 + 1) + n - 1 ); } Figure 1.3 A nonterminating recursive program Our recursive strategy to understand words is as follows: If we know the meaning of a word, then we are done; otherwise, we look the word up in the dictionary. If we understand all the words in the definition, we are done; otherwise, we figure out what the definition means by recursively looking up the words we don't know. This procedure will terminate if the dictionary is well defined but can loop indefinitely if a word is either not defined or circularly defined. Printing Out Numbers Suppose we have a positive integer, n , that we wish to print out. Our routine will have the heading print_out ( n ). Assume that the only I/O routines available will take a single-digit number and output it to the terminal. We will call this routine print_digit ; for example, print_digit (4) will output a 4 to the terminal. Recursion provides a very clean solution to this problem. To print out 76234, we need to first print out 7623 and then print out 4. The second step is easily accomplished with the statement print_digit ( n %10), but the first doesn't seem any simpler than the original problem. Indeed it is virtually the same problem, so we can solve it recursively with the statement print_out ( n /10). This tells us how to solve the general problem, but we still need to make sure that the program doesn't loop indefinitely. Since we haven't defined a base case yet, it is clear that we still have something to do. Our base case will be print_digit ( n ) if 0 n < 10. Now print_out ( n ) is defined for every positive number from 0 to 9, and larger numbers are defined in terms of a smaller positive number. Thus, there is no cycle. The entire procedure* is shown Figure 1.4. The term procedure refers to a function that returns void. We have made no effort to do this efficiently. We could have avoided using the mod routine (which is very expensive) because n %10 = n - n /10 (^) * 10. Recursion and Induction Let us prove (somewhat) rigorously that the recursive number-printing program works. To do so, we'll use a proof by induction. THEOREM 1. The recursive number-printing algorithm is correct for n 0. PROOF: First, if n has one digit, then the program is trivially correct, since it merely makes a call to print_digit. Assume then that print_out works for all numbers of k or fewer digits. A number of k + 1 digits is expressed by its first k digits followed by its least significant digit. But the number formed by the first k digits is exactly n /10 , which, by the indicated hypothesis is correctly printed, and the last digit is n mod10, so the program prints out any ( k + 1)-digit number correctly. Thus, by induction, all numbers are correctly printed. void print_out( unsigned int n ) / print nonnegative n */
{ if( n<10 ) print_digit( n ); else { print_out( n/10 ); print_digit( n%10 ); } } Figure 1.4 Recursive routine to print an integer This proof probably seems a little strange in that it is virtually identical to the algorithm description. It illustrates that in designing a recursive program, all smaller instances of the same problem (which are on the path to a base case) may be assumed to work correctly. The recursive program needs only to combine solutions to smaller problems, which are "magically" obtained by recursion, into a solution for the current problem. The mathematical justification for this is proof by induction. This gives the third rule of recursion:
This chapter sets the stage for the rest of the book. The time taken by an algorithm confronted with large amounts of input will be an important criterion for deciding if it is a good algorithm. (Of course, correctness is most important.)
**c. Give a precise closed-form expression for Fn. 1.10 Prove the following formulas:
There are many good textbooks covering the mathematics reviewed in this chapter. A small subset is [1], [2], [3], [11], [13], and [14]. Reference [11] is specifically geared toward the analysis of algorithms. It is the first volume of a three- volume series that will be cited throughout this text. More advanced material is covered in [6]. Throughout this book we will assume a knowledge of C [10]. Occasionally, we add a feature where necessary for clarity. We also assume familiarity with pointers and recursion (the recursion summary in this chapter is meant to be a quick review). We will attempt to provide hints on their use where appropriate throughout the textbook. Readers not familiar with these should consult [4], [8], [12], or any good intermediate programming textbook. General programming style is discussed in several books. Some of the classics are [5], [7], and [9].
Big-Oh, because Big-Oh allows the possibility that the growth rates are the same. To prove that some function T ( n ) = O ( f ( n )), we usually do not apply these definitions formally but instead use a repertoire of known results. In general, this means that a proof (or determination that the assumption is incorrect) is a very simple calculation and should not involve calculus, except in extraordinary circumstances (not likely to occur in an algorithm analysis). When we say that T ( n ) = O ( f(n )), we are guaranteeing that the function T ( n ) grows at a rate no faster than f ( n ); thus f ( n ) is an upper bound on T ( n ). Since this implies that f ( n ) = ( T ( n )), we say that T ( n ) is a lower bound on f ( n ). As an example, n^3 grows faster than n^2 , so we can say that n^2 = O ( n^3 ) or n^3 = ( n^2 ). f(n ) = n^2 and g ( n ) = 2 n^2 grow at the same rate, so both f ( n ) = O ( g ( n )) and f ( n ) = ( g ( n )) are true. When two functions grow at the same rate, then the decision whether or not to signify this with () can depend on the particular context. Intuitively, if g ( n ) = 2 n^2 , then g ( n ) = O ( n^4 ), g ( n ) = O ( n^3 ), and g ( n ) = O ( n^2 ) are all technically correct, but obviously the last option is the best answer. Writing g ( n ) = ( n^2 ) says not only that g ( n ) = O ( n^2 ), but also that the result is as good (tight) as possible. The important things to know are RULE 1: If T 1 ( n ) = O ( f ( n )) and T 2 ( n ) = O ( g ( n )), then ( a ) T 1 ( n ) + T 2 ( n ) = max ( O ( f ( n )), O ( g ( n ))), ( b ) T 1 ( n ) * T 2 ( n ) = O ( f ( n ) * g ( n )), Function Name
c Constant log n Logarithmic log^2 n Log-squared n Linear n log n n^2 Quadratic n^3 Cubic 2 n^ Exponential Figure 2.1 Typical growth rates RULE 2: If T ( x ) is a polynomial of degree n, then T ( x ) = ( xn ). RULE 3: log k^ n = O ( n ) for any constant k. This tells us that logarithms grow very slowly. To see that rule 1(a) is correct, note that by definition there exist four constants c 1 , c 2 , n 1 , and n 2 such that T 1 ( n ) c 1
f ( n ) for n n 1 and T 2 ( n ) c 2 g ( n ) for n n 2. Let n 0 = max (n 1 , n 2 ). Then, for n n 0 , T 1 ( n ) c 1 f ( n ) and T 2 ( n ) c 2 g ( n ), so that T 1 ( n ) + T 2 ( n ) c 1 f ( n ) + c 2 g ( n ). Let c 3 = max( c 1 , c 2 ). Then, T 1 ( n ) + T 2 ( n ) c 3 f ( n ) + c 3 g ( n ) c 3 ( f ( n ) + g ( n )) 2 c 3 max( f ( n ), g ( n )) c max( f ( n ), g ( n )) for c = 2 c 3 and n n 0. We leave proofs of the other relations given above as exercises for the reader. This information is sufficient to arrange most of the common functions by growth rate (see Fig. 2.1). Several points are in order. First, it is very bad style to include constants or low-order terms inside a Big-Oh. Do not say T ( n ) = O (2 n^2 ) or T ( n ) = O ( n^2 + n ). In both cases, the correct form is T ( n ) = O ( n^2 ). This means that in any analysis that will require a Big-Oh answer, all sorts of shortcuts are possible. Lower-order terms can generally be ignored, and constants can be thrown away. Considerably less precision is required in these cases. Secondly, we can always determine the relative growth rates of two functions f ( n ) and g ( n ) by computing lim n f ( n )/ g ( n ), using L'Hôpital's rule if necessary.* *L'Hôpital's rule states that if lim n f ( n ) = and lim n g ( n ) = , then lim n f ( n )/ g ( n ) = lim n f '( n ) / g '( n ), where f '( n ) and g '( n ) are the derivatives of f ( n ) and g ( n ), respectively. The limit can have four possible values: The limit is 0: This means that f ( n ) = o( g ( n )). The limit is c 0: This means that f ( n ) = ( g ( n )). The limit is : This means that g ( n ) = o( f ( n )). The limit oscillates: There is no relation (this will not happen in our context). Using this method almost always amounts to overkill. Usually the relation between f ( n ) and g ( n ) can be derived by simple algebra. For instance, if f ( n ) = n log n and g ( n ) = n 1.5, then to decide which of f ( n ) and g ( n ) grows faster, one really needs to determine which of log n and n 0.5^ grows faster. This is like determining which of log^2 n or n grows faster. This is a simple problem, because it is already known that n grows faster than any power of a log. Thus, g ( n ) grows faster than f ( n ). One stylistic note: It is bad to say f ( n ) O ( g ( n )), because the inequality is implied by the definition. It is wrong to write f ( n ) O ( g ( n )), which does not make sense.
In order to analyze algorithms in a formal framework, we need a model of computation. Our model is basically a normal computer, in which instructions are executed sequentially. Our model has the standard repertoire of simple instructions, such as addition, multiplication, comparison, and assignment, but, unlike real computers, it takes exactly one time unit to do anything (simple). To be reasonable, we will assume that, like a modern computer, our model has fixed size (say 32-bit) integers and that there are no fancy operations, such as matrix inversion or sorting, that clearly
