Concise Notes on Data Structures and Algorithms, Study notes of Data Structures and Algorithms

Download Concise Notes on Data Structures and Algorithms and more Study notes Data Structures and Algorithms in PDF only on Docsity!

Concise Notes on Data

Structures and Algorithms

Ruby Edition

Christopher Fox

James Madison University

iii

v

vi

• 1 I ntroductIon
  • What Are Data Structures and Algorithms?
  • Structure of the Book
  • The Ruby Programming Language
  • Review Questions
  • Exercises
  • Review Question Answers
• 2 BuIlt -I n types
  • Simple and Structured Types
  • Types in Ruby
  • Symbol: A Simple Type in Ruby
  • Review Questions
  • Exercises
  • Review Question Answers
• 3 ArrAys
  • Introduction
  • Varieties of Arrays
  • Arrays in Ruby
  • Review Questions
  • Exercises
  • Review Question Answers
• 4 AssertIons
  • Introduction
  • Types of Assertions
  • Assertions and Abstract Data Types
  • Using Assertions
  • Assertions in Ruby
  • Review Questions
  • Exercises
  • Review Question Answers
• 5 contAIners
  • Introduction
  • Varieties of Containers
  • A Container Taxonomy Contents
  • Interfaces in Ruby
  • Exercises
  • Review Question Answers
• 6 stAcks
  • Introduction
  • The Stack ADT
  • The Stack Interface
  • Using Stacks—An Example
  • Contiguous Implementation of the Stack ADT
  • Linked Implementation of the Stack ADT
  • Summary and Conclusion
  • Review Questions
  • Exercises
  • Review Question Answers
• 7 Queues
  • Introduction
  • The Queue ADT
  • The Queue Interface
  • Using Queues—An Example
  • Contiguous Implementation of the Queue ADT
  • Linked Implementation of the Queue ADT
  • Summary and Conclusion
  • Review Questions
  • Exercises
  • Review Question Answers
• 8 stAcks And recursIon
  • Introduction
  • Balanced Brackets
  • Infix, Prefix, and Postfix Expressions
  • Tail Recursive Algorithms
  • Summary and Conclusion
  • Review Questions
  • Exercises
  • Review Question Answers
• 9 collectIons Contents
  • Introduction
  • Iteration Design Alternatives
  • The Iterator Design Pattern
  • Iteration in Ruby
  • Collections, Iterators, and Containers
  • Summary and Conclusion
  • Review Questions
  • Exercises
  • Review Question Answers
• 10 lIsts
  • Introduction
  • The List ADT
  • The List Interface
  • Using Lists—An Example
  • Contiguous Implementation of the List ADT
  • Linked Implementation of the List ADT
  • Implementing Lists in Ruby
  • Summary and Conclusion
  • Review Questions
  • Exercises
  • Review Question Answers
• 11 AnAlyzIng AlgorIthms
  • Introduction
  • Measuring the Amount of Work Done
  • Which Operations to Count
  • Best, Worst, and Average Case Complexity
  • Review Questions
  • Exercises
  • Review Question Answers
• 12 FunctIon growth rAtes
  • Introduction
  • Definitions and Notation
  • Establishing the Order of Growth of a Function
  • Applying Orders of Growth
  • Summary and Conclusion
  • Review Questions Contents
  • Exercises
  • Review Question Answers
• 13 BAsIc sortIng AlgorIthms
  • Introduction
  • Bubble Sort
  • Selection Sort
  • Insertion Sort
  • Shell Sort
  • Summary and Conclusion
  • Review Questions
  • Exercises
  • Review Question Answers
• 14 recurrences
  • Introduction
  • Setting Up Recurrences
  • Review Questions
  • Exercises
  • Review Question Answers
• 15: merge sort And QuIcksort
  • Introduction
  • Merge Sort
  • Quicksort
  • Summary and Conclusion
  • Review Questions
  • Exercises
  • Review Question Answers
• 16 trees , heAps , And heApsort
  • Introduction
  • Basic Terminology
  • Binary Trees
  • Heaps
  • Heapsort
  • Summary and Conclusion
  • Review Questions
  • Exercises Contents
  • Review Question Answers
• 17 BInAry trees
  • Introduction
  • The Binary Tree ADT
  • The Binary Tree Class
  • Contiguous Implementation of Binary Trees
  • Linked Implementation of Binary Trees
  • Summary and Conclusion
  • Review Questions
  • Exercises
  • Review Question Answers
• 18 BInAry seArch And BInAry seArch trees
  • Introduction
  • Binary Search
  • Binary Search Trees
  • The Binary Search Tree Class
  • Summary and Conclusion
  • Review Questions
  • Exercises
  • Review Question Answers
• 19 sets
  • Introduction
  • The Set ADT
  • The Set Interface
  • Contiguous Implementation of Sets
  • Linked Implementation of Sets
  • Summary and Conclusion
  • Review Questions
  • Exercises
  • Review Question Answers
• 20 mAps
  • Introduction
  • The Map ADT
  • The Map Interface
  • Contiguous Implementation of the Map ADT
  • Linked Implementation of the Map ADT Contents
  • Summary and Conclusion
  • Review Questions
  • Exercises
  • Review Question Answers
• 21 hAshIng
  • Introduction
  • The Hashing Problem
  • Collision Resolution Schemes
  • Summary and Conclusion
  • Review Questions
  • Exercises
  • Review Question Answers
• 22 hAshed collectIons
  • Introduction
  • Hash Tablets
  • HashSets
  • Implementing Hashed Collections in Ruby
  • Summary and Conclusion
  • Review Questions
  • Exercises
  • Review Question Answers
• glossAry

1: Introduction The thing that makes an abstract data type abstract is that its carrier set and its operations are mathematical entities, like numbers or geometric objects; all details of implementation on a computer are ignored This makes it easier to reason about them and to understand what they are For example, we can decide how div and mod should work for negative numbers in the Integer ADT without having to worry about how to make this work on real computers Then we can deal with implementation of our decisions as a separate problem Once an abstract data type is implemented on a computer, we call it a data type Data type : An implementation of an abstract data type on a computer Thus, for example, the Boolean ADT is implemented as the boolean type in Java, and the bool type in C++; the Integer ADT is realized as the int and long types in Java, and the Integer class in Ruby; the String ADT is implemented as the String class in Java and Ruby Abstract data types are very useful for helping us understand the mathematical objects that we use in our computations, but, of course, we cannot use them directly in our programs To use ADTs in programming, we must figure out how to implement them on a computer Implementing an ADT requires two things:

• Representing the values in the carrier set of the ADT by data stored in computer memory, and
• Realizing computational mechanisms for the operations of the ADT Finding ways to represent carrier set values in a computer’s memory requires that we determine how to arrange data (ultimately bits) in memory locations so that each value of the carrier set has a unique representation Such things are data structures Data structure : An arrangement of data in memory locations to represent values of the carrier set of an abstract data type Realizing computational mechanisms for performing operations of the type really means finding algorithms that use the data structures for the carrier set to implement the operations of the ADT And now it should be clear why we study data structures and algorithms together: to implement an ADT, we must find data structures to represent the values of its carrier set and algorithms to work with these data structures to implement its operations A course in data structures and algorithms is thus a course in implementing abstract data types It may seem that we are paying a lot of attention to a minor topic, but abstract data types are really the foundation of everything we do in computing Our computations work on data This data must represent things and be manipulated according to rules These things and the rules for their manipulation amount to abstract data types Usually there are many ways to implement an ADT A large part of the study of data structures and algorithms is learning about alternative ways to implement an ADT and evaluating the alternatives to determine their advantages and disadvantages Typically some alternatives will be better for certain applications and other alternatives will be better for

1: Introduction other applications Knowing how to do such evaluations to make good design decisions is an essential part of becoming an expert programmer Structure of the Book In this book we will begin by studying fundamental data types that are usually implemented for us in programming languages Then we will consider how to use these fundamental types and other programming language features (such references) to implement more complicated ADTs Along the way we will construct a classification of complex ADTs that will serve as the basis for a class library of implementations We will also learn how to measure an algorithm’s efficiency and use this skill to study algorithms for searching and sorting, which are very important in making our programs efficient when they must process large data sets The Ruby Programming Language Although the data structures and algorithms we study are not tied to any program or programming language, we need to write particular programs in particular languages to practice implementing and using the data structures and algorithms that we learn In this book, we will use the Ruby programming language Ruby is an interpreted, purely object-oriented language with many powerful features, such as garbage collection, dynamic arrays, hash tables, and rich string processing facilities We use Ruby because it is a fairly popular, full-featured object-oriented language, but it can be learned well enough to write substantial programs fairly quickly Thus we will be able to use a powerful language and still have time to concentrate on data structures and algorithms, which is what we are really interested in Also, it is free Ruby is weakly typed, does not support design-by-contract, and has a somewhat frugal collection of features for object-oriented programming Although this makes the language easier to learn and use, it also opens up many opportunities for errors Careful attention to types, fully understanding preconditions for executing methods, and thoughtful use of class hierarchies are important for novice programmers, so we will pay close attention to these matters in our discussion of data structures and algorithms, and we will, when possible, incorporate this material into Ruby code This often results in code that does not conform to the style prevalent in the Ruby community However, programmers must understand and appreciate these matters so that they can handle data structures in more strongly typed languages such as Java, C++, or C# Review Questions 1 What would be the carrier set and some operations of the Character ADT? 2 How might the Bit String ADT carrier set be represented on a computer in some high level language? 3 How might the concatenation operation of the Bit String ADT be realized using the carrier set representation you devised for question two above?

2: B uIlt -I n types

Simple and Structured Types Virtually all programming languages have implementations of several ADTs built into them, thus providing the set of types provided by the language We can distinguish two sorts of built-in types: Simple types: The values of the carrier set are atomic, that is, they cannot be divided into parts Common examples of simple types are integers, Booleans, floating point numbers, enumerations, and characters Some languages also provide strings as built-in types Structured types : The values of the carrier set are not atomic, consisting instead of several atomic values arranged in some way Common examples of structured types are arrays, records, classes, and sets Some languages treat strings as structured types Note that both simple and structured types are implementations of ADTs, it is simply a question of how the programming language treats the values of the carrier set of the ADT in its implementation The remainder of this chapter considers some Ruby simple and structured types to illustrate these ideas Types in Ruby Ruby is a pure object-oriented language, meaning that all types in Ruby are classes, and every value in a Ruby program is an instance of a class This has several consequences for the way values can be manipulated that may seem odd to programmers familiar with languages that have values that are not objects For example, values in Ruby respond to method calls: The expressions 142.even? and “Hello”.empty? are perfectly legitimate (the first expression is true and the second is false) The fact that all types in Ruby are classes has consequences for the way data structures are implemented as well, as we will see later on Ruby has many built-in types because it has many built-in classes Here we only consider a few Ruby types to illustrate how they realize ADTs Symbol: A Simple Type in Ruby Ruby has many simple types, including numeric classes such as Integer, Fixnum, Bignum, Float, BigDecimal, Rational, and Complex, textual classes such as String, Symbol, and Regexp, and many more One unusual and interesting simple type is Symbol, which we consider in more detail to illustrate how a type in a programming language realizes an ADT Ruby has a String class whose instances are mutable sequences of Unicode characters Symbol class instances are character sequences that are not mutable, and consequently the Symbol class has far fewer operations than the String class Ruby in effect has

2: Built-In Types implementations of two String ADTs—we consider the simpler one, calling it the Symbol ADT for purposes of this discussion The carrier set of the Symbol ADT is the set of all finite sequences of characters over the Unicode characters set (Unicode is a standard character set of over 109,000 characters from 93 scripts) Hence this carrier set includes the string of zero characters (the empty string), all strings of one character, all strings of two characters, and so forth This carrier set is infinite The operations of the Symbol ADT are the following a == b —returns true if and only if symbols a and b are identical a<=b—returns true if and only if either symbols a and b are identical, or symbol a precedes symbol b in Unicode collating sequence order a < b —returns true if and only if symbol a precedes symbol b in Unicode collating sequence order empty? ( a )—returns true if and only if symbol a is the empty symbol a =~ b —returns the index of the first character of the first portion of symbol a that matches the regular expression b If there is not match, the result is undefined caseCompare ( a , b )—compares symbols a and b , ignoring case, and returns -1 if a < b , 0 if a == b , and 1 otherwise length ( a )—returns the number of characters in symbol a. capitalize ( a )—returns the symbol generated from a by making its first character uppercase and making its remaining characters lowercase downcase ( a )—returns the symbol generated from a by making all characters in a lowercase upcase ( a )—returns the symbol generated from a by making all characters in a uppercase swapcase ( a )—returns the symbol generated from a by making all lowercase characters in a uppercase and all uppercase characters in a lowercase charAt ( a , b )—returns the one character symbol consisting of the character of symbol a at index b (counting from 0); the result is undefined if b is less than 0 or greater than or equal to the length of a charAt ( a , b , c )—returns the substring of symbol a beginning at index b (counting from 0), and continuing for c characters; the result is undefined if b is less than 0 or greater than or equal to the length of a , or if c is negative If a + b is greater than the length of a , the result is the suffix of symbol a beginning at index b succ ( a )—returns the symbol that is the successor of symbol a If a contains characters or letters, the successor of a is found by incrementing the right-most letter or digit according to the Unicode collating sequence, carrying leftward if necessary when the last digit or letter in the collating sequence is encountered

2: Built-In Types A range can be inclusive , meaning that it includes the end value, or exclusive , meaning that it does not include the end value Inclusive ranges are written with two dots between the extremes, and exclusive ranges with three dots Hence the Range of Integers from 1 to 10 exclusive is the set {1, 2, 3, , 9} A type can be a range base type only if it supports order comparisons For example, the Integer, Real, and String types support order comparisons and so may be range base types, but Sets and Arrays do not, so they cannot be range base types The carrier set of a Range of T is the set of all sets of values v ∈ T such that for some start and end values s ∈ T and e ∈ T , either s ≤ v and v ≤ e (the inclusive ranges), or s ≤ v and v < s (the exclusive ranges), plus the empty set For example, the carrier set of the Range of Integer is the set of all sequences of contiguous integers The carrier set of the Range of Real is the set of all sets of real number greater than or equal to a given number, and either less than or equal to another, or less than another These sets are called intervals in mathematics The operations of the Range of T ADT includes the following, where a , b ∈ T and r is a value of Range of T : a..b —returns a range value (an element of the carrier set) consisting of all v ∈ T such that a ≤ v and v ≤ b a...b —returns a range value (an element of the carrier set) consisting of all v ∈ T such that a ≤ v and v < b a == b —returns true if and only if a and b are identical min ( r )—returns the smallest value in r The result is undefined if r is the empty set max ( r )—returns the largest value in r The result is undefined if r has no largest value (for example, the Range of Real 0 3 has no largest value because there is no largest Real number less than 3) cover? ( r , x )—returns true if and only if x ∈ r. The Range of T ADT is a structured type because the values in its carrier set are composed of values of some other type, in this case, sets of value of the base type T Ruby implements the Range of T ADT in its Range class Elements of the carrier set are represented in Range instances by recording the type, start, and end values of the range, along with an indication of whether the range is inclusive or exclusive Ruby implements all the operations above, returning nil when the ADT operations are undefined It is quite easy to see how to implement these operations given the representation elements of the carrier set In addition, the Range class provides operations for accessing the begin and end values defining the range, which are easily accessible because they are recorded Finally, the Range class has an include?()operation that tests range membership by stepping through the values of the range from start value to end value when the range is non-numeric This gives slightly different results from cover?()in some cases (such as with String instances)

2: Built-In Types Review Questions 1 What is the difference between a simple and a structured type? 2 What is a pure object-oriented language? 3 Name two ways that Symbol instances differ from String instances in Ruby 4 Is String a simple or structured type in Ruby? Explain 5 List the carrier set of Range of {1, 2, 3} In this type, what values are 1 1, 2 1, and 1 3? What is max(1 3)? Exercises 1 Choose a language that you know well and list its simple and structures types 2 Choose a language that you know well and compare its simple and structured types to those of Ruby Does one language have a type that is simple while the corresponding type in the other language is structured? Which language has more simple types or more structured types? 3 Every Ruby type is a class, and every Ruby value is an instance of a class What advantage and disadvantages do you see with this approach? 4 Write pseudocode to implement the cover?() operation for the Range class in Ruby 5 Give an example of a Ruby String range r and String instance v such that r.cover?(v) and r.include?(v) differ Review Question Answers 1 The values of a simple type cannot be divided into parts, while the values of a structured type can be For example, the values of the Integer type in Ruby cannot be broken into parts, while the values of a Range in Ruby can be (the parts are the individual elements of the ranges) 2 A pure object-oriented language is one whose types are all classes Java and C++, for example, are not pure object-oriented languages because they include primitive data types, such as int, float, and char, that are not classes Smalltalk and Ruby are pure object-oriented languages because they have no such types 3 Symbol instances in Ruby are immutable while String instances are mutable Symbol instances consisting of a particular sequence of characters are unique, while there may be arbitrarily many String instances with the same sequence of characters 4 String is a simple type in Ruby because strings are not composed of other values—in Ruby there is no character type, so a String value cannot be broken down into parts composed of characters If s is a String instance, then s[0] is not a character, but another String instance 5 The carrier set of Range of {1, 2, 3} is { {}, {1}, {2}, {3}, {1, 2}, {2, 3}, {1, 2, 3} } The value 1 1 is {1}, the value 2 1 is {}, and the value 1 3 is {1, 2}, and max(1 3) is 2

3: Arrays a segment of memory for element storage Thereafter the array may shrink or grow If the array shrinks during execution, then only an initial portion of allocated memory is used But if the array grows beyond the space allocated for it, a more complex reallocation procedure must occur, as follows: 1 A new segment of memory large enough to store the elements of the expanded array is allocated 2 All elements of the original (unexpanded) array are copied into the new memory segment 3 The memory used initially to store array values is freed and the newly allocated memory is associated with the array variable or reference This reallocation procedure is computationally expensive, so systems are usually designed to minimize its frequency of use For example, when an array expands beyond its memory allocation, its memory allocation might be doubled even if space for only a single additional element is needed The hope is that providing a lot of extra space will avoid many expensive reallocation procedures if the array expands slowly over time Dynamic arrays are convenient for programmers because they can never be too small— whenever more space is needed in a dynamic array, it can simply be expanded One drawback of dynamic arrays is that implementing language support for them is more work for the compiler or interpreter writer A potentially more serious drawback is that the expansion procedure is expensive, so there are circumstances when using a dynamic array can be dangerous For example, if an application must respond in real time to events in its environment, and a dynamic array must be expanded when the application is in the midst of a response, then the response may be delayed too long, causing problems Arrays in Ruby Ruby arrays are dynamic arrays that expand automatically whenever a value is stored in a location beyond the current end of the array To the programmer, it is as if arrays are unbounded and as many locations as are needed are available Locations not assigned a value in an expanded array are initialized to nil by default Ruby also has an interesting indexing mechanism for arrays Array indices begin at 0 (as in many other languages) so, for example, a[13] is the value in the 14th^ position of the array Negative numbers are the indices of elements counting from the current end of the array, so a[-1] is the last element, a[-2] is the second to last element, and so forth Array references that use an out-of-bound index return nil These features combine to make it difficult to write an array reference that causes an indexing error This is apparently a great convenience to the programmer, but actually it is not because it makes it so hard to find bugs: many unintended and erroneous array references are legal The ability to assign arbitrary values to arrays that automatically grow arbitrarily large makes Ruby arrays behave more like lists than arrays in other languages We will discuss the List ADT later on Another interesting feature of Ruby arrays has to do with the fact that it is a pure object- oriented language This means (in part) that every value in Ruby is an object, and hence

3: Arrays every value in Ruby is an instance of Object, the super-class of all classes, or one of its sub- classes Arrays hold Object values, so any value can be stored in any array! For example, an array can store some strings, some integers, some floats, and so forth This appears to be a big advantage for programmers, but again this freedom has a price: it much harder to find bugs For example, in Java, mistakenly assigning a string value to an array holding integers is flagged by the compiler as an error, but in Ruby, the interpreter does not complain Ruby arrays have many interesting and powerful methods Besides indexing operations that go well beyond those discussed above, arrays have operations based on set operations (membership, intersection, union, and relative complement), string operations (concatenation, searching, and replacement), stack operations (push and pop), and queue operations (shift and append), as well as more traditional array-based operations (sorting, reversing, removing duplicates, and so forth) Arrays are also tightly bound up with Ruby’s iteration mechanism, which will be discussed later Review Questions 1 If an array holds integers, each of which is four bytes long, how many bytes from the base location of the array is the location of the fifth element? 2 Is the formula for finding the location of an element in a dynamic array different from the formula for finding the location of an element in a static array? 3 When a dynamic array expands, why can’t the existing elements be left in place and extra memory simply be allocated at the end of the existing memory allocation? 4 If a Ruby array a has n elements, which element is a[n-1]? Which is element a[-1]? Exercises 1 Suppose a dynamic integer array a with indices beginning at 0 has 1000 elements and the line of code a[1000] = a[5] is executed How many array values must be moved from one memory location to another to complete this assignment statement? 2 Memory could be freed when a dynamic array shrinks What advantages or disadvantages might this have? 3 To use a static array, data must be recorded about the base location of the array, the size of the elements (for indexing), and the number of elements in the array (to check that indexing is within bounds) What information must be recorded to use a dynamic array? 4 State a formula to determine how far the from base location of a Ruby array an element with index i is when i is a negative number 5 Give an example of a Ruby array reference that will cause an indexing error at run time 6 Suppose the Ruby assignment a=(1 .. 100).to_a is executed What are the values of the following Ruby expressions? Hint: You can check your answers with the Ruby interpreter (a) a[5..10] (b) a[5...10]