Algorithms and Data Structures: C949 Exam Study Guide, Exams of Data Structures and Algorithms

Download Algorithms and Data Structures: C949 Exam Study Guide and more Exams Data Structures and Algorithms in PDF only on Docsity!

C949 v4 Final Exam Study Guide on

Algorithms and Data Structures

C949v4 Study Guide

Competencies and Topics

Explains Algorithms - 29% of assessment

Characteristics of Algorithms

Finiteness An algorithm must always have a finite number of steps before it ends. When the operation is finished, it must have a defined endpoint or output and not enter an endless loop. Definiteness An algorithm needs to have exact definitions for each step. Clear and straightforward directions ensure that every step is understood and can be taken easily. Input An algorithm requires one or more inputs. The values that are first supplied to the algorithm before its processing are known as inputs. These inputs come from a predetermined range of acceptable values. Output One or more outputs must be produced by an algorithm. The output is the outcome of the algorithm after every step has been completed. The relationship between the input and the result should be clear. Effectiveness An algorithm's stages must be sufficiently straightforward to be carried out in a finite time utilizing fundamental operations. With the resources at hand, every operation in the algorithm should be doable and practicable. Generality Rather than being limited to a single particular case, an algorithm should be able to solve a group of issues. It should offer a generic fix that manages a variety of inputs inside a predetermined range or domain.

Factors of an Algorithm

● Modularity: This feature was perfectly designed for the algorithm if you are given a problem and break it down into small-small modules or small-small steps, which is a basic definition of an algorithm.

● Randomized Algorithm : Utilizes randomness in its steps to achieve a solution, often used in situations where an approximate or probabilistic answer suffices.

Recursive algorithms

Recursive algorithms are a fundamental concept in computer science, particularly in the study of data structures and algorithms. A recursive algorithm is one that solves a problem by breaking it down into smaller instances of the same problem, which it then solves in the same way. This process continues until the problem is reduced to a base case, which is solved directly without further recursion. Key Concepts of Recursive Algorithms

1. Base Case : This is the condition under which the recursion stops. It represents the simplest instance of the problem, which can be solved directly without further recursion.
2. Recursive Case : This is the part of the algorithm that breaks the problem down into smaller instances of the same problem and then calls the algorithm recursively on these smaller instances.
3. Stack : Each recursive call is placed on the system call stack. When the base case is reached, the stack begins to unwind as each instance of the function returns its result. Example: Factorial Calculation The factorial of a number n (denoted as n!) is a classic example of a recursive algorithm. The factorial is defined as:

O! = 1 (Base Case) ● N! = n * (n-1)! For n > O (Recursive Case) Here’s how it looks in code: def factorial(n): if n == 0 : # Base Case return 1 else: # Recursive Case return n * factorial(n - 1 ) How It Works: ● Base Case : When n is 0, the function returns 1. ● Recursive Case : For any other value of n, the function calls itself with n−1 and multiplies the result by n. For example, calling factorial(3) would work as follows: ● factorial(3) calls factorial(2) ● factorial(2) calls factorial(1) ● factorial(1) calls factorial(0) ● factorial(0) returns 1, then: ● factorial(1) returns 1 * 1 = 1 ● factorial(2) returns 2 * 1 = 2 ● factorial(3) returns 3 * 2 = 6 Advantages of Recursion ● Simplicity : Recursive solutions are often more elegant and easier to understand than their iterative counterparts. ● Direct Translation : Some problems are naturally recursive, like tree traversals, making recursion the most straightforward approach. Disadvantages of Recursion ● Performance : Recursive algorithms can be less efficient due to the overhead of multiple function calls and potential stack overflow issues for deep recursion. ● Memory Usage : Recursion can consume more memory because each function call adds a new frame to the call stack. When to Use Recursion When a problem can naturally be divided into similar sub-problems (e.g., tree traversal, searching algorithms like binary search). ● When the recursive solution is significantly simpler or more intuitive than an iterative one.

2. Find the middle element of the current interval.
3. Compare the middle element with the target: ○ If they match, return the index of the middle element. ○ If the target is less than the middle element, repeat the search on the left half. ○ If the target is greater, repeat the search on the right half.
4. If the interval becomes invalid (low > high), return a "not found" indication. Time Complexity : O(log n), where n is the number of elements in the array. This logarithmic time complexity makes binary search significantly faster than linear search for large datasets. When to Use : ● When the array or list is sorted. ● When the array is large and efficiency is crucial. Example: def binary_search(arr, target): low = 0 high = len(arr) - 1 while low <= high: mid = (low + high) // 2 if arr[mid] == target: return mid elif arr[mid] < target: low = mid + 1 else: high = mid - 1

return - 1 # Return - 1 if the element is not found Comparison ● Efficiency : Binary search is faster than linear search, especially for large datasets, but it requires the array to be sorted. ● Simplicity : Linear search is simpler to implement and doesn't require the array to be sorted, making it more versatile for smaller or unsorted datasets. ● Use Cases : ○ Linear Search : Suitable for small or unsorted collections where the simplicity of the algorithm outweighs the need for speed. ○ Binary Search : Ideal for large, sorted collections where performance is a priority.

Step-By-Step Guide To Figuring Out The Array Elements Corresponding To The

Mid-Values In The First And Second Iterations Of A Binary Search arr = {45, 77,

89, 90, 94, 99, 100} and key = 100 Initial Setup : ● The array arr is {45, 77, 89, 90, 94, 99, 100}. ● The key to find is 100. ● Initialize two pointers: low (start of the array) and high (end of the array). First Iteration : ● Calculate the middle index mid using the formula: mid = (low + high) / 2 ● Check the value at arr[mid]. ● Compare arr[mid] with the key: ○ If arr[mid] is less than key, update low to mid + 1. ○ If arr[mid] is greater than key, update high to mid - 1. ○ If arr[mid] is equal to key, you have found the key (though you won't need a second iteration in this case). Second Iteration : ● Repeat the calculation for mid with the updated low and high values. ● Again, compare arr[mid] with the key and update low or high accordingly.

○ Best Case : O(1) — The target element is exactly where the interpolation suggests. ○ Average Case : O(log log n) — Uniformly distributed data. ○ Worst Case : O(n) — Highly skewed data distribution or worst interpolation.

Depth-First Search (DFS) and Breadth-First Search (BFS)

● Concept : Used primarily in graph and tree data structures. DFS explores as far as possible along one branch before backtracking, while BFS explores all neighbors at the present depth before moving on to nodes at the next depth level. ● Time Complexity : O(V+E), where V is the number of vertices and E is the number of edges. ● Use Case : Useful for searching nodes in graphs and trees. ● Depth-First Search (DFS) Worst, Average and Best ○ Best Case : O(1) — The target node is found immediately. ○ Average Case : O(V+E)— Typically when all nodes and edges must be explored. ○ Worst Case : O(V+E) — The target node is the last one discovered. ● Breadth-First Search (BFS) Worst, Average and Best ○ Best Case : O(1) — The target node is the root or the first node checked. ○ Average Case : O(V+E) — All nodes and edges need to be explored. ○ Worst Case : O(V+E) — The target node is the last one explored.

Sorting Algorithms

Sorting algorithms organize data in a particular order (usually ascending or descending). This makes

searching and other operations more efficient. Bubble Sort

● Use Case : Simple but inefficient for large datasets. Best used for educational purposes or small lists. ● Distinct Characteristics : ● Repeatedly swaps adjacent elements if they are in the wrong order. ● Simple, but inefficient for large datasets. ● “Bubbles” the largest element to the end of the list. ● Bubble Sort Worst, Average and Best ○ Best Case : O(n) — The array is already sorted (with an optimized version that stops early). ○ Average Case : O(n^2 ) — Average case with random elements. ○ Worst Case : O(n^2 ) — The array is sorted in reverse order. ● Exchange)

Selection Sort

● Use Case : Inefficient for large lists, but useful when memory writes are more expensive than comparisons. Bubble : Look for something that swaps so the result can “bubble” to the top. (Swap,

● Distinct Characteristics : ● Finds the minimum element and swaps it with the first unsorted element. ● Reduces the problem size by one in each iteration. ● Always performs O(n^2 ) comparisons, regardless of input. ● Selection Sort Worst, Average and Best ○ Best Case : O(n^2 ) — Selection sort does not improve with better input, always O(n^2 ). ○ Average Case : O(n^2 ) — Average case with random elements. ○ Worst Case : O(n^2 ) — Selection sort is insensitive to input order. ● Selection: Look for code that repeatedly finds the minimum (or maximum) element and moves it to the beginning (or end) of the list. (Select minimum, Swap with start)

Insertion Sort

● Use Case : Good for small or nearly sorted lists. ● Distinct Characteristics : ● Builds a sorted list one element at a time. ● Efficient for small or nearly sorted datasets. ● Shifts elements to make space for the current element. ● Insertion Sort Worst, Average and Best ○ Best Case : O(n) — The array is already sorted. ○ Average Case : O(n^2 ) — Average case with random elements. ○ Worst Case : O(n^2 ) — The array is sorted in reverse order. ● Insertion: Look for code that builds a sorted portion of the list one element at a time by inserting each new element into its correct position within the alreadysorted part. (Insert, Shift Element)

Merge Sort

● Use Case : Efficient and stable; good for large datasets. ● Distinct Characteristics : ● Divides the list into halves, sorts each half, and then merges them. ● Stable and efficient for large datasets. ● Requires additional space for merging. ● Merge Sort Worst, Average and Best

○ Average Case : O(n+k). ○ Worst Case : O(n+k).

Radix Sort

● Use Case : Effective for sorting large numbers or strings with a fixed length. ● Distinct Characteristics : ● Sorts numbers by processing individual digits. ● Non-comparative, stable, and efficient for specific data types. ● Often combined with counting sort. ● Radix Sort Worst, Average and Best ○ Best Case : O(nk) — k is the number of digits in the largest number. ○ Average Case : O(nk). ○ Worst Case : O(n*k). ● Radix Sort: Look for code that sorts numbers based on their individual digits, starting from the least significant digit (LSD) or the most significant digit (MSD). (Count, Frequency, Sum)

Bucket Sort

● Use Case : Good for uniformly distributed data. ● Distinct Characteristics : ● Distributes elements into buckets and sorts each bucket individually. ● Efficient when the input is uniformly distributed. ● Often combined with another sorting algorithm like insertion sort. ● Bucket Sort Worst, Average and Best ○ Best Case : O(n+k) — k is the number of buckets; assumes uniform distribution. ○ Average Case : O(n+k). ○ Worst Case : O(n^2 ) — All elements end up in one bucket (degenerate case). ● Bucket : Look for something that distributes the values into “buckets” where they are individually sorted. (Bucket)

Shell Sort

● Distinct Characteristics : ○ Generalization of insertion sort with a gap sequence. Sorts elements far apart and gradually reduces the gap. ○ Efficient for medium-sized datasets. ○ Time Complexity: Depends on the gap sequence; commonly O(n3/2). ● Shell Sort Worst, Average and Best ○ Best Case : O(n log n) — Occurs when the array is already sorted or nearly sorted, especially when using a good gap sequence like the Knuth sequence.

○ Average Case : O(n^(3/2)) or O(n^1.5) — Highly dependent on the gap sequence used. With commonly used sequences like the Knuth sequence, the average-case complexity is approximately O(n^1.5). ○ Worst Case : O(n^2 ) — Can degrade to O(n^2 ), particularly with poorly chosen gap sequences like the original Shell sequence (where the gaps are halved each time). ● Shell Sort: Look for code that sorts elements at specific intervals and gradually reduces the interval until it performs a final insertion sort. (Gap, Interval) Summary of all Searching & Sorting Algorithms ● Linear Search : Simple, sequential; O(n). ● Binary Search : Sorted data, divide and conquer; O(log n). ● Bubble Sort : Swaps, bubbles up; O(n^2). ● Selection Sort : Finds minimum, swaps; O(n^2). ● Insertion Sort : Builds sorted list, shifts; O(n^2), O(n) best case. ● Merge Sort : Divide and conquer, merge; O(n log n). ● Quick Sort : Pivot, partition; O(n log n) average, O(n^2) worst case. ● Heap Sort : Max-heap, extract max; O(n log n). ● Counting Sort : Counts occurrences, non-comparative; O(n + k). ● Radix Sort : Sorts by digits, non-comparative; O(nk). ● Bucket Sort : Distributes into buckets, sorts; O(n + k). ● Shell Sort : Gap sequence, insertion-like; O(n^3/2). Key Observations ● Bubble Sort, Selection Sort, and Insertion Sort : These are simple but inefficient for large datasets, especially in the worst case. ● Merge Sort and Heap Sort : Stable and consistent in performance, regardless of the input. ● Quick Sort : Very efficient on average but can degrade to O(n^2 ) in the worst case without proper pivot selection. ● Counting Sort, Radix Sort, and Bucket Sort : Efficient for specific types of data (e.g., integers within a fixed range) but less versatile. Choosing the Right Algorithm ● Small datasets : Simpler algorithms like bubble sort, selection sort, or insertion sort might suffice. ● Large datasets : More efficient algorithms like merge sort, quick sort, or heap sort are preferred. ● Sorted data : Algorithms like insertion sort can be very efficient. ● Special conditions : Use counting sort, radix sort, or bucket sort if the data is within a certain range or has other specific properties.

○ Description : The runtime doubles with each additional element in the input. Common in algorithms that solve problems by brute force or explore all possible solutions. ○ Example : Recursive algorithms for the Fibonacci sequence, certain dynamic programming problems. ○ Efficiency : Very poor, impractical for large inputs.

○ Description : The runtime increases factorially with the size of the input. Common in algorithms that generate all permutations of an input set. ○ Example : Traveling salesman problem via brute force. ○ Efficiency : Extremely poor, infeasible for even moderate input sizes. Time Complexity Best Case, Worst Case, and Average Case ● Best Case : The scenario where the algorithm performs the minimum possible number of operations. It’s often less relevant because it’s optimistic. ○ Example : For linear search, the best case is O(1), where the target element is the first one in the array. ● Worst Case : The scenario where the algorithm performs the maximum possible number of operations. Big O notation typically describes the worst-case complexity. ○ Example : For linear search, the worst case is O(n), where the target element is the last one in the array or isn’t present at all. ● Average Case : The scenario that represents the expected number of operations for a typical input. It’s more complex to calculate because it depends on the distribution of inputs. ○ Example : For linear search, the average case is O(n/2), but in Big O notation, we simplify this to O(n).

Calculation Rules

Ignore Constants: ● Rule : In Big O notation, constant factors are ignored. ● Why : Big O notation focuses on the growth rate as the input size (n) increases, so a constant multiplier doesn't affect the growth rate. ● Example : O(2n) simplifies to O(n) Focus on the Dominant Term: ● Rule : Only the term with the highest growth rate is considered. ● Why : As n becomes large, the term with the highest growth rate will dominate the others. ● Example : O(n^2 +n) simplifies to O(n^2 ) O(2n) - Exponential Time : O(n!) - Factorial Time :

Drop Lower Order Terms: ● Rule : Lower-order terms are ignored because they become insignificant as n grows. ● Why : Similar to focusing on the dominant term, lower-order terms have a negligible impact on large inputs. ● Example : O(n^2 + n + log n) simplifies to O(n^2 ) Multiplicative Constants Can Be Ignored: ● Rule : Coefficients that multiply variables (e.g., 2n, 3n^2) are ignored. ● Why : Like constants, they don't change the growth rate. ● Example : O(3n^2 ) simplifies to O(n^2 ) Additive Constants Can Be Ignored: ● Rule : Constant terms that don't depend on n are ignored. ● Why : They don't affect the overall growth rate as n increases. ● Example : O(n+10) simplifies to O(n) Logarithms with Different Bases: ● Rule : Logarithms with different bases can be considered equivalent in Big O notation. ● Why : Changing the base of a logarithm only introduces a constant factor, which is ignored in Big O notation. ● Example : O(log 2 n) simplifies to O(log n) Non-Dominant Polynomial Terms:

■ Arrays : A collection of elements of the same data type. ■ Structures (e.g., struct in C): A collection of different data types. ○ User-Defined Data Types : Created by the user, typically by combining primitive data types. ■ Enumerations (enum), Classes , etc. Enumeration: ● Enumeration, often referred to as "enum," is a data type in programming that allows a variable to be a set of predefined constants. ● These constants are typically related and represent a set of possible values that a variable of the enumeration type can hold. ● Enums improve code readability, make it easier to manage sets of related values, and reduce errors by limiting the values a variable can take. Key Concepts of Enumeration:

1. Definition : An enumeration is defined using the enum keyword (syntax can vary by language). It consists of a set of named constants, each representing a unique value.
2. Values : The values in an enum are usually integers by default, starting from 0, but they can be assigned specific values as needed.
3. Usage : Enums are commonly used when a variable can only take one out of a small set of possible values, like days of the week, directions, states, etc. Example in Different Languages: C/C++ enum Direction { NORTH, EAST, SOUTH, WEST }; Direction dir = NORTH; Java public enum Direction { NORTH, EAST, SOUTH,

WEST

Direction dir = Direction.NORTH; Python (using enum module) from enum import Enum class Direction(Enum): NORTH = 1 EAST = 2 SOUTH = 3 WEST = 4 dir = Direction.NORTH Advantages of Using Enums: ● Readability : Code is easier to read and understand. ● Maintainability : Easier to update and maintain related values. ● Type Safety : Prevents assigning invalid values to variables of the enum type. Enums are useful in scenarios where a variable should only be allowed to take one out of a small set of specific values, helping prevent errors and making the code clearer and more reliable. Key Characteristics: ● Memory Allocation : Data types determine how much memory is allocated for storing the data. ● Operations : Each data type supports a set of operations, like arithmetic operations for integers or concatenation for strings. Data Structures What is a Data Structure? ● Definition : A data structure is a specific way of organizing and storing data in a computer so that it can be accessed and modified efficiently. Data structures use data types as their underlying foundation. ● Examples : ○ Arrays : A collection of elements stored in contiguous memory locations. ○ Linked Lists : A series of connected nodes, where each node contains data and a reference to the next node. ○ Stacks : A collection of elements with Last-In-First-Out (LIFO) access. ○ Queues : A collection of elements with First-In-First-Out (FIFO) access. ○ Trees : A hierarchical structure with a root element and sub-elements called nodes.