Download Algorithm Analysis: Asymptotic Notations, GCD Computation, and Backtracking and more Study notes Design and Analysis of Algorithms in PDF only on Docsity!
UNIT – 1
INTRODUCTION
1.1 Notion of Algorithm 1.2 Review of Asymptotic Notation 1.3 Mathematical Analysis of Non-Recursive and Recursive Algorithms 1.4 Brute Force Approaches: Introduction 1.5 Selection Sort and Bubble Sort 1.6 Sequential Search and Brute Force String Matching.
1.1 Notion of Algorithm
Need for studying algorithms: The study of algorithms is the cornerstone of computer science. It can be recognized as the core of computer science. Computer programs would not exist without algorithms. With computers becoming an essential part of our professional & personal life‘s, studying algorithms becomes a necessity, more so for computer science engineers. Another reason for studying algorithms is that if we know a standard set of important algorithms, They further our analytical skills & help us in developing new algorithms for required applications Algorithm An algorithm is finite set of instructions that is followed, accomplishes a particular task. In addition, all algorithms must satisfy the following criteria:
- Input. Zero or more quantities are externally supplied.
- Output. At least one quantity is produced.
- Definiteness. Each instruction is clear and produced.
- Finiteness. If we trace out the instruction of an algorithm, then for all cases, the algorithm terminates after a finite number of steps.
- Effectiveness. Every instruction must be very basic so that it can be carried out, in principal, by a person using only pencil and paper. It is not enough that each operation be definite as in criterion 3; it also must be feasible.
An algorithm is composed of a finite set of steps, each of which may require one or more op- erations. The possibility of a computer carrying out these operations necessitates that certain constraints be placed on the type of operations an algorithm can include. The fourth criterion for algorithms we assume in this book is that they terminate after a finite number of opera- tions.
Criterion 5 requires that each operation be effective; each step must be such that it can, at least in principal, be done by a person using pencil and paper in a finite amount of time. Performing arithmetic on integers is an example of effective operation, but arithmetic with real numbers is not, since some values may be expressible only by infinitely long decimal expansion. Adding two such numbers would violet the effectiveness property.
- Algorithms that are definite and effective are also called computational procedures.
- The same algorithm can be represented in same algorithm can be represented in several ways
- Several algorithms to solve the same problem
- Different ideas different speed Example: Problem:GCD of Two numbers m,n Input specifiastion :Two inputs,nonnegative,not both zero Euclids algorithm -gcd(m,n)=gcd(n,m mod n) Untill m mod n =0,since gcd(m,0) =m Another way of representation of the same algorithm Euclids algorithm
algorithm independent of the hardware/software environment. Therefore theoretical analysis can be used for analyzing any algorithm
Framework for Analysis
We use a hypothetical model with following assumptions
- Total time taken by the algorithm is given as a function on its input size
- Logical units are identified as one step
- Every step require ONE unit of time
- Total time taken = Total Num. of steps executed Input’s size: Time required by an algorithm is proportional to size of the problem instance. For e.g., more time is required to sort 20 elements than what is required to sort 10 elements. Units for Measuring Running Time: Count the number of times an algorithm‘s basic operation is executed. ( Basic operation : The most important operation of the algorithm, the operation contributing the most to the total running time.) For e.g., The basic operation is usually the most time- consuming operation in the algorithm‘s innermost loop. Consider the following example:
ALGORITHM sum_of_numbers ( A[0… n-1] ) // Functionality : Finds the Sum // Input : Array of n numbers // Output : Sum of „n‟ numbers i 0 sum 0 while i < n sum sum + A[i] n i i + 1 return sum
Total number of steps for basic operation execution, C (n) = n NOTE: Constant of fastest growing term is insignificant: Complexity theory is an Approximation theory. We are not interested in exact time required by an algorithm to solve the problem. Rather we are interested in order of growth. i.e How much faster will algorithm run on computer that is twice as
fast? How much longer does it take to solve problem of double input size? We can crudely estimate running time by T (n) ≈ Cop � C (n) Where, T (n): running time as a function of n. Cop : running time of a single operation. C (n): number of basic operations as a function of n. Order of Growth: For order of growth, consider only the leading term of a formula and ignore the constant coefficient. The following is the table of values of several functions important for analysis of algorithms.
Worst-case, Best-case, Average case efficiencies Algorithm efficiency depends on the input size n. And for some algorithms efficiency depends on type of input. We have best, worst & average case efficiencies.
- Worst-case efficiency: Efficiency (number of times the basic operation will be executed) for the worst case input of size n. i.e. The algorithm runs the longest among all possible inputs of size n.
- Best-case efficiency: Efficiency (number of times the basic operation will be executed) for the best case input of size n. i .e. The algorithm runs the fastest among all possible inputs of size n.
- Average-case efficiency: Average time taken (number of times the basic operation will be executed) to solve all the possible instances (random) of the input. NOTE: NOT the average of worst and best case
Big Theta- Θ notation
Definition: A function t (n) is said to be in Θ(g (n)), denoted t(n) = Θ (g (n)), if t (n) is bounded both above nda below by some constant multiple of g (n) for all large n, i.e., if there exist some positive constant andc c2 and some nonnegative integer n0 such that
c2 g (n) ≤ t (n) ≤ c1 g (n) for all n ≥ n
Basic Efficiency classes
The time efficiencies of a large number of algorithms fall into only a few classes.
fast
low time efficiency slow
1.2 Mathematical Analysis of Non-Recursive and Recursive Algorithms
Mathematical analysis (Time Efficiency) of Non-recursive Algorithms General plan for analyzing efficiency of non-recursive algorithms:
- Decide on parameter n indicating input size
- Identify algorithm‘s basic operation
- Check whether the number of times the basic operation is executed depends only on the input size n. If it also depends on the type of input, investigate worst, average, and best case efficiency separately.
- Set up summation for C(n) reflecting the number of times the algorithm‘s basic operation is executed. 5. Simplify summation using standard formulas Example: Finding the largest element in a given array ALOGORITHM MaxElement(A[0 ..n-1]) //Determines the value of largest element in a given array //nput: An array A[0.. n-1] of real numbers //Output: The value of the largest element in A currentMax ← A[0] for i ← 1 to n - 1 do if A[i] > currentMax
log n n n log n n^2 n^3 2 n n!
constant^ High^ time^ efficiency logarithmic linear n log n quadratic cubic exponential factorial
Analysis
- Input size: number of elements = n (size of the array)
- Basic operation: Comparison
- Best, worst, average cases EXISTS. Worst case input is an array giving largest comparisons.
- Array with no equal elements
- Array with last two elements are the only pair of equal elements
- Let C ( n ) denotes number of comparisons in worst case: Algorithm makes one comparison for each repetition of the innermost loop i.e., for each value of the loop‘s variable j between its limits i + 1 and n – 1 ; and this is repeated for each v alue of the outer loop i.e, for each value of the loop‘s variable i between its limits 0 and n – 2
Mathematical analysis (Time Efficiency) of recursive Algorithms
General plan for analyzing efficiency of recursive algorithms:
1. Decide on parameter n indicating input size 2. Identify algorithm‘s basic operation
- Check whether the number of times the basic operation is executed depends only on the input size n. If it also depends on the type of input, investigate worst, average, and best case efficiency separately.
- Set up recurrence relation , with an appropriate initial condition, for the number of times the algorithm‘s basic operation is executed.
- Solve the recurrence.
Example: Factorial function ALGORITHM Factorial (n) //Computes n! recursively //Input: A nonnegative integer n //Output: The value of n! if n = = 0
Example: Find the number of binary digits in the binary representation of a positive decimal integer ALGORITHM BinRec (n) //Input: A positive decimal integer n //Output: The number of binary digits in n‟s binary representation if n = = 1 return 1 else return BinRec (└ n/2 ┘) + 1
Analysis:
- Input size: given number = n
- Basic operation: addition
- NO best, worst, average cases.
- Let A ( n ) denotes number of additions. A ( n ) = A (└ n/2 ┘) + 1 for n > 1 A (1) = 0 initial condition Where: A (└ n/2 ┘) : to compute BinRec (└ n/2 ┘) 1 : to increase the returned value by 1
- Solve the recurrence: A ( n ) = A (└ n/2 ┘) + 1 for n > 1 Assume n = 2k^ (smoothness rule) A (2k) = A (2k-1) + 1 for k > 0; A (20) = 0 Solving using “Backward substitution method”: A (2k) = A (2k-1) + 1 = [A (2k-2) + 1] + 1 = A (2k-2) + 2 = [A (2k-3) + 1] + 2 = A (2k-3) + 3 … In the ith recursion, we have = A (2k-i) + i
When i = k, we have = A (2k-k) + k = A (2^0 ) + k Since A (2^0 ) = 0 A (2k) = k Since n = 2k, HENCE k = log 2 n A ( n ) = log 2 n A ( n ) = Θ ( log n)
1.2 Brute Force Approaches:
Introduction
Brute force is a straightforward approach to problem solving, usually directly based on the problem‘s statement and definitions of the concepts involved.Though rarely a source of clever or algorithms,the brute-force approach should not be overlooked as an strategies, brute force is applicable to a very wide variety of problems.For some important problems (e.g., sorting, searching, string matching),the brute-force approach yields reasonable algorithms of at least some practical value with no limitation on instance size Even if too inefficient in general, a brute-force algorithm can still be useful for solving small-size instances of a problem. A brute-force algorithm can serve an important theoretical or educational purpose.
1.3 Selection Sort and Bubble Sort
Problem:Given a list of n orderable items (e.g., numbers, characters from some alphabet, character strings), rearrange them in nondecreasing order. Selection Sort ALGORITHM SelectionSort(A [0 .. n - 1] ) //The algorithm sorts a given array by selection sort //Input: An array A [0 .. n - 1] of orderable elements //Output: Array A [0.. n - 1] sorted in ascending order for i= 0 to n - 2 do min=i for j=i + 1 to n - 1 do if A [ j ] <A [ min ] min=j
efficient^ important^ algorithm design strategy. Unlike some of the other
if A [ j + 1] <A [ j ] swap A [ j ] and A [ j + 1] Example
The first 2 passes of bubble sort on the list 89, 45, 68, 90, 29, 34, 17. A new line is shown after a swap of two elements is done. The elements to the right of the vertical bar are in their final positions and are not considered in subsequent iterations of the algorithm
Bubble Sort the analysis Clearly, the outer loop runs n times. The only complexity in this analysis in the inner loop. If we think about a single time the inner loop runs, we can get a simple bound by noting that it can never loop more than n times. Since the outer loop will make the inner loop complete n times, the comparison can't happen more than O(n^2 ) times. The number of key comparisons for the bubble sort version given above is the same for all arrays of size n.
The number of key swaps depends on the input. For the worst case of decreasing arrays, it is the same as the number of key comparisons.
Observation: if a pass through the list makes no exchanges, the list has been sorted and we can stop the algorithm Though the new version runs faster on some inputs, it is still in O (n^2 ) in the worst and average cases. Bubble sort is not very good for big set of input. How ever bubble sort is very simple to code. General Lesson From Brute Force Approach A first application of the brute-force approach often results in an algorithm that can be improved with a modest amount of effort. Compares successive elements of a given list with a given search key until either a match is encountered (successful search) or the list is exhausted without finding a match (unsuccessful search)
1. 4 Sequential Search and Brute Force String Matching.
Sequential Search
ALGORITHM SequentialSearch2(A [0 ..n ], K) //The algorithm implements sequential search with a search key as a sentinel //Input: An array A of n elements and a search key K //Output: The position of the first element in A [0.. n - 1] whose value is // equal to K or -1 if no such element is found A [ n ]= K i= 0 while A [ i ] = K do i=i + 1 if i < n return i else return
Brute-Force String Matching
Given a string of n characters called the text and a string of m characters ( m = n) called the pattern , find a substring of the text that matches the pattern. To put it more precisely, we want to find i —the index of the leftmost character of the first matching
UNIT - 2
DIVIDE & CONQUER
1.1 Divide and Conquer
1.2 General Method
1.3 Binary Search
1.4 Merge Sort
1.5 Quick Sort and its performance
1.1 Divide and Conquer
Definition: Divide & conquer is a general algorithm design strategy with a general plan as follows:
- DIVIDE: A problem‘s instance is divided into several smaller instances of the same problem, ideally of about the same size.
- RECUR: Solve the sub-problem recursively.
- CONQUER: If necessary, the solutions obtained for the smaller instances are combined to get a solution to the original instance.
Diagram 1 shows the general divide & conquer plan
Problem of size n Problem of size n Problemof size n
Solution to sub Solution to sub problem 1 problem 1
Solution to original problem
NOTE:
The base case for the recursion is sub-problem of constant size.
Advantages of Divide & Conquer technique:
- For solving conceptually difficult problems like Tower Of Hanoi, divide & conquer is a powerful tool
- Results in efficient algorithms
- Divide & Conquer algorithms are adapted foe execution in multi-processor machines
- Results in algorithms that use memory cache efficiently.
Limitations of divide & conquer technique:
- Recursion is slow
- Very simple problem may be more complicated than an iterative approach. Example: adding n numbers etc
1.2 General Method
General divide & conquer recurrence: An instance of size n can be divided into b instances of size n/b, with ―a‖ of them needing to be solved. [ a ≥ 1, b > 1]. Assume size n is a power of b. The recurrence for the running time T(n) is as follows:
T(n) = aT(n/b) + f(n) where: f(n) – a function that accounts for the time spent on dividing the problem into smaller ones and on combining their solutions
Therefore, the order of growth of T(n) depends on the values of the constants a & b and the order of growth of the function f(n).
Master theorem
Theorem: If f(n) Є Θ (nd) with d ≥ 0 in recurrence equation T(n) = aT(n/b) + f(n), then
Θ (nd) if a < bd T(n) = Θ (ndlog n) if a = bd Θ (nlogba^ ) if a > bd
Example:
Let T(n) = 2T(n/2) + 1, solve using master theorem. Solution: Here: a = 2 b = 2 f(n) = Θ(1)
