Dsa ch10 2 | Study notes Data Structures and Algorithms

Data$structure$and$Algorithm$(BEIT$III$&$BESE$III)$

Nepal$College$of$Information$Technology$

Prepared$by:$Madan$Kadariya$

Page 1 of 5!

Chapter 10: Algorithms

10.1. Deterministic and Non-deterministic algorithm

Deterministic algorithms can be defined in terms of a state machine: a state describes what a machine

is doing at a particular instant in time. State machines pass in a discrete manner from one state to

another. Just after we enter the input, the machine is in its initial state or start state. If the machine is

deterministic, this means that from this point onwards, its current state determines what its next state

will be; its course through the set of states is predetermined. Note that a machine can be deterministic

and still never stop or finish, and therefore fail to deliver a result.

A nondeterministic algorithm is one in which for a given input instance each intermediate step has one

or more possibilities. This means that there may be more than one path from which the algorithm may

arbitrarily choose one. Not all paths terminate successfully to give the desired output. The

nondeterministic algorithm works in such a way so as to always choose a path that terminates

successfully, thus always giving the correct result.

What makes algorithms non-deterministic?

Ø If it uses external state other than the input, such as user input, a global variable, a hardware

timer value, a random value, or stored disk data.

Ø If it operates in a way that is timing-sensitive, for example if it has multiple processors writing

to the same data at the same time. In this case, the precise order in which each processor writes

its data will affect the result.

Ø If a hardware error causes its state to change in an unexpected way.

Procedures of a Nondeterministic Algorithm:

The nondeterministic algorithm uses three basic procedures as follows:

1. CHOICE(1,n) or CHOICE(S) : This procedure chooses and returns an arbitrary element, in favour

of the algorithm, from the closed interval [1,n] or from the set S.

2. SUCCESS : This procedure declares a successful completion of the algorithm.

3. FAILURE : This procedure declares an unsuccessful termination of the algorithm.

10.2. Divide and conquer algorithm

Many useful algorithms are recursive in structure: to solve a given problem, they call themselves

recursively one or more times to deal with closely related sub problems. These algorithms typically

follow a divide-and-conquer approach: they break the problem into several sub problems that are

similar to the original problem but smaller in size, solve the sub problems recursively, and then

combine these solutions to create a solution to the original problem.

The divide-and-conquer paradigm involves three steps at each level of the recursion:

Ø Divide the problem into a number of sub problems.

Ø Conquer the sub problems by solving them recursively. If the sub problem sizes are small

enough, however, just solve the sub problems in a straightforward manner.

Ø Combine the solutions to the sub problems into the solution for the original problem.

The merge sort algorithm closely follows the divide-and-conquer paradigm. Intuitively, it operates as

follows.

Ø Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements

each.

Ø Conquer: Sort the two subsequences recursively using merge sort.

Ø Combine: Merge the two-sorted subsequences to produce the sorted answer.

The recursion "bottoms out" when the sequence to be sorted has length 1, in which case there is no

work to be done, since every sequence of length 1 is already in sorted order.

The key operation of the merge sort algorithm is the merging of two sorted sequences in the

Partial preview of the text

Download Dsa ch10 2 and more Study notes Data Structures and Algorithms in PDF only on Docsity!

Nepal College of Information Technology Chapter 10: Algorithms 10.1. Deterministic and Non-deterministic algorithm Deterministic algorithms can be defined in terms of a state machine: a state describes what a machine is doing at a particular instant in time. State machines pass in a discrete manner from one state to another. Just after we enter the input, the machine is in its initial state or start state. If the machine is deterministic, this means that from this point onwards, its current state determines what its next state will be; its course through the set of states is predetermined. Note that a machine can be deterministic and still never stop or finish, and therefore fail to deliver a result. A nondeterministic algorithm is one in which for a given input instance each intermediate step has one or more possibilities. This means that there may be more than one path from which the algorithm may arbitrarily choose one. Not all paths terminate successfully to give the desired output. The nondeterministic algorithm works in such a way so as to always choose a path that terminates successfully, thus always giving the correct result. What makes algorithms non-deterministic? Ø If it uses external state other than the input, such as user input, a global variable, a hardware timer value, a random value, or stored disk data. Ø If it operates in a way that is timing-sensitive, for example if it has multiple processors writing to the same data at the same time. In this case, the precise order in which each processor writes its data will affect the result. Ø If a hardware error causes its state to change in an unexpected way. Procedures of a Nondeterministic Algorithm: The nondeterministic algorithm uses three basic procedures as follows:

CHOICE(1,n) or CHOICE(S) : This procedure chooses and returns an arbitrary element, in favour of the algorithm, from the closed interval [1,n] or from the set S.
SUCCESS : This procedure declares a successful completion of the algorithm.
FAILURE : This procedure declares an unsuccessful termination of the algorithm. 10.2. Divide and conquer algorithm Many useful algorithms are recursive in structure: to solve a given problem, they call themselves recursively one or more times to deal with closely related sub problems. These algorithms typically follow a divide-and-conquer approach: they break the problem into several sub problems that are similar to the original problem but smaller in size, solve the sub problems recursively, and then combine these solutions to create a solution to the original problem. The divide-and-conquer paradigm involves three steps at each level of the recursion: Ø Divide the problem into a number of sub problems. Ø Conquer the sub problems by solving them recursively. If the sub problem sizes are small enough, however, just solve the sub problems in a straightforward manner. Ø Combine the solutions to the sub problems into the solution for the original problem. The merge sort algorithm closely follows the divide-and-conquer paradigm. Intuitively, it operates as follows. Ø Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each. Ø Conquer: Sort the two subsequences recursively using merge sort. Ø Combine: Merge the two-sorted subsequences to produce the sorted answer. The recursion "bottoms out" when the sequence to be sorted has length 1, in which case there is no work to be done, since every sequence of length 1 is already in sorted order. The key operation of the merge sort algorithm is the merging of two sorted sequences in the

Nepal College of Information Technology "combine" step. To perform the merging, we use an auxiliary procedure MERGE(A, p, q, r), where A is an array and p, q, and r are indices numbering elements of the array such that p ≤ q < r. The procedure assumes that the subarrays A[p ☐ q] and A[q + 1 ☐ r] are in sorted order. It merges them to form a single sorted subarray that replaces the current subarray A[p ☐ r]. 10.3. Series and Parallel Algorithm In computer science, a parallel algorithm or concurrent algorithm, as opposed to a traditional sequential (or serial) algorithm, is an algorithm which can be executed a piece at a time on many different processing devices, and then combined together again at the end to get the correct result. Some algorithms are easy to divide up into pieces in this way. For example, splitting up the job of checking all of the numbers from one to a hundred thousand to see which are primes could be done by assigning a subset of the numbers to each available processor, and then putting the list of positive results back together. Algorithms are also used for things such as rubik's cubing and for hash decryption. Most of the available algorithms to compute pi (π), on the other hand, cannot be easily split up into parallel portions. They require the results from a preceding step to effectively carry on with the next step. Such problems are called inherently serial problems. Some problems are very difficult to parallelize, although they are recursive. One such example is the depth-first search of graphs. Parallel algorithms are valuable because of substantial improvements in multiprocessing systems and the rise of multi-core processors. In general, it is easier to construct a computer with a single fast processor than one with many slow processors with the same throughput. But processor speed is increased primarily by shrinking the circuitry, and modern processors are pushing physical size and heat limits. These twin barriers have flipped the equation, making multiprocessing practical even for small systems. The cost or complexity of serial algorithms is estimated in terms of the space (memory) and time (processor cycles) that they take. Parallel algorithms need to optimize one more resource, the communication between different processors. There are two ways parallel processors communicate, shared memory or message passing. Shared memory processing needs additional locking for the data, imposes the overhead of additional processor and bus cycles, and also serializes some portion of the algorithm. Message passing processing uses channels and message boxes but this communication adds transfer overhead on the bus, additional memory need for queues and message boxes and latency in the messages. 10.4. Heuristic and Approximate Algorithm Many important computational problems are di!cult to solve optimally. In fact, many of those problems are NP-hard1,which means that no polynomial-time algorithm exists that solves the problem optimally unless P=NP. A well-known example is the Euclidean traveling salesman problem (Euclidean TSP): given a set of points in the plane, find a shortest tour that visits all the points. Another famous NP-hard problem is independent set: given a graph G = (V, E), find a maximum-size independent set V ∗ ⊂ V. (A subset is independent if no two vertices in the subset are connected by an edge.) What can we do when faced with such di!cult problems, for which we cannot expect to find polynomial-time algorithms? Unless the input size is really small, an algorithm with exponential running time is not useful. We therefore have to give up on the requirement that we always solve the problem optimally, and settle for a solution close to optimal. Ideally, we would like to have a guarantee on how close to optimal the solution is. For example, we would like to have an algorithm for Euclidean TSP that always produces a tour whose length is at most a factor ρ times the minimum length of a tour, for a (hopefully small) value of ρ. We call an algorithm producing a solution that is guaranteed to be within some factor of the optimum an approximation algorithm. This is in contrast to

Nepal College of Information Technology Following table shows the growth rates for a given N. 10.5.2. Estimating the growth Rate Algorithms are developed in a structured way; they combine simple statements into complex blocks in four ways: Ø Sequence, writing one statement below another Ø Decision, if-then or if-then-else Ø Loops Ø Subprogram call Let us estimate the big-O of some algorithm structures. Simple statements: We assume that statement does not contain a function call. It takes a fixed amount to execute. We denote the performance by O(1), if we factor out the constant execution time we are left with 1. Sequence of simple statements : It takes an amount of execution time equal to the sum of execution times of individual statements. If the performance of individual statements are O(1), so is their sum. Decision : For estimating the performance, then and else parts of the algorithm are considered independently. The performance estimate of the decision is taken to be the largest of the two individual big Os. For the case structure, we take the largest big O of all the case alternatives. Simple counting loop : This is the type of loop in which the counter is incremented or decrement each time the loop is executed (for loop). If the loop contains simple statements and the number of times the loop executes is a constant; in other words, independent of the problem size then the performance of the whole loop is O(1). On the other hand if the loop is like Eg: for (i=0; i< N; i++) The number of trips depends on N; the input size, so the performance is O(N). Nested loops : The performance depends on the counters at each nested loop. For ex: for (i=0; i< N; i++) { for (j=0; j< N; i++) { sequence of simple statements } } The outer loop count is N but the inner loop executes N times for each time. So the body of the inner loop will execute N*N and the entire performance will be O(N 2 ). for (i=1; i<=N; i++) { for (j=0; j< i; j++) {

Nepal College of Information Technology sequence of simple statement} } In this case outer count trip is N, but the trip count of the inner loop depends not only N, but the value of the outer loop counter as well. If outer counter is 1, the inner loop has a trip count 1 and so on. If outer counter is N the inner loop trip count is N. How many times the body will be executed? 1+2+3+……(N-1)+N = N(N+1) / 2 = ((N^2 ) +N )/ Therefore the performance is O(N 2 ). Generalization: A structure with k nested counting loops where the counter is just incremented or decrement by one has performance O(Nk) if the trip counts depends on the problem size only.

Dsa ch10 2, Study notes of Data Structures and Algorithms

Related documents

Partial preview of the text

Download Dsa ch10 2 and more Study notes Data Structures and Algorithms in PDF only on Docsity!