Download Complexity Theory: Classification of Problems and more Study notes Complexity Theory in PDF only on Docsity!
INTRODUCTION
Complexity theory focuses on classifying computational problems according to their inherent difficulty and relating those classes to each other. The need to be able to measure the complexity of a problem, algorithm or structure, and to obtain bounds and quantitive relations for complexity arises in more and more sciences: besides computer science, the traditional branches of mathematics, statistical physics, biology, medicine, social sciences and engineering are also confronted more and more frequently with this problem. Complexity is defined as the running time to execute a process and how much space it consumes. Complexity is of two types which are space complexity and time complexity. In the approach taken by computer science, complexity is measured by the quantity of computational resources (time, storage, program, communication) used up by a particular task. Computation theory can basically be divided into three parts of different character. First, the exact notions of algorithm, time, storage capacity must be introduced. For this, different mathematical machine models must be defined, and the time and storage needs of the computations performed on these need to be clarified generally measured as a function of the size of input. Each computational problem belongs to a given complexity class which describes the level of difficulty of that problem, We shall be looking at time complexity and space complexity and later on insist on complexity classes.
1.TIME COMPLEXITY
The time complexity deals with the quantification of the amount of time taken by a set of code or algorithm to process or run as a function of the amount of input. In other words, time complexity is essentially efficiency, or how a program function takes to process a given input. It can be applied to almost any algorithm or function but is more useful to recursive functions. Use of time complexity makes it easy to estimate the running time of a program. performing an accurate calculation of a program’s operation time is a labor intensive process depending on the compiler and the type of computer or speed of the processor. Complexity can be viewed as the maximum number of primitive operations that a program may execute. regular operations are single additions, multiplications, assignments, etc …
Since algorithm running time may vary among different inputs of the same size, one commonly considers the worst case time complexity, which is the maximum amount of time required for inputs of a given size. There is the average case complexity which is the average time taken on inputs of a given size. We also have the best case complexity which is the minimum amount of time required for inputs of a given size.
The time complexity is commonly expressed using the big O notation.
printf(“Hello world”);
}
int i=0 , will be executed only once
i< N , will be executed N+1 times
i++ will be executed N times
So the number of operations required by this loop is 1+(N+1)+N=2N+2.
Hence the time complexity is in the order of N, that is O(N).
2. SPACE COMPLEXITY
Space complexity is the amount of memory used by the algorithm including the input values to the algorithm to execute and produce the result. While executing, algorithm uses memory space for three reasons.
- Instruction space It is the amount of memory used to save the compiled version of instructions.
- Environmental stack Sometimes an algorithm may be called inside another algorithm(function).In such a situation, the current variables are pushed onto the system stack, where they wait for further execution and then call to the inside algorithm(function) is made.
- Data space Amount of space used by the variables and constants.
But while calculating the space complexity of an algorithm, we usually consider only data space. That means we calculate only the memory required to store variables, constants, structures, etc… To calculate the space complexity, one must know the memory required to store different data type values according to the compiler. For example the c programming language compiler requires the following.
- 2 bytes to store integer values
- 4 bytes to store floating point value
- 1 byte to store character value
- 6 or 8 bytes to store double value.
Lets consider the following peace of code int square(int a) {
Return aa; }*
In this code, it requires 2 bytes of memory to store a variable ‘a’ and another 2 bytes of memory is used for return value. That means totally it requires 4 bytes of memory to complete its execution. And this 4 bytes of memory is fixed for any input value of ‘a’. this space complexity is said to be constant space complexity.
Consider the following piece of code int sum(int A[ ], int n) { int sum=0,i; for(i=0; i< n; i++) Sum=sum + A[i]; return sum; }
In the above code, n*2 bytes of memory is used to store array variable a[ ]. 2 bytes for integer parameter n. 4 bytes for local integer ‘sum’ and ‘i’(2 bytes each), 2 bytes of memory for return value,which means totally it requires 2n+8 bytes of memory to complete its execution.
The set P is defined as the set of all problems that can be solved in polynomial worse case time. It is the set of problems whose time is O(Nk) for some k.
A language L is in P if and only if there exists a deterministic Turing machine M, such that:
M runs for polynomial time on all inputs. For all x in L, M outputs 1(yes). For all x not in L, M outputs 0(no). As an example of P class problem, we have the path problem which concerns directed graphs. A directed graph G contains nodes s and t as shown in the following figure. The path problem is to determine whether there is a directed path from s to t. let
PATH={〈 〉| }
A polynomial time algorithm M for PATH operates as follows. M=”on input 〈 〉 where G is a directed graph with nodes s and t;
- place a mark on node s.
- repeat the following until no additional nodes are marked
- scan all the edges of G, if an edge 〈 〉 is found going from from a marked node a to an unmarked node b, mark node b.
- if t is marked, accept. Otherwise, reject.
Now we analyze this algorithm to show that it runs in polynomial time. Obviously, stages 1 and 4 are executed only once. Stage three runs at most m times because each time except the last it marks an additional node in G. thus the total number of stages used is at most 1+1+m, giving a polynomial in the size of G.
Stages 1 and 4 of M are easily implemented in polynomial time on any reasonable deterministic model. Stage 3 involves a scan of the input and a test of whether certain nodes are marked, which also is easily implemented in polynomial time. Hence M is a polynomial time algorithm for PATH.
3.1.2. CLASS NP
NP is the set of decision problems solvable in polynomial time by a theoretical non deterministic Turing machine. Similarly , NP is the set of all problems for which a given Candidate solution can be tested in polynomial time by a deterministic Turing machine. NP is defined using deterministic Turing machines as verifiers.
For instance let us consider the subset sub problem and assume that we are given some integers, and we wish to know whether some of these integers sum up to a value t. To answer this one will need to write down an algorithm that obtains all possible subsets, and given a particular subset also called a certificate, one can check whether it sum is t. An algorithm that verifies whether a given subset has sum t is called verifier.
Both definitions using NTM and DTM are equivalent because a Non deterministic Turing machine could solve an NP problem in polynomial time by non deterministically selecting a certificate and running the verifier on the certificate.
Another example of NP problems include:
The clique problem A clique in an undirected graph is a subgraph, wherein every two nodes are connected by an edge. A k-clique is a clique that contains k nodes. The following figure illustrate a graph having a 5 clique.
3.1.3. POLYNOMIAL TIME REDUCIBILITY
3.1.4 NP COMPLETE
A decision problem is NP complete if:
- It is in NP, and
- Every problem in NP is reducible to NP complete in polynomial time.
This follows from the definition of polynomial time reducibility.
NP complete problems are studied because the ability to quickly verify solutions to a problem(NP) seems to correlate with the ability to quickly solve that problem(P). It is not known whether every problem in NP can be quickly solved, This is called the P versus NP problem.
But a researcher trying to show that P is unequal to NP may focus on NP complete problem. If any problem in NP requires more than polynomial time, an NP complete one does. Furthermore, a researcher attempting to prove that P is equals to NP only needs to find a polynomial time algorithm for an NP complete problem to achieve this goal. But the phenomenon of NP complete may prevent wasting time searching for a non existing polynomial time algorithm to solve a particular problem. Even though we may not have the necessary mathematics to prove that the problem is unsolvable in polynomial time, we believe that P is unequal to NP, so proving that the problem is NP complete is strong evidence of nonpolymiality. As an example of NP complete problems, there is:
The satisfiability problem
The satisfiability problem is the problem of determining whether there exists an interpretation that satisfies a given Boolean formula. It asks whether the variables of a given Boolean formula evaluates to true and if it is the case the formula is called satisfiable. On the other hand, if no such assignment exists, the function expressed by the formula is false for all possible variable assignments and the formula is unsatisfiable.
Of how to show that satisfiability problem is NP complete is done using the Cook-Levin theorem which states that the Boolean satisfiability problem is NP complete, that is any problem in NP can be reduced in polynomial time by a deterministic Turing machine to the problem of determining whether a Boolean formula is satisfiable.
There are also many other time complexity classes which we haven’t mentioned in the above like co-NP(the complement of NP),NP-hard,etc…
3.2.SPACE COMPLEXITY CLASSES
Let M be a deterministic Turing machine that halts on all inputs. The space complexity of M is the function f: N R+. f(n) is the maximum number of tape cells that M scans on any input of length n. if the space complexity of M is
The following diagram depicts the relationship among these classes.
EXPTIME is the set of all decision problems that have exponential runtime on the deterministic Turing machine. It can be reformulated as the set of problems that can be solved by an alternating Turing machine in polynomial time.
3.2.2. PSPACE COMPLETE
A decision problem is PSPACE complete if it can be solved using an amount of memory that is polynomial in the input length and if every other problem that can be solved in polynomial space can be transformed to it in polynomial time.
The problems that are PSPACE complete can be thought of as the hardest problems in PSPACE, because a solution to any one problem could easily be used to solve any other problem in PSPACE.
These are some examples of PSPACE complete problems.
Regular expressions and automata Given a regular expression R, determining whether it generates every string over its alphabet is PSPACE-complete.
There are also many other space complexity classes which we haven’t discussed in the above like N(logarithm amount of space by a TM), NL(logarithm amount of space by a NTM), co-PSPACE,etc…
4.THE POLYNOMIAL HIERARCHY
Before getting to the concept of Polynomial hierarchy, it will be important for us to go through the notion of Oracle machines
4.1.THE ORACLE MACHINE
It is an abstract machine used to study decision problems. It is a Turing machine with a black box called an oracle also able to solve decision problems in a single operation.
The polynomial hierarchy is a hierarchy of complexity classes that generalizes the classes P,NP, and co NP to oracle machines. We have already talked on the classes P,NP, and co NP but here it is with respect to an oracle machine and not a Turing machine.
4.2. HIERARCHY THEOREM
Common sense suggests that giving the Turing machine more time or more space should increase the class of problems that it can solve. For example Turing machines should be able to decide more languages in time n^3 than they can in time n2.^ .The hierarchy theorem proves that this intuition is correct. We
be found in nature and used in computations. This leads us to the notion of a randomized Turing machine or probabilistic Turing machine.
A probabilistic Turing machine is a non deterministic Turing machine which chooses between the available transitions at each point according to some probability distribution. In the case of equal probabilities for the transitions, it can be defined as a deterministic Turing machine having an additional “write” instruction where the value of the write is uniformly distributed in the Turing Machine’s alphabet.
As a consequence, probabilistic Turing machines can have random results; on given inputs , it may have different runtimes, or it may not halt at all; further it may accept an input in one execution and reject the same input in another execution. Therefore the notion of acceptance of a string by a probabilistic Turing machine can be defined in different ways. Various polynomial time randomized complexity classes that results from different definition of acceptance include RP, co RP, BPP,ZPP. And if the machine is restricted to logarithmic space instead of polynomial time then the complexity classes RL, co-RL, BPL, and ZPL are obtained.
We shall be looking at the complexity classes RP,BPP briefly.
5.1. CLASS RP(Randomized polynomial time)
It is the complexity class of problems for which a probabilistic Turing machine exists with the following properties:
It always runs in polynomial time in the input size. If the correct answer is NO, it always returns NO. If the correct answer is YES, then it returns YES with probability at least ½(otherwise it returns NO)
RP says that a YES answer is always right and that a NO answer might be wrong because a question with a YES answer can sometimes be answered NO.
In other words, while NO questions are always answered NO, one cannot trust the NO answer, it may be a mistaken answer for a YES question.
The class co-RP is the complement of the class RP.
5.2. CLASS BPP(Bounded error probabilistic polynomial
time)
Which is the class of all decision problems solvable by a probabilistic Turing machine in polynomial time with an error probability bounded away from ½ for all instances.
A problem is in BPP if there exists an algorithm for it that has the following properties
It is allowed to flip coins and make random decisions It is guaranteed to run in polynomial time On any given run of the algorithm, it has at most 1/3 of giving the wrong answer, whether the answer is YES, or NO. The class BPP describes algorithms that can give incorrect answers on both YES and NO instances.
CONCLUSION
We have seen so far that calculating how much time and space in terms of seconds or bytes an algorithm may take can be very tedious due to large inputs or even the complexity of the problem itself, but with complexity classes, it is quite easier to estimate that amount of time or space and thereby be able to compare the efficiency of different algorithms for some problem, each problem is found in some complexity class which shows their level of difficulty and we have also seen how those classes can be related to each other which helps in reducing the cost of finding the complexity of some particular problems by the intermediary of another problem.
