Download Hardness Amplification, Lecture Slide - Computer Science and more Slides Computer Architecture and Organization in PDF only on Docsity!
Analysis of Boolean Functions (CMU 18-859S, Spring 2007)
Lecture 17: Hardness Amplification
March 20, 2007 Lecturer: Ryan O’Donnell Scribe: Eric Blais
1 Hardness amplification
In this lecture, we examine how hard it is to compute, or even to approximately compute, boolean functions with small circuits. We begin by defining what we mean exactly by “hard to compute”.
Definition 1.1. The function g : {− 1 , 1 }m^ → {− 1 , 1 } is θ-hard for circuits of size s if for every circuit C of size at most s, Pr[C(x) = g(x)] ≤ θ.
The definition is valid for 1 / 2 ≤ θ ≤ 1. Somewhat counter-intuitively, smaller values of θ denote harder functions. When θ = 1, there may be a circuit of size s that correctly computes g. When θ = 1 − 2 −m, the function g is “worst-case hard”, and no circuit of size s can exactly compute g. At the other extreme of the range, if a function is 1 / 2 -hard for circuits of size s, then no circuit of that size can do better than guessing. Note that no function can be more than 1 / 2 -hard for any circuit size, since one of the constant circuits returning 1 or − 1 must correctly predict at least half of the outputs of g. In this lecture, we will mostly concentrate on the case where s is polynomial in m. We will also focus on topics that revolve around the following general question.
Question: Given a function g that is hard for circuits of size s, how can we construct some other function h, much harder for circuits of size ≈ s?
The original motivations for this question came from the fields of cryptography and complexity theory. In cryptography, methods to construct very hard functions from slightly hard functions can be used to convert weak hard-core predicates into stronger hard-core predicates. In complexity theory, methods of constructing very hard functions from slightly hard functions can be used to show that certain complexity classes are very hard-on-average, assuming they are slightly hard. This is in fact our main motivation for this lecture. In particular, we would like to show that, assuming NP can not be computed by polynomial-size circuits, there are languages in NP that can not be computed for even 1 /2 + of their inputs by small circuits. The various notions of hardness of complexity classes can be associated with our definition of θ-hardness of functions.
- “NP 6 ⊆ P/poly”: ∃L ∈ NP such that if gm is the indicator function of L ∩ {− 1 , 1 }m^ for all s = poly(m), then for sufficiently large m, gm is 1 − 2 −m-hard for circuits of size s.
- “NP is slightly hard-on-average”: ∃L ∈ NP such that if gm is the indicator function of L ∩ {− 1 , 1 }m^ for all s = poly(m), then for sufficiently large m, gm is 1 − (^) poly^1 (m) -hard for circuits of size s.
- “NP is very hard-on-average”’ : ∃L ∈ NP such that if gm is the indicator function of L ∩ {− 1 , 1 }m^ for all s = poly(m), then for sufficiently large m, gm is 12 + om(1)-hard for circuits of size s.
Using these definitions, we can now restate our goal theorem as follows:
Goal Theorem. If NP is slightly hard-on-average, then NP is very hard-on-average.
It would be even better if we could show that P 6 = NP implies that NP is very hard-on- average, but for our result we have to start with the stronger assumption that NP is slightly hard-on-average. Actually, this stronger statement is known for other complexity classes, such as PSPACE and P#P. For example, it is known that if PSPACE is not computable by polynomial- size circuits, then PSPACEis slightly hard-on-average, by a random self-reducibility argument.
2 Impagliazzo’s hard-core set theorem
In order to prove our goal theorem, we would like to understand why certain functions may be hard to compute for circuits of polynomial size. Let us consider a couple of examples of boolean functions that may be hard to compute, and analyze why these functions are hard. For our first example, let
g(x) =
some hard function , if x 1 = x 2 = · · · = xlog(1/) = 1 1 , otherwise.
The function g is easy to compute for most of its inputs, but very hard to compute when x 1 = x 2 = · · · = xlog(1/) = 1. So g is (1 − ) · 1 + · 12 ≈ 1 − 2 -hard for poly(m) size. As a second example, let us consider g(x) = permanent(x). It’s hard to say, but maybe no single input seems harder than any other input in g. So with these two examples, we see that maybe there are two different reasons why a function can be hard: perhaps it only is hard for a subset of its input, or it may be the case that a function is hard for all of its inputs. It turns out – and this is a key theorem in what we want to prove – that the hardness of every function lies in a subset of its inputs. To state this theorem precisely, we introduce the definition of hard-core sets.
Definition 2.1. H ⊆ {− 1 , 1 }m^ is a γ-hard-core set against size s for g : {− 1 , 1 }m^ → {− 1 , 1 } if for every circuit C of size at most s, Prx∼H [C(x) = g(x)] ≤ 12 + γ.
In other words, H is a hard-core set for g if no circuit of size s can do much better than guessing for predicting the output of g on any input chosen from H.
For any x /∈ S, pick t circuits C 1 ,... , Ct independently from C, and set D = Maj(C 1 ,... , Ct). The size of the circuit D is at most ts′^ + O(t), since each circuit Ci is of size s′, and we can build a linear size circuit to take the majority of their outputs. So now we want to show that D does very well on g with high probability. Since each Ci is correct with probability ≥ 12 + 10 γ , by Chernoff bound D(x) = g(x) except with probability ≤ exp(−Ω(γ^2 t)). Take t = O( (^) γ^12 log( (^) γ^2 )). Then D(x) 6 = g(x) with probability at most γ 2.
So if we look at all x ∈ {− 1 , 1 }m, D has error with respect to g of at most (1− γ 2 )2m^ + γ 2 2 m^ < · 2 m. But D has size O(ts′) ≤ s, so we have contradicted the fact that g is 1 − -hard for circuits of size s. By the above, we have shown that there always is a “hard-core measure” H. To complete the proof, we need to get a hard-core set H 0 from H. To do this, choose each x ∈ {− 1 , 1 }m^ to be in H 0 independently with probability PrH∼H[x ∈ H]. The expected cardinality of H 0 is ≥ · 2 m, and with probability at least 12 , it is at least · 2 m. With very high probability, H 0 is 2 γ-hard-core for g. (We omit the proof of this statement, but it can be obtained easily: simply apply the union bound over all circuits.)
Remark 2.4. If H is a γ-hard-core set for g, then g is almost balanced on H. By playing with H slightly, can get g exactly balanced on H, losing only a factor of 2 on γ.
3 Yao’s XOR Lemma
Having gained more understanding into the reasons why functions can be hard, let us return to our original question: given a hard function, how can we get an even harder function? As we mentioned above, there has been much work on this question in the field of cryptography, in order to turn hard-core predicates into strong hard-core predicates. A classic result in this area is Yao’s XOR Lemma, which basically says that if you have a hard function g, then you can take the XOR of many copies of g; since g was hard to compute originally, then the resulting function is even harder to compute.
Theorem 3.1 (Yao’s XOR Lemma). If g : {− 1 , 1 }m^ → {− 1 , 1 } is (1 − )-hard for size s, then h : {− 1 , 1 }n·m^ → {− 1 , 1 } defined by h(x^1 ,... , xn) = g(x^1 ) · · · · · g(xn) is 12 + (1 − )n^ + γ-hard for size Ω(γ^2 log( (^) γ^1 )) · s.
Let’s examine what we can obtain from this theorem. Suppose for example that (gm) is a function family, gm : {− 1 , 1 }m^ → {− 1 , 1 } which is 1 − (^) m^1100 -hard for circuits of size poly(m).
Take n = m^200 , γ = (^) m 1000001. Then h : {− 1 , 1 }m 300 → {− 1 , 1 }, when h is obtained by taking the
XOR of n functions gm, is 12 + (1 − (^) m^1100 )m 200
- (^) m 1000001 ≈ 12 + (^) m 1000001 -hard for circuits of size poly(m). And m^100000 is an extremely large polynomial in the input length m^300. Therefore, we get a family (h) which is 12 + (^) poly(^1 N ) -hard for circuits of size poly(N ). The above example shows that the XOR Lemma accomplishes a lot: we can start with a func- tion family that is slightly hard for polynomial circuits, and create a function family that is tremen- dously hard for polynomial circuits. Then again, that fact by itself is no big deal: we can just
pick a random function, and it is extremely hard for polynomial size circuits. Yao’s XOR Lemma becomes more interesting if we have some sort of upper bound on the complexity of the original function family (gm). What we want to achieve is to be able to show that, supposing we have a function family in some complexity class that is slightly hard, then we can get some other function family that is extremely hard for some other related complexity class. For example, as a corollary of the previous example.
Corollary 3.2. If PSPACE is 1 − (^) n 1001 -hard for polynomial circuits, then it’s 12 + (^) poly(^1 n) -hard.
This result is especially interesting because we also know that if PSPACE is 1 − 2 −n-hard for polynomial circuits, then PSPACE is 1 − (^1) n -hard. This is an old result due to Lipton (although Lipton actually showed this result for P#P^ instead of PSPACE). Let us now turn back to our favorite complexity class: can we obtain a corollary equivalent to Corollary 3.2 for NP instead of PSPACE? Not with Yao’s XOR Lemma, since (gm) ∈ NP does not imply that the function h = gm ⊕ gm ⊕ · · · ⊕ gm is also in NP. (In fact, if that were the case, then it would imply NP = coNP.) So if we want to obtain a similar corollary for NP, we need to find a different approach. A natural direction to take is to see if we can replace the XOR function in Yao’s XOR Lemma with other functions.
Definition 3.3. Let f : {− 1 , 1 }n^ → {− 1 , 1 }, and let g : {− 1 , 1 }m^ → {− 1 , 1 }. Then f ⊗ g : {− 1 , 1 }nm^ → {− 1 , 1 } is the function defined by f ⊗ g(x^1 ,... , xn) = f (g(x^1 ),... , g(xn)).
Since we are interested in identifying functions f that will help us amplify hardness in NP, our first task is to determine which functions f can be composed with some function family (gm) ∈ NP to obtain a function (f ⊗ gm) ∈ NP.
Proposition 3.4. Suppose (gm) ∈ NP, f is monotone and (f ) ∈ NP. Then (f ⊗ gm) is in NP.
The plan to obtain hardness amplification of NP is the following: we will look at various monotone functions f ’s in NP and try to understand how hard f ⊗ g is as a function of: (1) g’s hardness, and (2) intrinsic properties of f. As we will see in the next lecture, what we will need is that f has high “noise sensitivity”.
