Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
The concept of falsifiability in economic theories using tools from computational complexity. The authors model the idea of a short proof and provide examples of economic theories that are falsifiable in the usual sense but not with the additional requirement of a short proof. They consider various definitions of 'short proof' and the implications on falsifiability under different assumptions about the difficulty of computation. relevant for students and researchers in economics, mathematics, and computer science, particularly those interested in the philosophy of science and computational complexity.
Typology: Study notes
1 / 19
This page cannot be seen from the preview
Don't miss anything!
Northwestern University April 11, 2013
Tractable Falsifiability
Abstract We propose to strengthen Popper’s notion of falsifiability by adding the requirement that when an observation is inconsistent with a theory, there must be a “short proof” of this inconsistency. We model the concept of a short proof using tools from computational complexity, and provide some examples of economic theories that are falsifiable in the usual sense but not with this additional requirement. We consider several variants of the definition of “short proof” and several assumptions about the difficulty of computation, and study their different implications on the falsifiability of theories.
1 Introduction
Popper’s notion of falsifiability is a foundational concept in philosophy of science that has been influential in economic theory. According to Popper, a theory is scientific if something could potentially occur that would contradict the theory’s assertions. That is, if a theory is false, then there is some observation that conflicts with a prediction of the theory. The scientific assertions of a theory are then those that, if wrong, can be demonstrated as such. The concept of falsifiability is illustrated in the following quote of Albert Einstein: “No amount of experimentation can ever prove me right; a single experiment can prove me wrong.” What does it mean for an experiment to prove Einstein wrong? Such ∗Kellogg School of Management, Northwestern University, Evanston, IL 60208, USA. E-mail: [email protected] †. Kellogg School of Management, Northwestern University, Evanston, IL 60208, USA. E-mail: [email protected]
is given by a polynomial-time algorithm that receives as input some observations and an argument by the plaintiff claiming that these observations conflict with the theory, and outputs a verdict. Importantly, the notion of polynomial-time, as well as all other notions from computational complexity, are asymptotic in nature. In this formulation, a protocol runs in polynomial-time if there exists some polynomial p such that the protocol decides whether a given evidence violates the theory in time at most p(n), where n is the amount of evidence. Choice theory provides several examples of what we have in mind. Consider first a theory that asserts that an agent’s preferences can be rationalized by a linear order over alternatives – that is, when choosing between a set of alternatives he picks the one that is ranked highest in his order. At first glance, if some observed choice behavior violates this theory, then this violation can be easily demonstrated
choice behavior is rationalizable by two linear orders. Similar proofs of N P-completeness of rationalization have been given for other economic theories – see Kalyanaraman and Umans (2008, 2009), Apesteguia and Ballester (2010), and Demuynck (2011). In general, economists have been skeptical about the implication and meaning of these results, especially because of the worst-case assumption behind computational com- plexity analysis. Our focus is somewhat different than of the aforementioned papers: We are not concerned with the difficulty of rationalizing per se, but only with its implication about the falsifiability of the theory. What matters for us is whether there exists a short way to demonstrate that a theory is wrong. We view this as a desirable, though idealistic, property of a scientific theory. Our definition of a theory identifies a theory with its predictions. This approach, which is already strongly influenced by Popper’s falsifiability stipulation, takes ob- servable objects as primitive. Different theories make different predictions about relationships between these same observables, as in the examples of rational choice and choice by multiple rationales mentioned above. Economic theories fit this frame- work well, since the observables, such as choices and prices, have (or at least are assumed to have by economic theorists) a well defined meaning independent of the economic theory. In contrast, this approach is not suitable for formalizing physical theories since “physical observables” are such as weight, time, electric current are al- ready theory laden: they change their meaning from one physical theory to another, a point which is central in Pierre Duhem’s (1954) argument. Our paper is related to a recent paper of Chambers et al. (2010). They call a theory “completely falsifiable” if, whenever the theory is incorrect, there exists some data set that falsifies it. Chambers et al. (2010) emphasize the requirement that the data set that falsifies a theory be finite. A typical example is the theory that the choice of an agent can be rationalized by a real-valued utility function, as opposed to a linear order. In this case, if the underlying set of alternatives is infinite, it may be that observed preference will admit no such rationalization but no finite data set will reveal this. Our notion of tractable falsifiability is more restrictive: Not only do we require that the data set that falsifies a theory be finite, we also require that the plaintiff be able to prove the falsification quickly. Consider again the theory that an agent’s choice can be rationalized by two linear orders. This theory is completely falsifiable in the sense of Chambers et al. (2010): Even if the domain of alternatives is infinite, for every choice function that cannot be rationalized by a pair of linear orders
implications for the notion of falsifiability. In Section 2 we formalize the concepts of theory and evidence in a way that make them susceptible to complexity analysis. In Section 3 we give our first definitions of argument and court’s protocol and the derived notion of tractable falsifiability, and provide examples of theories that are not tractably falsifiable under the assumption that N P 6 = coN P (the class coN P is the complement of N P, and the statement N P 6 = coN P, while stronger than the famous P 6 = N P, is still widely believed). In Section 4 we consider a more general definition of a proof, in which the court is allowed to challenge the plaintiff with questions, and show that with this definition more theories become tractably falsifiable. In Section 5 we then study the implication of the assumption that the universe is limited in the complexity of objects it can produce, so that we are only interested in falsifications of theories relative to this assumption. The appendix contains some formal definitions from computational complexity that we use. Our approach and examples should be intelligible without the definitions in the appendix, but we appeal to them in our proofs and in places where the reference might be useful for readers who are familiar with computational complexity.
2 A formal model for theories
Definition 2.1 (theory) A theory is a subset T ⊆ { 0 , 1 }∗, where { 0 , 1 }∗^ is the set of all binary strings.
Consider the following simple example.
Example 2.2 A society is a graph whose nodes are called individuals, and in which two individuals are connected by an edge if they know each other. Every society can be encoded as an element of { 0 , 1 }∗^ by the adjacency matrix of its graph^1. The theory Tcl is the set of all elements in { 0 , 1 }∗^ that represent a graph of some size n with a clique of size log n.
This simple example illustrates our formalization of a theory. There is some set of conceivable objects (in this case conceivable societies, which may or may not have a large clique), and each object can be encoded as a binary string. A theory, modeled (^1) The adjacency matrix of an n-node graph is an n × n matrix in which the ij’th entry is 1 if node i is connected to node j, and 0 otherwise.
as a set of codes, says that only some of these objects (i.e. those whose codes are elements of the theory) can actually occur in our world. The description of the theory is usually not given by a list of the codes of these objects (which is usually infinite) but as a general property that all these objects satisfy. The theory Tcl says that in every society that occurs in our world there is a large set of individuals who know each other. The coding of objects is necessary for complexity considerations, which are asymp- totic in the length of the code. There is a strong sense in which all natural codes (i.e. codes that are not overly redundant and are simple to encode and decode) lead to the same computational consequences and are therefore equivalent for our purposes. For more on this issue, see for example Section 1.2.1 of Goldreich (2008). In the sequel we usually do not explicitly specify the code, and, slightly abusing terminology, iden- tify an object with its code. (Strictly speaking, it is also possible that some string x ∈ { 0 , 1 }∗^ is not a proper encoding of an object. For instance, in Example 2.2 it could be that x /∈ Tcl because |x| 6 = n^2 for any n. For our purposes, however, this will not matter, since we assume that the properness of encodings is simple to check.) We now return to the examples from the introduction – the theory of rational choice and the multiple rationales theory of Kalai et al. (2002).
Example 2.3 The observed choice behavior of an individual is a collection of pairs (S, a), where S is a set of alternatives represented by a subset of { 1 ,... , m} for some m, and a is the element of S that is the individual’s choice. Every observed choice behavior can be encoded as an element of { 0 , 1 }∗. The theory TRC is the set of all elements C ∈ { 0 , 1 }∗^ that represent observed choice behavior satisfying the following condition: There exists a linear order over { 1 ,... , m} for some m such that for every set of alternatives S in the collection C, the corresponding choice a is maximal in S for that linear order.
Example 2.4 The theory TKRS is the set of all elements C ∈ { 0 , 1 }∗^ that represent observed choice behavior of an individual satisfying the following condition: There exists a pair of linear orders over { 1 ,... , m} for some m such that for every set of alternatives S in the collection C, the corresponding choice a is maximal in S for at least one of the two linear orders.
If x is an object such that x /∈ T then the theory T predicts that the object x will
Now, if x /∈ TRC then the plaintiff can always provide such a y, and then when V (x, y) = 1 the court can conclude that indeed x /∈ TRC. If, however, x ∈ TRC, then regardless of the argument y provided by the plaintiff, the protocol V (x, y) will output 0, since there will never be a cycle. Note that in this example, y is shorter than x (and so in particular it is polynomial in the length of x). Also, note that the algorithm V that makes the checks above runs in polynomial time.
Definition 3.1 (tractably falsifiable theories) A theory T ⊆ { 0 , 1 }∗^ is tractably falsifiable if there exists a polynomial p(n) and a polynomial-time algorithm V such that the following two conditions hold:
A pair (x, y) as in the first bullet is called a falsification, where x is the evidence and y is the argument for x /∈ T. The algorithm V is called the falsification protocol for T.
The first requirement in Definition 3.1 states the completeness of the falsification protocol: if the theory is wrong, i.e. if an object x such that x /∈ T does pop up in our world, then when x is observed there is an argument, or a way to demonstrate the fact that x refutes the theory. The second requirement states the soundness: the plaintiff cannot convince the court that a theory is wrong if the evidence x does not contradict the theory. Definition 3.1 is essentially the definition of the complexity class coN P, and we thus propose to identify the class of tractably falsifiable theories with this class. coN P is the complement of the class N P – see Appendix A for a formal definition. As we discussed in the introduction, we think about the argument y against x in Definition 3.1 as a demonstration that a plaintiff can present in court to show that x violates the theory. The demonstration must be short, but we make no assumptions about the complexity of finding it. We now return to the examples from the beginning of the section:
Example 3.2 Assume N P 6 = coN P. Then the theories Tcl and TKRS from Exam- ples 2.2 and 2.4 are not tractably falsifiable.
Thus, if N P 6 = coN P it may be the case that the theories are violated in our world, i.e. that there exists a society without a large clique and individuals whose observed choice behavior cannot be rationalized by two linear orders, but still there is no short way to demonstrate the violations of the theories. We emphasize that these assertions do not rely solely on the assumption that P 6 = N P, but rather on the stronger assumption that N P 6 = coN P. The statement of Example 3.2 follows from the facts that Tcl and TKRS are N P- complete (see Demuynck (2010) for a proof of the latter) and from Observation 3. below.
Observation 3.3 Assume that N P 6 = coN P. If a theory T is N P-complete then T is not tractably falsifiable.
This observation follows from our definition of tractable falsifiability as a theory in the complexity class coN P and the fact that if N P 6 = coN P then no theory that is N P-complete can be in coN P (see for example Section 2.4.3 of Goldreich (2008) for a proof). Note that if a theory is in P, then it is tractably falsifiable, since P ⊆ coN P. Note also that even if a theory T 6 ∈ P, this does not immediately imply that T is not tractably falsifiable – in particular, it could be the case that T ∈ coN P \ P. It is also interesting to note that in principle there is no direct implication between the predictive power of a theory and the difficulty of demonstrating that a given outcome violates the theory. For a pair of theories T and T ′^ let us say that T ′^ has more predictive power than T if T ⊆ T ′^ (i.e., if any observation that is compatible with T is also compatible with T ′). The theory TRC is a sub-theory of TKRS, and, as we have argued before, the former is tractably falsifiable while the latter is not. On the other hand, consider again the framework of Example 2.2 and let T ′^ be the theory that in every society there is a pair of individuals that know each other. Clearly Tcl is a sub-theory T ′, but the former is not tractably falsifiable while the latter is. The last example also shows that our definition of tractable falsifiability is de- manding in the sense that we require the existence of a short demonstration for every object that violates the theory. So, even if the theory is not tractably falsifiable, it may be that some of its predictions can be easily checked. For example, the theory Tcl implies that in every society there is at least one pair of people who knows each other. This prediction is easy to check, and a demonstration that a society includes
The output of the interaction is yk.
Definition 4.1 (interactively falsifiable theories) A theory T ⊆ { 0 , 1 }∗^ is interactively falsifiable if for every ε > 0 there exists a two-party protocol with an algorithm P and a probabilistic polynomial-time algorithm V such that the following two conditions hold:
The algorithms V is called the interactive falsification protocol for T. The algorithm P is called the argument or of the plaintiff.
The first requirement of Definition 4.1 states the completeness of the falsification protocol: if x is indeed a falsification of T then the plaintiff has a way to convince the court that T is wrong – namely, he follows the procedure P. The second requirement states soundness: If x does not falsify the theory T , then regardless of the argument P ∗^ made by the plaintiff the court will not be convinced otherwise. Note that while the court is limited to a polynomial time algorithm V , the plaintiff is not restricted. Definition 4.1 is very close to the definition of interactive proofs, initially introduced by Goldwasser et al. (1989) and Babai (1985), and characterized by the complexity class IP (see Appendix). To see the power of interaction, contrast the following with Example 3.2.
Example 4.2 The theories Tcl and TKRS from Examples 2.2 and 2.4 are interactively falsifiable.
This holds because of the following observation.
Observation 4.3 If T is in N P then T is interactively falsifiable.
Many economic theories, particularly those claiming that behavior can be rational- ized, are in the class N P (see also Chambers et al. (2011)). Observation 4.3 then implies that these theories can be interactively falsified. The proof of Observation 4. makes use of a deep theorem in complexity theory proved by Shamir (1993), and we therefore defer the proof to the appendix.
5 Tractable Universe
The definitions of tractable falsifiability and interactive falsifiability are “worst-case” definitions, in the sense that the falsification protocol must be short for any object that contradicts a given theory. In this section we add another twist to our framework and restrict our attention to objects that “typically” appear in the universe, and study the implication of this restriction on falsifiability. Of course, an immediate difficulty is, what is typical? This question is addressed by the theory of average-case complexity initiated by Levin (1986). Levin’s theory begins with the Church-Turing thesis. The Church-Turing thesis was formulated in the beginning of the twentieth century following the work of var- ious authors, most notably and Alan Turing and Alonzo Church, to formally define the intuitive notion of effective computability. In its most simple form the hypothesis identifies effective computability with Turing Machine computability. A more ex- treme version states that the outcome of any physical process is a Turing computable function possibly with appeal to pure randomness. There is an extensive literature in philosophy about the ramification and implication of the physical Church-Turing thesis. The Church-Turing thesis also appears in economic literature. For example it is natural to assume that the choice of an economic agent, as the outcome of a physical or cognitive process, is a computable function of the given set of choices. Intuitively, the physical Church-Turing thesis implies that the universe can be sim- ulated by a computer with unlimited time and memory resources and access to pure randomness. Levin’s theory relies on a stronger version of the hypothesis, sometimes called the Strong Church-Turing thesis, that the objects that appear in the universe can be created in polynomial-time (in the size of their canonical description). We call a universe that satisfies this assumption tractable.^3 Recall that in order for a theory to be tractably falsifiable (Definition 3.1), for every instance x that violates the theory there must be a way to demonstrate this violation. In the following we modify the definition by requiring an argument only against “typical” instances, which are those that can be produced in a tractable universe. However, because we also assume some randomness in the universe, we need to allow for a small probability that the universe does end up producing an (^3) The Strong Church-Turing thesis is challenged by quantum computers, which seems to be phys- ically realizable in principle but cannot be simulated efficiently on a Turing Machine (Bernstein and Vazirani (1997)).
Observation 5.3 In Heuristica, every theory that is in N P is tractably falsifiable in a tractable universe.
Proof: Fix A and ε as in Definition 5.1, and let q be a polynomial and B be an algorithm for deciding membership T such that P
TIME(B, x) > q(n)
< ε for x = A(0n), where TIME(B, x) is the running time of B on x. Such an algorithm B exists in Heuristica. Let V (x, y) = B(x) if TIME(B, x) ≤ q(n) and V (x, y) = 0 otherwise.
A Definitions from Computational Complexity
In this section we briefly and somewhat informally provide the computational com- plexity background used in this paper. See Goldreich (2008) or any other textbook on computational complexity for further details. While in this paper we study theories, as defined in Definition 2.1, the standard nomenclature for the same object in the field of computational complexity is called a decision problem. The corresponding problem is to decide, for a given input x ∈ { 0 , 1 }∗^ whether x ∈ T. For every code x ∈ { 0 , 1 }∗^ we denote its length by |x|. An algorithm A is called polynomial-time algorithm if Time(B, x) < p(|x|) for every x ∈ { 0 , 1 }∗^ where Time(B, x) is the running time (more specifically, number of operations) of B on x. A decision problem T is in the class P if there is a polynomial-time algorithm A such that that, for any input x ∈ { 0 , 1 }∗, A(x) = 1 if and only if x ∈ T. A decision problem T is in the class N P if there exists a polynomial p(n) and a polynomial-time algorithm V such that the following two conditions hold:
Roughly, this is the class for which there is a polynomial-time algorithm that verifies that x ∈ T with the aid of a witness y. Clearly P ⊆ N P. The widely believed assumption that P = N P is at the core of computational complexity theory. The hardest problems in N P are called N P-complete. We don’t provide the formal definition here since we don’t make use of the concept of N P-completentess
in our definitions, but we use of the fact that the decisions problems in Examples 2. and 2.4 are N P-complete. The class coN P is the class of complements of decision problems in N P – it contains those decision problems {{ 0 , 1 }∗^ \ T : T ∈ N P}. Definition 3.1 characterizes this class. It is widely believed that N P 6 = coN P. This assumption is stronger than P 6 = N P. (Indeed, if P = N P then P = N P = coN P since the class P is closed under complement.)
B Proof of Observation 4.
The proof makes use of a non-trivial theorem of complexity theory. Before describing the theorem we define two additional complexity classes: PSPACE and IP. The class PSPACE contains all decision problems T for which there is an algo- rithm that, on any input x, determines whether or not x ∈ T , and such that the following holds: The number of bits of memory used by the algorithm must be at most polynomial in |x|. It is easy to verify that N P ⊆ PSPACE. A decision problem T ⊆ { 0 , 1 }∗^ is in the class IP if for every ε > 0 there exists a two-party protocol with an algorithm P and a probabilistic polynomial-time algorithm V such that the following two conditions hold:
Shamir (1993) proved that the classes IP and PSPACE are identical. We use Shamir’s theorem in the proof of Observation 4.3.
Proof of Observation 4.3: According to definition 4.1, a theory T is interactively falsifiable if and only if its complement is in IP. Indeed, an interactive proof that x /∈ T can be viewed as an interactive proof that x ∈ T c, and vice versa. Shamir (1993) proved that the classes IP and PSPACE are equal. Since PSPACE is closed under complement, it follows that a theory is interactively falsifiable if and only if it is in PSPACE. The observation follows from the fact that N P ⊆ PSPACE.
[13] T. Hu and E. Shmaya. Expressible inspections. Theoretical Economics (2012).
[14] G. Kalai, A. Rubinstein, and R. Spiegler. Rationalizing choice functions by mul- tiple rationales. Econometrica 70(6), 2002.
[15] S. Kalyanaraman and C. Umans. The complexity of rationalizing matchings. Pro- ceedings of the 19th International Symposium on Algorithms and Computation, ISAAC 2008.
[16] S. Kalyanaraman and C. Umans. The complexity of rationalizing network for- mation. Proceedings of the 50th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2009.
[17] L.A. Levin. Average case complete problems. SIAM Journal on Computing 15(1),
[18] W. Olszewski and A. Sandroni. Falsifiability. The American Economic Review 101 (2011).
[19] A. Shamir. IP = PSPACE. Journal of the ACM 39(4), 1992.
