Lecture Notes on The Lambda Calculus, Lecture notes of Programming Languages

Download Lecture Notes on The Lambda Calculus and more Lecture notes Programming Languages in PDF only on Docsity!

Collected Lecture Notes

15-814: Types and Programming Languages

Frank Pfenning

Lecture 1–

Fall 2020

L ECTURE N OTES FALL 2020

Lecture Notes on

The Lambda Calculus

15-814: Types and Programming Languages

Frank Pfenning

Lecture 1

Tuesday, September 1, 2020

1 Introduction

This course is about the principles of programming language design, many of which derive from the notion of type. Nevertheless, we will start by studying an exceedingly pure notion of computation based only on the notion of function, that is, Church’s -calculus [CR36 ]. There are several reasons to do so.

• We will see a number of important concepts in their simplest possible form, which means we can discuss them in full detail. We will then reuse these notions frequently throughout the course without the same level of detail.
• The -calculus is of great historical and foundational significance. The independent and nearly simultaneous development of Turing Ma- chines [Tur36] and the -Calculus [CR36] as universal computational mechanisms led to the Church-Turing Thesis, which states that the ef- fectively computable (partial) functions are exactly those that can be implemented by Turing Machines or, equivalently, in the -calculus.
• The notion of function is the most basic abstraction present in nearly all programming languages. If we are to study programming languages, we therefore must strive to understand the notion of function.
• It’s cool!

L ECTURE N OTES T UESDAY, S EPTEMBER 1, 2020

The Lambda Calculus L1.

x. Bound variables can be renamed consistently in a term So x. x + 5 = y. y + 5 = whatever. whatever + 5. Generally, we rename variables silently because we identify terms that differ only in the names of -bound variables. But, if we want to make the step explicit, we call it ↵-conversion.

x. e = (^) ↵ y.[y/x]e provided y not free in e

The proviso is necessary, for example, because x.x y 6 = y.y y. We capture the rule for function application with (x. e 2 ) e 1 = (^) [e 1 /x]e (^2)

and call it -conversion. Some care has to be taken for the substitution to be carried our correctly—we will return to this point later. If we think beyond mere equality at computation, we see that -conversion has a definitive direction: we apply is from left to right. We call this - reduction and it is the engine of computation in the -calculus. (x. e 2 ) e 1 ! (^) [e 1 /x]e (^2)

3 Simple Functions and Combinators

The simplest functions are the identity function and the constant function. The identity function, called I, just returns its argument x.

I = x. x

The constant function returning x could be written as

y. x

We calculate (y. x) e ! (^) x for any expression e since y does not occur in the expression x. This is somewhat incomplete in the sense the expression y. x has a free variable which is therefore fixed. What we would like is a closed expression K (one without free variables) such K x is the constant function, always returning x. But that’s easy: we just abstract over x! K = x. y. x

Then K x ! (^) y. x is the constant function returning x. A combinator for us is just a closed -expression like I or K. We will see more interesting combinators in the next lecture.

L ECTURE N OTES T UESDAY, S EPTEMBER 1, 2020

L1.4 The Lambda Calculus

4 Summary of -Calculus

-Expressions.

Variables x Expressions e ::= x. e | e 1 e 2 | x

x. e binds x with scope e , which is as large as possible while remaining consistent with the given parentheses. Juxtaposition e 1 e 2 is left-associative.

Equality.

Substitution [e 1 /x]e 2 (capture-avoiding, see Lecture 2) ↵-conversion x. e = (^) ↵ y.[y/x]e provided y not free in e -conversion (x. e 2 ) e 1 = (^) [e 1 /x]e (^2)

We generally apply ↵ -conversion silently, identifying terms that differ only in the names of the bound variables.

Reduction.

-reduction (x. e 2 ) e 1 ! (^) [e 1 /x]e (^2)

5 Representing Booleans

Before we can claim the -calculus as a universal language for computation, we need to be able to represent data. The simplest nontrivial data type are the Booleans, a type with two elements: true and false. The general technique is to represent the values of a given type by normal forms, that is, expressions that cannot be reduced. Furthermore, they should be closed, that is, not contain any free variables. We need to be able to distinguish between two values, and in a closed expression that suggest introducing two bound variables. We then define rather arbitrarily one to be true and the other to be false true = x. y. x false = x. y. y

The next step will be to define functions on values of the type. Let’s start with negation: we are trying to define a -expression not such that

not true = (^) false not false = (^) true

L ECTURE N OTES T UESDAY, S EPTEMBER 1, 2020

L1.6 The Lambda Calculus

An important observation here is that

not = b. b (x. y. y) (x. y. x) 6 = b. x. y. b y x = not^0

Both of these are normal forms (they cannot be reduced) and therefore repre- sent values (the results of computation). Both correctly implement negation on Booleans, but they are different. This is evidence that when computing with particular data representations in the -calculus it is not extensional: even though the functions behave the same on all the arguments we care about (here just true and false), the are not convertible. To actually see that they are not convertible we need the Church-Rosser theorem which says if e 1 and e 2 are ↵ -convertible then there is a common reduct e such that e 1 ! ⇤ e and e 2 ! ⇤ e. As a next exercise we try exclusive conjunction. We want to define a -expression and such that

and true true = (^) true and true false = (^) false and false true = (^) false and false false = (^) false

Learning from the negation, we start by guessing

and = b. c. b (.. .) (.. .)

where we arbitrarily put b first. Looking at the equations, we see that if b is true then the result is always c.

and = b. c. b c (.. .)

If b is false the result is always just false, no matter what c is.

and = b. c. b c false

Again, it is now a simple matter to verify the desired equations and that, in fact, the right-hand side of these equations is obtained by reduction.

6 The L AMBDA Language

In lecture, we used a toy implementation of a the -calculus in a language called L AMBDA. This implementation uses a concrete syntax where is

L ECTURE N OTES T UESDAY, S EPTEMBER 1, 2020

The Lambda Calculus L1.

written as a backslash ‘\ ’. A program consists of a sequence of declarations, of which there are three forms:

defn x = e variable x stands for e norm x = e variable x stands for the normal form of e conv e 1 = e 2 verify that e 1 and e 2 have the same normal form

Allowing definitions is a convenience, but it does not change the expressive power of the -calculus, because we can replace defn x = e by (x... .) e where ‘... ’ represents the scope of the definition. The norm and conv declarations initiate computation and allow the programmer to examine the normal form of an expression (if it exists). In addition, declarations can be negated with! , for example, to check that two expressions are not convertible. 1 % represent booleans as closed expressions in normal form 2 3 defn true = \x. \y. x 4 defn false = \x. \y. y 5 6 defn not = \b. b false true 7 defn not’ = \b. \x. \y. b y x 8 9 (* confirm that not and not’ are not convertible (^) *) 10 !conv not = not’ 11 12 % normalize "not true" 13 norm _ = not true 14 15 % test not and not’ against their specification 16 conv not true = false 17 conv not false = true 18 19 conv not’ true = false 20 conv not’ false = true

Listing 1: Booleans in L AMBDA

For more information on LAMBDA , consult the Software page for the course.

L ECTURE N OTES T UESDAY, S EPTEMBER 1, 2020

Lecture Notes on

Primitive Recursion

15-814: Types and Programming Languages

Frank Pfenning

Lecture 2

Thursday, September 3, 2020

1 Introduction

In this lecture we continue our exploration of the -calculus and the repre- sentation of data and functions on them. We give schematic forms to define functions on natural numbers and give uniform ways to represent them in the -calculus. We begin with the schema of iteration and then proceed the more complex schema of primitive recursion. In the next lecture we will arrive at the fully general scheme of recursion.

2 Function Composition

One the most fundamental operation on functions in mathematics is to compose them. We might write

(f ○ g)(x) = f (g(x))

Having -notation we can first explicitly denote the result of composition (with some redundant parentheses)

f ○ g = x. f (g(x))

As a second step, we realize that ○ itself is a function, taking two functions as arguments and returning another function. Ignoring the fact that it is usually written in infix notation, we define

○ = B = f. g. x. f (g x)

L ECTURE N OTES T HURSDAY, S EPTEMBER 3, 2020

L2.2 Primitive Recursion

We call it B because that’s its traditional name as a combinator. The unit of composition should the identity function, as defined by I = x. x. Composing any other function f with I should just yield f. In other words, we expect B f I =? f =? B I f Let’s calculate: B f I = (f. g. x. f (g x)) f I ￿→ (g. x. f (g x)) I ￿→ x. f (I x) ￿→ x. f x =^? f

We see the result is not exactly f as we expected, but x. f x. However, these two expressions always behave the same when applied to an arbitrary argument so they are extensionally equal. To capture this we add one more rule to the -calculus:

⌘-conversion x. e x =⌘ e provided x ￿∈ FV(e)

The proviso that x not be among the free variables of e is needed, because x. x x =￿ x. y x. The first applies the argument to itself, the second applies y to the given argument. It is possible to orient this equation and investigate the notion of ⌘ - reduction. However, it turns out this is somewhat artificial because exten- sionality is a reasoning principle for equality and not a priori a computa- tional principle. Interestingly, in the setting of typed -calculi it makes more sense to use the equation from right to left, called ⌘ -expansion, but some discipline has to be imposed or expansion does not terminate. We should remember that this form of extensionality does not extend to functions defined over specific representations. For example, we saw there are (at least) two formulations of negation on Booleans which are not equal, even if we throw in the rule of ⌘-conversion.

3 Nontermination

At this point we pause briefly to ask three natural questions:

1. Does every expression have a normal form?
2. Can we always compute a normal form if one exists?

L ECTURE N OTES T HURSDAY, S EPTEMBER 3, 2020

L2.4 Primitive Recursion

The simplest one is

⌦ = (x. x x) (x. x x)

Indeed, there is only one possible -reduction and it immediately leads to exactly the same term:

⌦ = (x. x x) (x. x x) ￿→ (x. x x) (x. x x) ￿→ (x. x x) (x. x x) ￿→...

So ⌦ reduces in one step to itself and only to itself.

Can we always compute a normal form if one exists?

The answer here is “yes”, although it is not easy to prove that this is the case. Let’s consider an example (recall that K = x. y. x):

K I ⌦ ￿→ (y. I) ⌦ ￿→ I

So the expression K I ⌦ does have a normal form, even though ⌦ does not. This is because the constant function K I ignores its argument. On the other hand we also have

K I ⌦ ￿→ K I ⌦ ￿→ K I ⌦ ￿→ ￿

because we have the ⌦ ￿→ ⌦ and reduction can be applied anywhere in an expression. Fortunately, there is a strategy which turns out to be complete in the sense that if an expression has a normal form, this strategy will find it. It is called leftmost-outermost or normal-order reduction. This strategy scans through the expression from left to right and when it find a redex (that is, an expression of the form (x. e) e ′^ ) it applies -reduction and then returns to the beginning of the result expression. In particular, it does not consider any redex in e or e ′^ , only the “outermost” one. Also, in an expression ((x. e 1 ) e 2 ) e 3 it does not consider any potential redex in e 3 , only the leftmost one. This strategy works in our example: the redex in ⌦ would not be consid- ered, only the redex K I and then the redex (y. I) ⌦. The implementation of L AMBDA uses a straightforward function for leftmost-outermost reduction, complicated very slightly by the fact that

L ECTURE N OTES T HURSDAY, S EPTEMBER 3, 2020

Primitive Recursion L2.

names such as K or I which in the notes are only abbreviations at the mathematical level of discourse, are actual language-level definitions in the implementation. So we have to expand the definition of K , for exam- ple, before applying -reduction, but we do not officially count this as a substitution. The notion of leftmost-outermost reduction is closely related to the notion of call-by-name evaluation in programming languages (and, with a little more distance, to call-by-need which is employed in Haskell). In contrast, call-by-value would reduce the argument of a function before applying the -reduction, which is not complete, as our example shows. The analogy is not exact, however, since in programming languages such as ML or Haskell we also do not reduce under -abstractions, a fact that represents a sharp dividing line between foundational calculi such as the - calculus and actual programming languages. We will justify and understand these decisions in a few lectures.

Are normal forms unique?

The outcome of a computation starting from e is its normal form. At any point during a computation there may be many redices. Ideally, the out- come would be independent of the reduction strategy we choose as long as we reach a normal form. Otherwise, the meaning of an expression (as represented by its normal form) may be ambiguous. Therefore, Church and Rosser [? ] spend considerable effort in proving the uniqueness of nor- mal forms. The key technical device is a property called confluence (also referred to as the Church-Rosser property). It is often depicted in the following diagram: e

e 1 e (^2)

e ′

∗ ∗

∗ ∗

In words: if we can reduce e to e 1 and also e to e 2 then there exists an e ′ such that e 1 and e 2 both reduce to e ′^. The solid lines are given reduction sequences while the reduction sequences represented by dashed lines have to be shown to exist. Reduction here is in multiple steps (indicated by the star “ ∗”). For the -calculus (and the original Church-Rosser Theorem), this reduction would usually be -reduction. Very roughly, the proof shows how

L ECTURE N OTES T HURSDAY, S EPTEMBER 3, 2020

Primitive Recursion L2.

We cannot carry out the correctness proof in closed form as we did for the Booleans since there would be infinitely many cases to consider. Instead we calculate generically (using mathmetical notation and properties) succ n = s. z. s (n z s) = s. z. s (s n^ (z)) = s. z. s n+^1 (z) = n + 1

A more formal argument might use mathematical induction over n. Using the iteration property we can now define other mathematical functions over the natural numbers. For example, addition of n and k iterates the successor function n times on k. plus = n. k. n succ k

You are invited to verify the correctness of this definition by calculation. Similarly: times = n. k. n (plus k) zero exp = b. e. e (times b) (succ zero)

5 The Schema of Iteration

As we saw in the first lecture, a natural number n is represented by a function n that iterates its first argument n times applied to the second: n g c = g (... (g ￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿￿ n times

c)). Another way to specify such a function schematically is

f 0 = c f (n + 1 ) = g (f n) If a function satisfies such a schema of iteration then it can be defined in the -calculus on Church numerals as f = n. n g c

which is easy to verify. The class of function definable this way is total (that is, defined on all natural numbers if c and g are), which can easily be proved by induction on n. Returning to examples from the last lecture, let’s consider multiplication again. times 0 k = 0 times (n + 1 ) k = k + times n k

L ECTURE N OTES T HURSDAY, S EPTEMBER 3, 2020

L2.8 Primitive Recursion

This doesn’t exactly fit our schema because k is an additional parameter. That’s usually allowed for iteration, but to avoid generalizing our schema the times function can just return a function by abstracting over k.

times 0 = k. 0 times (n + 1 ) = k. k + times n k

We can read off the constant c and the function g from this schema

c = k. zero g = r. k. plus k (r k)

and we obtain

times = n. n (r. k. plus k (r k)) (k. zero)

which is more complicated than the solution we constructed by hand

plus = n. k. n succ k times′^ = n. k. n (plus k) zero

The difference in the latter solution is that it takes advantage of the fact that k (the second argument to times) never changes during the iteration. We have repeated here the definition of plus, for which there is a similar choice between two versions as for times.

6 The Schema of Primitive Recursion

It is easy to define very fast-growing functions by iteration, such as the exponential function, or the “stack” function iterating the exponential.

exp = b. e. e (times b) (succ zero) stack = b. n. n (exp b) (succ zero)

Everything appears to be going swimmingly until we think of a very simple function, namely the predecessor function defined by

pred 0 = 0 pred (n + 1 ) = n

You may try for a while to see if you can define the predecessor function, but it is difficult. The problem is that we have to go from s. z. s (... (s z))

L ECTURE N OTES T HURSDAY, S EPTEMBER 3, 2020

L2.10 Primitive Recursion

In other words, letpair passes the elements of the pair to a “continuation” k. Using letpair we start as

pred 2 (n + 1 ) = letpair (pred 2 n) (x. y... .)

If pred 2 satisfies it specification then reduction will substitute n for x and n ￿ 1 for y. From these we need to construct the pair ￿n + 1 , n￿ which we can do, for example, with ￿x + 1 , x￿. This gives us

pred 2 0 = ￿ 0 , 0 ￿ pred 2 (n + 1 ) = letpair (pred 2 n) (x. y. ￿x + 1 , x￿)

pred n = letpair (pred 2 n) (x. y. y)

6.2 Defining Pairs

The next question is how to define pairs and letpair. The idea is to simply abstract over the continuation itself! Then letpair isn’t really needed because the functional representation of the pair itself will apply its argument to the two components of the pair, but if want to write it out it would be the identity. ￿x, y￿ = k. k x y pair = x. y. k. k x y letpair = p. p

6.3 Proving the Correctness of the Predecessor Function

Summarizing the above and expanding the definition of letpair we obtain

pred 2 = n. n (p. p (x. y. pair (succ x) x)) (pair zero zero) pred = n. pred 2 n (x. y. y)

Let’s do a rigorous proof of correctness of pred, especially since we got it wrong when we worked in a hurry during lecture. For the representation of natural numbers, it is convenient to assume its correctness in the form

0 g c = c n + 1 g c = g (n g c)

Lemma 1 pred 2 n = ￿n, n ￿ 1 ￿

L ECTURE N OTES T HURSDAY, S EPTEMBER 3, 2020

Primitive Recursion L2.

Proof: By mathematical induction on n.

Base: n = 0. Then

pred 2 0 = 0 (.. .) (pair zero zero) = pair zero zero By repn. of 0 = ￿ 0 , 0 ￿ = ￿ 0 , 0 ￿ 1 ￿ By repn. of 0 and pairs

Step: n = m + 1. Then

pred 2 m + 1 = m + 1 (p. p (x. y. pair (succ x) x)) (pair zero zero) = (p. p (x. y. pair (succ x) x)) (m (p... .) (.. .)) By repn. of m + 1 = (p. p (x. y. pair (succ x) x)) (pred 2 m) By defn. of pred 2 = (p. p (x. y. pair (succ x) x)) ￿m, m ￿ 1 ￿ By ind. hyp. on m = ￿m, m ￿ 1 ￿ (x. y. pair (succ x) x) = pair (succ m) m By repn. of pairs = ￿m + 1 , m￿ By repn. of successor and pairs = ￿m + 1 , (m + 1 ) ￿ 1 ￿ By defn. of ￿

￿

Theorem 2 pred n = n ￿ 1

Proof: Direct, from Lemma ??.

pred n = (n. pred 2 n (x. y. y)) n = pred 2 n (x. y. y) = ￿n, n ￿ 1 ￿ (x. y. y) By Lemma ?? = (k. k n, n ￿ 1 ) (x. y. y) By repn. of pairs = n ￿ 1 ￿

An interesting consequence of the Church-Rosser Theorem is that if e = e ′^ where e ′^ is in normal form, then e ￿→∗ e ′^.

6.4 General Primitive Recursion

The general case of primitive recursion follows by a similar argument. Recall

f 0 = c f (n + 1 ) = h n (f n)

L ECTURE N OTES T HURSDAY, S EPTEMBER 3, 2020