A Course in Discrete Structures, Study notes of Discrete Mathematics

Download A Course in Discrete Structures and more Study notes Discrete Mathematics in PDF only on Docsity!

A Course in Discrete Structures

Rafael Pass

Wei-Lung Dustin Tseng

Preface

Discrete mathematics deals with objects that come in discrete bundles, e.g., 1 or 2 babies. In contrast, continuous mathematics deals with objects that vary continuously, e.g., 3.42 inches from a wall. Think of digital watches versus analog watches (ones where the second hand loops around continuously without stopping). Why study discrete mathematics in computer science? It does not directly help us write programs. At the same time, it is the mathematics underlying almost all of computer science. Here are a few examples:

• Designing high-speed networks and message routing paths.
• Finding good algorithms for sorting.
• Performing web searches.
• Analysing algorithms for correctness and efficiency.
• Formalizing security requirements.
• Designing cryptographic protocols.

Discrete mathematics uses a range of techniques, some of which is sel- dom found in its continuous counterpart. This course will roughly cover the following topics and specific applications in computer science.

1. Sets, functions and relations
2. Proof techniques and induction
3. Number theory a) The math behind the RSA Crypto system
4. Counting and combinatorics
5. Probability a) Spam detection b) Formal security
6. Logic a) Proofs of program correctness
7. Graph theory

i

Contents

• 1 Sets, Functions and Relations Contents iii
  • 1.1 Sets
  • 1.2 Relations
  • 1.3 Functions
  • 1.4 Set Cardinality, revisited
• 2 Proofs and Induction
  • 2.1 Basic Proof Techniques
  • 2.2 Proof by Cases and Examples
  • 2.3 Induction
  • 2.4 Inductive Definitions
  • 2.5 Fun Tidbits
• 3 Number Theory
  • 3.1 Divisibility
  • 3.2 Modular Arithmetic
  • 3.3 Primes
  • 3.4 The Euler φ Function
  • 3.5 Public-Key Cryptosystems and RSA
• 4 Counting
  • 4.1 The Product and Sum Rules
  • 4.2 Permutations and Combinations
  • 4.3 Combinatorial Identities
  • 4.4 Inclusion-Exclusion Principle
  • 4.5 Pigeonhole Principle
• 5 Probability
  • 5.1 Probability Spaces
  • 5.2 Conditional Probability and Independence
  • 5.3 Random Variables
  • 5.4 Expectatation
  • 5.5 Variance
• 6 Logic
  • 6.1 Propositional Logic
  • 6.2 Logical Inference
  • 6.3 First Order Logic
  • 6.4 Applications
• 7 Graphs
  • 7.1 Graph Isomorphism
  • 7.2 Paths and Cycles
  • 7.3 Graph Coloring
  • 7.4 Random Graphs [Optional]
• 8 Finite Automata
  • 8.1 Deterministic Finite Automata
  • 8.2 Non-Deterministic Finite Automata
  • 8.3 Regular Expressions and Kleene’s Theorem
• A Problem Sets
  • A.1 Problem Set A
• B Solutions to Problem Sets
  • B.1 Problem Set A

2 sets, functions and relations

• R+, the positive reals

Given a collection of objects (a set), we may want to know how large is the collection:

Definition 1.3 (Set cardinality). The cardinality of a set A is the number of (distinct) objects in A, written as |A|. When |A| ∈ N (a finite integer), A is a finite set; otherwise A is an infinite set. We discuss the cardinality of infinite sets later.

Example 1.4. |{ 1 , 2 , 3 }| = |{ 1 , 2 , { 1 , 2 }}| = 3.

Given two collections of objects (two sets), we may want to know if they are equal, or if one collection contains the other. These notions are formalized as set equality and subsets:

Definition 1.5 (Set equality). Two sets S and T are equal, written as S = T , if S and T contains exactly the same elements, i.e., for every x, x ∈ S ↔ x ∈ T.

Definition 1.6 (Subsets). A set S is a subset of set T , written as S ⊆ T , if every element in S is also in T, i.e., for every x, x ∈ S → x ∈ T. Set S is a strict subset of T, written as S ⊂ T if S ⊆ T , and there exist some element x ∈ T such that x /∈ S.

Example 1.7.

• { 1 , 2 } ⊆ { 1 , 2 , 3 }.
• { 1 , 2 } ⊂ { 1 , 2 , 3 }.
• { 1 , 2 , 3 } ⊆ { 1 , 2 , 3 }.
• { 1 , 2 , 3 } 6 ⊂ { 1 , 2 , 3 }.
• For any set S, ∅ ⊆ S.
• For every set S 6 = ∅, ∅ ⊂ S.
• S ⊆ T and T ⊆ S if and only if S = T.

Finally, it is time to formalize operations on sets. Given two collection of objects, we may want to merge the collections (set union), identify the objects in common (set intersection), or identify the objects unique to one collection (set difference). We may also be interested in knowing all possible ways of picking one object from each collection (Cartesian product), or all possible ways of picking some objects from just one of the collections (power set).

Definition 1.8 (Set operations). Given sets S and T , we define the following operations:

1.1. SETS 3

• Power Sets. P(S) is the set of all subsets of S.
• Cartesian Product. S × T = {(s, t) | s ∈ S, t ∈ T }.
• Union. S ∪ T = {x | x ∈ S or x ∈ T }, set of elements in S or T.
• Intersection. S ∩ T = {x | x ∈ S, x ∈ T }, set of elements in S and T.
• Difference. S − T = {x | x ∈ S, x /∈ T }, set of elements in S but not T.
• Complements. S = {x | x ∈/ S}, set of elements not in S. This is only meaningful when we have an implicit universe U of objects, i.e., S = {x | x ∈ U, x /∈ S}.

Example 1.9. Let S = { 1 , 2 , 3 }, T = { 3 , 4 }, V = {a, b}. Then:

• P(T ) = {∅, { 3 }, { 4 }, { 3 , 4 }}.
• S × V = {(1, a), (1, b), (2, a), (2, b), (3, a), (3, b)}.
• S ∪ T = { 1 , 2 , 3 , 4 }.
• S ∩ T = { 3 }.
• S − T = { 1 , 2 }.
• If we are dealing with the set of all integers, S = {... , − 2 , − 1 , 0 , 4 , 5 ,... }.

Some set operations can be visualized using Venn diagrams. See Figure 1.1. To give an example of working with these set operations, consider the following set identity.

Theorem 1.10. For all sets S and T , S = (S ∩ T ) ∪ (S − T ).

Proof. We can visualize the set identity using Venn diagrams (see Figure 1.1b and 1.1c). To formally prove the identity, we will show both of the following:

S ⊆ (S ∩ T ) ∪ (S − T ) (1.1)

(S ∩ T ) ∪ (S − T ) ⊆ S (1.2) To prove (1.1), consider any element x ∈ S. Either x ∈ T or x /∈ T.

• If x ∈ T , then x ∈ S ∩ T , and thus also x ∈ (S ∩ T ) ∪ (S − T ).
• If x /∈ T , then x ∈ (S − T ), and thus again x ∈ (S ∩ T ) ∪ (S − T ).

To prove (1.2), consider any x ∈ (S ∩ T ) ∪ (S − T ). Either x ∈ S ∩ T or x ∈ S − T

• If x ∈ S ∩ T , then x ∈ S

1.2. RELATIONS 5

• [n] is the set { 0 , 1 ,... , n − 1 }.

Commonly seen set includes { 0 , 1 }n^ as the set of n-bit strings, and { 0 , 1 }∗ as the set of finite length bit strings. Also observe that |[n]| = n. Before we end this section, let us revisit our informal definition of sets: an unordered “collection” of objects. In 1901, Russel came up with the following “set”, known as Russel’s paradox^1 :

S = {x | x /∈ x}

That is, S is the set of all sets that don’t contain themselves as an element. This might seem like a natural “collection”, but is S ∈ S? It’s not hard to see that S ∈ S ↔ S ∈/ S. The conclusion today is that S is not a good “collection” of objects; it is not a set. So how will know if {x | x satisfies some condition} is a set? Formally, sets can be defined axiomatically, where only collections constructed from a careful list of rules are considered sets. This is outside the scope of this course. We will take a short cut, and restrict our attention to a well-behaved universe. Let E be all the objects that we are interested in (numbers, letters, etc.), and let U = E ∪ P(E) ∪ P(P(E)), i.e., E, subsets of E and subsets of subsets of E. In fact, we may extend U with three power set operations, or indeed any finite number of power set operations. Then, S = {x | x ∈ U and some condition holds} is always a set.

1.2 Relations

Definition 1.12 (Relations). A relation on sets S and T is a subset of S × T. A relation on a single set S is a subset of S × S.

Example 1.13. “Taller-than” is a relation on people; (A, B) ∈ ”Taller-than” if person A is taller than person B. “≥” is a relation on R; “≥”= {(x, y) | x, y ∈ R, x ≥ y}.

Definition 1.14 (Reflexitivity, symmetry, and transitivity). A relation R on set S is:

• Reflexive if (x, x) ∈ R for all x ∈ S.
• Symmetric if whenever (x, y) ∈ R, (y, x) ∈ R. (^1) A folklore version of this paradox concerns itself with barbers. Suppose in a town, the only barber shaves all and only those men in town who do not shave themselves. This seems perfectly reasonable, until we ask: Does the barber shave himself?

6 sets, functions and relations

• Transitive if whenever (x, y), (y, z) ∈ R, then (x, z) ∈ R

Example 1.15.

• “≤” is reflexive, but “<” is not.
• “sibling-of” is symmetric, but “≤” and “sister-of” is not.
• “sibling-of”, “≤”, and “<” are all transitive, but “parent-of” is not (“ancestor-of” is transitive, however).

Definition 1.16 (Graph of relations). The graph of a relation R over S is an directed graph with nodes corresponding to elements of S. There is an edge from node x to y if and only if (x, y) ∈ R. See Figure 1.2.

Theorem 1.17. Let R be a relation over S.

• R is reflexive iff its graph has a self-loop on every node.
• R is symmetric iff in its graph, every edge goes both ways.
• R is transitive iff in its graph, for any three nodes x, y and z such that there is an edge from x to y and from y to z, there exist an edge from x to z.
• More naturally, R is transitive iff in its graph, whenever there is a path from node x to node y, there is also a direct edge from x to y.

Proof. The proofs of the first three parts follow directly from the definitions. The proof of the last bullet relies on induction; we will revisit it later. 

Definition 1.18 (Transitive closure). The transitive closure of a relation R is the least (i.e., smallest) transitive relation R∗^ such that R ⊆ R∗.

Pictorially, R∗^ is the connectivity relation: if there is a path from x to y in the graph of R, then (x, y) ∈ R∗.

Example 1.19. Let R = {(1, 2), (2, 3), (1, 4)} be a relation (say on set Z). Then (1, 3) ∈ R∗^ (since (1, 2), (2, 3) ∈ R), but (2, 4) ∈/ R∗. See Figure 1.2.

Theorem 1.20. A relation R is transitive iff R = R∗.

Definition 1.21 (Equivalence relations). A relation R on set S is an equiva- lence relation if it is reflexive, symmetric and transitive.

Equivalence relations capture the every day notion of “being the same” or “equal”.

8 sets, functions and relations

• f : R+^ → R+, f (x) = x^2 is injective.
• f : R → R, f (x) = x^2 is not injective since (−x)^2 = x^2.

Definition 1.26 (Surjection). f : S → T is surjective (onto) if the image of f equals its range. Equivalently, for every t ∈ T , there exists some s ∈ S such that f (s) = t.

Example 1.27.

• f : N → N, f (x) = 2x is not surjective.
• f : R+^ → R+, f (x) = x^2 is surjective.
• f : R → R, f (x) = x^2 is not injective since negative reals don’t have real square roots.

Definition 1.28 (Bijection). f : S → T is bijective, or a one-to-one corre- spondence, if it is injective and surjective.

See Figure 1.3 for an illustration of injections, surjections, and bijections.

Definition 1.29 (Inverse relation). Given a function f : S → T , the inverse relation f −^1 on T and S is defined by (t, s) ∈ f −^1 if and only if f (s) = t.

If f is bijective, then f −^1 is a function (unique inverse for each t). Similarly, if f is injective, then f −^1 is a also function if we restrict the domain of f −^1 to be the image of f. Often an easy way to show that a function is one-to-one is to exhibit such an inverse mapping. In both these cases, f −^1 (f (x)) = x.

1.4 Set Cardinality, revisited

Bijections are very useful for showing that two sets have the same number of elements. If f : S → T is a bijection and S and T are finite sets, then |S| = |T |. In fact, we will extend this definition to infinite sets as well.

Definition 1.30 (Set cardinality). Let S and T be two potentially infinite sets. S and T have the same cardinality, written as |S| = |T |, if there exists a bijection f : S → T (equivalently, if there exists a bijection f ′^ : T → S). T has cardinality at larger or equal to S, written as |S| ≤ |T |, if there exists an injection g : S → T (equivalently, if there exists a surjection g′^ : T → S).

To “intuitively justify” Definition 1.30, see Figure 1.3. The next theorem shows that this definition of cardinality corresponds well with our intuition for size: if both sets are at least as large as the other, then they have the same cardinality.

1.4. SET CARDINALITY, REVISITED 9

X Y

(a) An injective function from X to Y.

X Y

(b) A surjective function from X to Y.

X Y

(c) A bijective function from X to Y. Figure 1.3: Injective, surjective and bijective functions.

Theorem 1.31 (Cantor-Bernstein-Schroeder). If |S| ≤ |T | and |T | ≤ |S|, then |S| = |T |. In other words, given injective maps, g : S → T and h : T → S, we can construct a bijection f : S → T.

We omit the proof of Theorem 1.31; interested readers can easily find multiple flavours of proofs online. Set cardinality is much more interesting when the sets are infinite. The cardinality of the natural numbers is extra special, since you can “count” the numbers. (It is also the “smallest infinite set”, a notion that is outside the scope of this course.) Definition 1.32. A set S is countable if it is finite or has the same cardinality as N+. Equivalently, S is countable if |S| ≤ |N+|. Example 1.33.

• { 1 , 2 , 3 } is countable because it is finite.
• N is countable because it has the same cardinality as N+; consider f : N+^ → N, f (x) = x − 1.

1.4. SET CARDINALITY, REVISITED 11

show that there cannot be a bijection with N+, we proceed by contradiction. Suppose f : N+^ → S is a bijection. We can then enumerate these strings using f , producing a 2-dimensional table of digits:

f (1) = s^1 = (d^11 , d^12 , d^13 ,... ) f (2) = s^2 = (d^21 , d^22 , d^23 ,... ) f (3) = s^3 = (d^31 , d^32 , d^33 ,... )

Now consider s∗^ = (1 − d^11 , 1 − d^22 , 1 − d^33 ,... ), i.e., we are taking the diagonal of the above table, and flipping all the digits. Then for any n, s∗^ is different from sn^ in the nth^ digit. This contradicts the fact that f is a bijection. 

Theorem 1.36. The real interval [0, 1] (the set of real numbers between 0 and 1, inclusive) is uncountable.

Proof. We will show that |[0, 1]| ≥ |S|, where S is the same set as in the proof of Theorem 1.35. Treat each s = (d 1 , d 2 ,... ) ∈ S as the real number between 0 and 1 with the binary expansion 0.d 1 d 2 · · ·. Note that this does not establish a bijection; some real numbers have two binary expansions, e.g., 0 .1 = 0. 0111 · · · (similarly, in decimal expansion, we have 0.1 = 0. 0999 · · · 2 ). We may overcome this “annoyance” in two ways:

• Since each real number can have at most two decimal representations (most only have one), we can easily extend the above argument to show that |S| ≤ |[0, 2]| (i.e., map [0, 1] to one representation, and [1, 2] to the other). It remains to show that |[0, 1]| = |[0, 2]| (can you think of a bijection here?).
• We may repeat Cantor’s diagonlization argument as in the proof of The- orem 1.35, in decimal expansion. When we construct s∗, avoid using the digits 9 and 0 (e.g., use only the digits 4 and 5). 

A major open problem in mathematics (it was one of Hilbert’s 23 famous problems listed 1900) was whether there exists some set whose cardinality is between N and R (can you show that R has the same cardinality as [0, 1]?). Here is a naive candidate: P(N). Unfortunately, P(N) has the same cardi- nality as [0, 1]. Note that every element S ∈ P(N) corresponds to an infinitely long sequence over digits { 0 , 1 } (the nth^ digit is 1 if and only if the number n ∈ S). Again, we arrive at the set S in the proof of Theorem 1.35. (^2) For a proof, consider letting x = 0. 0999 · · · , and observe that 10x − x = 0. 999 · · · −

1. 0999 · · · = 0.9, which solves to x = 0.1.

12 sets, functions and relations

The Continuum Hypothesis states that no such set exists. G¨odel and Cohen together showed (in 1940 and 1963) that this can neither be proved nor disproved using the standard axioms underlying mathematics (we will talk more about axioms when we get to logic).

14 proofs and induction

Claim 2.2. Let n be an integer. If n^2 is even, then n is even.

Proof by contrapositive. Suppose that n is not even. Then by Claim 2.1, n^2 is not even as well. (Yes, the proof ends here.) 

A proof by contradiction, on the other hand, assumes both that the premise X is true and the conclusion Y is false, and reach a logical fallacy. We give another proof of Claim 2.2 as example.

Proof by contradiction. Suppose that n^2 is even, but n is odd. Applying Claim 2.1, we see that n^2 must be odd. But n^2 cannot be both odd and even! 

In their simplest forms, it may seems that a direct proof, a proof by con- trapositive, and a proof and contradiction may just be restatements of each other; indeed, one can always phrase a direct proof or a proof by contrapositive as a proof by contradiction (can you see how?). In more complicated proofs, however, choosing the “right” proof technique sometimes simplify or improve the aesthetics of a proof. Below is an interesting use of proof by contradiction.

Theorem 2.3.

2 is irrational.

Proof by contradiction. Assume for contradiction that

2 is rational. Then there exists integers p and q, with no common divisors, such that

2 = p/q (i.e., the reduced fraction). Squaring both sides, we have:

p^2 q^2 ⇒ 2 q^2 = p^2

This means p^2 is even, and by Claim 2.2 p is even as well. Let us replace p by 2 k. The expression becomes:

2 q^2 = (2k)^2 = 4k^2 ⇒ q^2 = 2k^2

This time, we conclude that q^2 is even, and so q is even as well. But this leads to a contradiction, since p and q now shares a common factor of 2. 

We end the section with the (simplest form of the) AM-GM inequality.

Theorem 2.4 (Simple AM-GM inequality). Let x and y be non-negative reals. Then, x + y 2

xy

2.2. PROOF BY CASES AND EXAMPLES 15

Proof by contradiction. Assume for contradiction that x + y 2

xy

⇒

(x + y)^2 < xy squaring non-negative values ⇒ x^2 + 2xy + y^2 < 4 xy ⇒ x^2 − 2 xy + y^2 < 0 ⇒ (x − y)^2 < 0

But this is a contradiction since squares are always non-negative. 

Note that the proof Theorem 2.4 can be easily turned into a direct proof; the proof of Theorem 2.3, on the other hand, cannot.

2.2 Proof by Cases and Examples

Sometimes the easiest way to prove a theorem is to split it into several cases.

Claim 2.5. (n + 1)^2 ≥ 2 n^ for all integers n satisfying 0 ≤ n ≤ 5.

Proof by cases. There are only 6 different values of n. Let’s try them all:

n (n + 1)^2 2 n 0 1 ≥ 1 1 4 ≥ 2 2 9 ≥ 4 3 16 ≥ 8 4 25 ≥ 16 5 36 ≥ 32 

Claim 2.6. For all real x, |x^2 | = |x|^2.

Proof by cases. Split into two cases: x ≥ 0 and x < 0.

• If x ≥ 0, then |x^2 | = x^2 = |x|^2.
• If x < 0, then |x^2 | = x^2 = (−x)^2 = |x|^2. 

When presenting a proof by cases, make sure that all cases are covered! For some theorems, we only need to construct one case that satisfy the theorem statement.

Claim 2.7. Show that there exists some n such that (n + 1)^2 ≥ 2 n.