Algorithmic Problem Solving, Thesis of Advanced Computer Programming

Download Algorithmic Problem Solving and more Thesis Advanced Computer Programming in PDF only on Docsity!

Principles and Applications

of

Algorithmic Problem Solving

João Fernando Peixoto Ferreira

Thesis submitted to The University of Nottingham

for the degree of Doctor of Philosophy

December 2010

To the memory of my father, Fernando Ferreira, whose encouragement

and inspiration shaped who I am today.

i

Acknowledgements

Many people supported me in preparing this thesis, and I would like to express my thanks and gratitude.

First, I would like to thank my supervisor Roland Backhouse for accepting me as his PhD student and for being an excellent supervisor. His experience and impressive ability to simplify and solve problems are inspiring, and always helped me whenever I got stuck. Also, his detailed and valuable feedback on my work will surely have an impact on the rest of my career. I have learnt a great deal from Roland.

I would also like to thank Luís Soares Barbosa, my co-supervisor, for his constant sup- port and encouragement. Luis is a dear and caring person, without whom I would never have started my PhD studies. I owe him a lot. Thanks, too, to José Nuno Oliveira, who, through his enthusiasm, inspired me to pursuit a research career.

Many people gave me useful and constructive feedback on parts of this thesis. In partic- ular, I would like to thank my internal examiner Natasha Alechina, who accompanied my progress since the first year. Also, thanks to my external examiner Shin-Cheng Mu for all the valuable comments and corrections. I am also grateful to Wei Chen, who is a good friend, collaborator, and who is always posing me interesting problems. Thanks to Arjan Mooij, Eric Macaulay, Jeremy Weissmann, and Paula Valença for their com- ments and suggestions on parts of this thesis. Thanks, too, to Filipe Oliveira, who was one of my favourite teachers at university and who introduced me to number theory.

Thanks to Nocas, Marta, Hélder, and Silvino for their good advice and for making me feel special. Also, thanks to Paulo Abrantes for the support and for always trying to make me smile. Special thanks to Sarawut Jindarat, who helped me a lot when I first arrived to Nottingham.

My mother, Irene Ferreira, has always been a constant source of inspiration and love. Mãe, obrigado pela tua presença e amor constantes. Sem ti, nunca teria chegado aqui.

Finally, my deepest and biggest thanks go to my beloved wife Alexandra. Her love, support, and encouragement are what make my life worthwhile.

iii

A Fundação para a Ciência e a Tecnologia apoiou o trabalho desenvolvido nesta tese com a bolsa de investigação SFRH/BD/24269/2005, financiada pelo POPH - QREN - Tipologia 4.1 - Formação Avançada (comparticipado pelo Fundo Social Europeu e por fundos nacionais do MCTES).

iv

• 1 Introduction
  • 1.1 Algorithmic problem solving: what is it all about?
  • 1.2 A first example
    • 1.2.1 A conventional proof
    • 1.2.2 An algorithmic proof
  • 1.3 Contributions
  • 1.4 Related work
    • 1.4.1 Mathematics of program construction
    • 1.4.2 Calculational proofs and structured derivations
    • 1.4.3 Education and research on algorithmic problem solving
    • 1.4.4 Classical problem solving
  • 1.5 Structure and organisation
• 2 Principles of Algorithmic Problem Solving
  • 2.1 Identifying algorithmic problems
  • 2.2 On the use of formalism
  • 2.3 Goal-oriented investigations
  • 2.4 On concision and avoidance of unnecessary detail
• 3 Techniques for Algorithmic Problem Solving
  • 3.1 Problem decomposition
  • 3.2 Symmetry
  • 3.3 Distributivity
  • 3.4 Invariants
  • 3.5 Proving program termination
  • 3.6 Algorithm Inversion
• 4 A Calculational and Algorithmic Approach to Elementary Number Theory
  • 4.1 Introduction
  • 4.2 Divisibility theory
    • 4.2.1 Integer division
    • 4.2.2 Division relation CONTENTS
    • 4.2.3 Constructing Euclid’s algorithm
    • 4.2.4 Greatest common divisor
  • 4.3 Euclid’s algorithm as a verification interface
    • 4.3.1 Exploring the invariant
    • 4.3.2 ▽ on the left side
    • 4.3.3 A geometrical property
  • 4.4 Euclid’s algorithm as a construction interface
    • 4.4.1 Distributivity properties
    • 4.4.2 Enumerating the rationals
  • 4.5 The theory of congruences
    • 4.5.1 Basic properties of congruences
    • 4.5.2 Modular exponentiation
    • 4.5.3 On a simple version of the Chinese remainder theorem
  • 4.6 Designing an algorithmic proof of the two-squares theorem
    • 4.6.1 Euclid’s algorithm
    • 4.6.2 Inverting Euclid’s algorithm
    • 4.6.3 Reversed sequences of vectors
    • 4.6.4 Length of the sequence of vectors
    • 4.6.5 Sum of two positive squares
    • 4.6.6 Discussion
  • 4.7 Conclusion
  • 4.8 Appendix: historical remarks on the trees of rationals
    • 4.8.1 Stern’s paper
    • 4.8.2 Brocot, the watchmaker
    • 4.8.3 Conclusion
• 5 Supporting the Teaching of Algorithmic Problem Solving
  • 5.1 Teaching scenarios
  • 5.2 How to create a teaching scenario
    • 5.2.1 Brief description and goals
    • 5.2.2 Problem statement
    • 5.2.3 Prerequisites
    • 5.2.4 Resolution
    • 5.2.5 Notes for the teacher
    • 5.2.6 Extensions and exercises
    • 5.2.7 Further reading
  • 5.3 A catalogue of teaching scenarios CONTENTS
• 6 Conclusion
  • 6.1 Future work
• References
• I Teaching Scenarios for Teaching Algorithmic Problem Solving
  • 1 Exploring Algebraic Symmetries
  • 2 Calculating Orderings Between Two Numbers
  • 3 The Island of Knights and Knaves
  • 4 Portia’s Casket
  • 5 A Logical Race
  • 6 A Calculational Proof of the Handshaking Lemma
  • 7 Moving a Heavy Armchair
  • 8 Exchanging the Values of Two Variables
  • 9 The Chameleons of Camelot
  • 10 Will This Algorithm Terminate?
  • 11 Constructing Euclid’s Algorithm
  • 12 The King Who Loved Diagonals

CHAPTER 1: INTRODUCTION

technology, rather than equip them with the skills to develop new applications or to solve new problems. In this thesis, we argue that this situation should change.

One of our main claims is that goal-oriented, calculational algorithmic skills can be used to enrich and reinvigorate the teaching of mathematics and computing. As a result, the main contribution of this thesis is educational material that supports that claim. The material is problem-driven and it is aimed at the boundary between pre- university and university level. Along the way, we also elaborate on some principles and techniques that can be used to solve algorithmic problems more effectively. In section 1.3, we explain in more detail the contributions of this thesis. But first, we show an example that may help to understand the nature of algorithms and their relation to problem solving.

1.2 A first example

A concrete example may help to understand better how algorithmic techniques can be used to do mathematics in a precise and effective way. We consider a well-known the- orem on the Fibonacci numbers. The theorem and the first proof we show were taken from the book [Bur05, p. 286], where the Fibonacci sequence is defined as follows^3 :

fib.1 = fib.2 = 1 fib. n = fib.( n − 1 ) + fib.( n − 2 ) for n ≥ 3.

The theorem states that any two consecutive Fibonacci numbers are coprime, that is, their greatest common divisor (gcd) is 1:

Theorem 1.2.1 For the Fibonacci sequence, gcd(fib. n , fib.( n + 1 )) = 1 for every n ≥ 1.

2

You may want to prove the theorem before reading further. We first show the proof taken from [Bur05], which, we believe, is representative of the conventional style in which these proofs are shown to students. We label it as “conventional proof”. We then show how we can use an algorithm and the notion of invariance to prove it.

1.2.1 A conventional proof

Let us suppose that the integer d > 1 divides both fib. n and fib.( n + 1 ). Then their differ- ence fib.( n + 1 ) − fib. n = fib.( n − 1 ) is also divisible by d. From this and from the relation (^3) In [Bur05], the nth (^) Fibonacci number is denoted as un. We denote it as fib. n ; the name fib is more informative and we use an infix dot for function application.

2

CHAPTER 1: INTRODUCTION

fib. n − fib.( n − 1 ) = fib.( n − 2 ), it may be concluded that d \ fib.( n − 2 ). Working back- ward, the same argument shows that d \ fib.( n − 3 ), d \ fib.( n − 4 ),... , and finally that d \ fib.1. But fib.1 = 1, which is certainly not divisible by any d > 1. This contradiction ends our proof.

Some comments on the conventional proof This proof captures several aspects of conventional mathematical method. First, it uses mostly natural language to express the connection between the steps. Second, it is based on implication rather than equal- ity (the use of natural language usually forces one to use the connective “then”, which corresponds to implication). Third, it is a proof by contradiction: we start by assuming that d is a common divisor of fib. n and fib.( n + 1 ) at least 2, but we reach the contradic- tion that d has to divide 1. Other conventional aspects are the use of a prefix notation to the associative gcd operator and the use of the so-called dot-dot-dot notation (... ), which reveals some imprecision in the argument. (By using a prefix notation for gcd, the au- thor forces a syntactic distinction between the equivalent expressions gcd( m , gcd( n , p )) and gcd(gcd( m , n ), p ). In what follows, we change to an infix notation.) Finally, one could also argue that the proof is not at all clear about the properties being used (e.g., which property is the author using when he writes “the difference is also divisible”?).

In this thesis, we will discuss most of these conventional aspects and we will propose some alternatives that we think are better. We will, for example, avoid the use of natural language to connect steps in our arguments, and we will use a systematic proof format that allows us to be more precise about the properties that we use.

1.2.2 An algorithmic proof

More than two thousand years ago (c. 300 B.C.), in Book 7, Propositions 1 and 2, of his seminal mathematical treatise Elements [HE56, p. 296], Euclid has given a method to compute the greatest common divisor of two positive numbers^4. That method is now known as Euclid’s algorithm and, using the Guarded Command Language (GCL), we can formulate it as:

{ 0 < m ∧ 0 < n } x , y := m , n ; { Invariant : m gcd n = x gcd y } (^4) According to Donald Knuth [Knu97, p. 334], some scholars believe that the method was known up to 200 years earlier. Knuth also writes that we might call Euclid’s method the granddaddy of all algorithms, because it is the oldest nontrivial algorithm that has survived to the present day.

3

CHAPTER 1: INTRODUCTION

The goal is to prove that fib.( n + 1 ) gcd fib. n = 1, i.e., the final value of the variables x and y is 1. Observing that the Fibonacci sequence is increasing, we can simplify part of the postcondition as follows:

x and y are two consecutive Fibonacci numbers ∧ x = y = { there are only two equal consecutive Fibonacci numbers: fib.1 and fib.2, which are both 1 } x = y = 1.

This proves that on termination the value of fib.( n + 1 ) gcd fib. n is 1. Because Euclid’s algorithm always terminates, we conclude that the following algorithm establishes the- orem 1.2.1:

x , y := fib.( n + 1 ) , fib. n ; { Invariant : x and y are two consecutive Fibonacci numbers ∧ fib.( n + 1 ) gcd fib. n = x gcd y } do y < x → x := x − y 2 x < y → y := y − x od { x = y = 1 ∧ fib.( n + 1 ) gcd fib. n = 1 = 1 }

Although this proof based on Euclid’s algorithm may seem more complex than the one labelled as conventional, we will show in this thesis that we can systematise it. We hope to convince the reader that the emphasis on algorithmic skills and techniques can indeed be used to reinvigorate mathematics education!

1.3 Contributions

The main contribution of this thesis is educational material, capturing calculational and algorithmic problem-solving techniques, that supports the teaching of mathemat- ics and computing. The material is problem-driven, it is aimed at the boundary be- tween pre-university and university level, and it is divided in two main parts.

First, in chapter 4 , we show how a fresh approach to introductory number theory that focuses on the algorithmic content of the theory can combine practicality with math- ematical elegance. We prove both old and well-known, and new and previously un-

5

CHAPTER 1: INTRODUCTION

known, theorems related with the greatest common divisor and rational numbers. For example, based on Euclid’s algorithm, we derive the following new results: we calcu- late sufficient conditions for a natural-valued function to distribute over the greatest common divisor, we derive an efficient algorithm to enumerate the positive rational numbers in two different ways, and we develop a new and constructive proof of the two-squares theorem.

We believe that the material on number theory that we have developed can be used to support a course on elementary number theory. Nevertheless, and although the ma- terial shown contains educational remarks, we are convinced that the teaching of al- gorithmic problem solving is more effective if the teacher has access to detailed guide- lines on how to solve and present specific algorithmic problems. Towards that end, we propose the introduction of educational material in the form of teaching scenarios , which are fully worked out solutions to algorithmic problems together with detailed guidelines on the principles captured by the problem, how the problem is tackled, and how it is solved. So, the second part of the material is a set of teaching scenar- ios that illustrate the principles and techniques discussed in this thesis. The scenarios are example-driven and have a recreational flavour, making them especially suitable for extra-curricular math clubs. Although they can be directly used by the students, they are primarily written for the teacher. Moreover, they are designed to promote self-discovery, since we believe that the success of teaching depends on the amount of discovery that is left for the students: if the teacher discloses all the information needed to solve a problem, students act only as spectators and become discouraged; if the teacher leaves all the work to the students, they may find the problem too difficult and become discouraged too. Scenarios are designed to maintain a balance between these two extremes. Some of the problems and solutions shown in the teaching scenar- ios are not new, but we capture them in a new and accessible way: as a catalogue of problems and solutions having a consistent format.

1.4 Related work

1.4.1 Mathematics of program construction

Many principles and ideas discussed in this thesis were created or developed by com- puting scientists working in the area of mathematics of program construction. Adopting a simplistic view, we can say that this area started in the 1960s, when programmers started recognising that there were serious problems in the programming field and that it was necessary to prove the correctness of programs. At the time, software engi-

6

CHAPTER 1: INTRODUCTION

PhD thesis of Netty van Gasteren “On the shape of mathematical arguments” [vG90], where she presents a study about proofs (proofs of correctness of programs included). In particular, in chapter 11 of her thesis, she shows how Euclid’s algorithm can be used to prove theorems about the greatest common divisor of two numbers. In this thesis we expand substantially the material shown in that chapter.

In a way, this project is a continuation of Van Gasteren’s study, but while she did a broad study, we are concerned specifically with algorithmic problems: construction of new algorithms and usage of algorithmic skills to demystify mathematical invention.

1.4.2 Calculational proofs and structured derivations

One of the products of the attempts to streamline mathematical arguments mentioned above is the so-called calculational method, which aims at reducing proofs as much as possible to elementary syntactic calculation. The calculational proof style has been adopted widely by the community of computing scientists working on formal pro- gramming methods. For example, the textbooks written by Roland Backhouse [Bac03], by Richard Bird and Oege de Moor [BdM96], by David Gries and Fred Schneider [GS93], and by Jan van de Snepscheut [vdS93] are well-known examples.

Gries and Schneider have also studied the use of calculational proofs for teaching mathematics [GS95]. Also, Back et al. introduced the concept of structured derivations [BGvW96, BvW97, BPSvW04, Bac09, BvW06], which is a further development of the calculational proof style created by Feijen and Dijkstra. Essentially, structured deriva- tions add a mechanism for doing subderivations and for handling assumptions in proofs. Moreover, in 2001, they have initiated a study in Finland to investigate whether structured derivations could be used to integrate logic, proof and formal reasoning throughout secondary-school mathematics education [BvW06]. The results were posi- tive and the test group outperformed the control group.

Two of the main resources on the calculational method are the websites “E. W. Dijkstra Archive” (http://userweb.cs.utexas.edu/users/EWD) and “mathmeth.com – Disci- pline in Thought” (http://mathmeth.com).

1.4.3 Education and research on algorithmic problem solving

Computing science is all about solving algorithmic problems (or, as some authors pre- fer to say, it is all about instructing computers to solve problems). Below, we briefly survey related projects that aim at using and improving algorithmic skills and tech-

CHAPTER 1: INTRODUCTION

niques.

Algorithmic Problem Solving The algorithmic problem solving research group at the University of Nottingham conducts research into mathematical method, in partic- ular the problem-solving skills involved in the formulation and solution of algorithmic problems. It also offers a module entitled “Algorithmic Problem Solving” to first-year undergraduates in computing science. As Roland Backhouse explains in [Bac06], the name of the module is deliberately ambiguous. Parsed as algorithmic-problem solving, it is about solving problems that involve the construction of an algorithm for their solu- tion. Parsed as algorithmic problem-solving, it is about problem solving in general, us- ing the principles that have been learnt in the development of correct-by-construction algorithm-design techniques.

The material included in this thesis was developed in the context of the group’s re- search plan. It was also done in the context of MATHIS, a project that aims to reinvigo- rate secondary-school mathematics by exploiting insights of the dynamics of algorith- mic problem solving [FMBB09].

Computer Science Unplugged Our work is related to the work developed within the project “Computer Science Unplugged” [BWF06], whose goal is to teach principles of computing science through games and puzzles. They provide a series of activity worksheets that can be directly used in the classroom. These worksheets are similar to the teaching scenarios that we propose, but their goals are slightly different: whilst they want to convey general principles and ideas of computing, we want to focus on calculational and algorithmic principles and techniques that can be used to reinvigorate mathematics. Also, their material is suitable for people of all ages and the material shown in this thesis is aimed at the boundary between pre-university and university level.

The project was started by Tim Bell, Mike Fellows and Ian Witten, and is now being explored by several dozen contributors working in many countries (including New Zealand, USA, Sweden, Australia, China, Korea, Taiwan and Canada). Additional in- formation about the project can be found at the website http://csunplugged.org.

Computational thinking We can say that our work fits with what is now usually called “computational thinking” [Win06]. We, too, want to transfer skills created and developed within computing science and we want to illustrate the value of compu- tational thinking to everyone interested in problem solving. In particular, we believe

CHAPTER 1: INTRODUCTION

Kits, cats, sacks, wives How many were going to St Ives?

Figure 1.1: The Rhind Mathematical Papyrus, written around 1650 B.C.

One of the contemporary leaders of this trend was Martin Gardner, who published thousands of puzzles in his books and various journals. Also, Zbigniew and Matthew Michalewicz recently published the book Puzzle-Based Learning: An introduction to crit- ical thinking, mathematics, and problem solving [MM08], where they support the teaching of problem solving based on puzzles that are inherently unstructured. We are sympa- thetic with their view and we also support a problem-driven approach to algorithmic problem solving. For example, most of the problems discussed in the teaching scenar- ios are recreational.

1.5 Structure and organisation

The goal-oriented, calculational algorithmic skills that we believe can be used to enrich and reinvigorate the teaching of mathematics and computing are presented and dis- cussed in chapters 2 and 3. In chapter 2 , we discuss principles of algorithmic problem

CHAPTER 1: INTRODUCTION

solving, that is, general rules that we think should be used whenever solving (algo- rithmic) problems. In chapter 3 , we discuss techniques for algorithmic problem solving. The main difference between the two is that, whilst principles apply to all problems, the same problem can be solved using different techniques. To make these chapters accessible to a wider audience, we illustrate the principles and techniques with simple and recreational examples. The material shown in these two chapters is essentially a summary of some relevant material that the community of “mathematics and program construction” has been developing.

To prove that these principles and techniques can be used to do mathematics in a prac- tical way, we use them in chapter 4 to rewrite some material on elementary number theory. We prove both old and well-known, and new and previously unknown, theo- rems.

We think that the best way to convey our message is to provide abundant and suffi- ciently varied educational material. So, in chapter 5 , we propose the introduction of educational material in the form of teaching scenarios, we explain how we think teach- ing scenarios should be constructed, and we describe a catalogue of scenarios that is included in appendix I. The catalogue in appendix suggests problems that can be used to illustrate the principles and techniques discussed in this thesis.

Finally, in chapter 6 , we discuss what was achieved and we suggest some directions for future work.