Theory of computing, Manuais, Projetos, Pesquisas de Informática

Baixe Theory of computing e outras Manuais, Projetos, Pesquisas em PDF para Informática, somente na Docsity!

6, 71 c

8 9 t 5 , s•

1O

A-8 __ 19 2

1 21 2

Hit

A14 (^) T3...I3 (^1 ) (^18) it,

809 #10 21 2)

15 14i81 11

Theory of Computing

A Gentle (^) Introduction

Library of Congress Cataloging-in-Publication Data

Kinber, Efim Theory of computing: a gentle introduction / Carl Smith, Efim Kinber, p. cm. Includes bibliological references^ and^ index. ISBN: 0-13-027961-

1. Electronic data processing. I. Smith, Carl, 1950 April 25 - II. Title. QA76 .K475 2001 004-dc2l 00-

Acquisition Editor: Petra J. Recter Editorial Assistant: Sarah Burrows Vice President and Editorial Director of Engineering and Computer Science: Marcia Horton Assistant Vice President and Director of Production and Manufacturing, ESM: David W. Riccardi Editorial/Production Supervision: BarbaraA. Till Managing Editor: David A. George Executive Managing Editor: Vince O'Brien Manufacturing Buyer: Pat Brown Manufacturing Manager: Trudy Pisciotti Senior Marketing Manager: Jennie Burger Marketing Assistant: Cynthia Szollose Creative Director: Paul Belfanti Art Director: Jayne Conte Cover Designer: Bruce Kenselaar

0© 200L by Prentice-Hall, Inc. Upper Saddle River, New Jersey 07458

All rights reserved. No part of this book may be reproduced, in any form or by any means, without permission in writing from the publisher.

The author and publisher of this book have used their best efforts in preparing this book. These efforts include the development, research, and testing of the theories and programs to determine their effectiveness. The author and publisher make no warranty of any kind, expressed or implied, with regard to these programs or the documentation contained in this book. The author and publisher shall not be liable in any event for incidental or consequential damages in connection with, or arising out of, the furnishing, performance, or use of these programs.

Printed in the United States of America

10 9 8 7 6 5 4 3 2

Prentice-Hall International (UK) Limited, London Prentice-Hall of Australia Pty. Limited, Sydney Prentice-Hall Canada Inc., Toronto Prentice-Hall Hispanoamericana, S.A., Mexico Prentice-Hall of ndia Private. Limited, New Delhi Prentice-Hall of Japan, Inc.,Tokyo Pearson Education Asia Pte. Ltd. Editor Prentice-Hall do Brasil, Ltda., Rio de Janeiro

Dedicated to those who created the material herein

and to those who taught it to us.

• 1 Introduction Preface xiii - 1.1 Why Study the Theory of Computing? - 1.2 What Is Computation? - 1.3 The Contents of This Book - 1.4 Mathematical Preliminaries - Exercises
• 2 Finite Automata - 2.1 Deterministic Finite Automata - 2.2 Nondeterministic Finite Automata - 2.3 Determinism versus Nondeterminism - 2.4 Regular Expressions - 2.5 Nonregular Languages - 2.6 Algorithms for Finite Automata - 2.7 The State Minimization Problem
  • Exercises
• 3 Context-Free Languages - 3.1 Context-Free Grammars - 3.2 Parsing - 3.3 Pushdown Automata - 3.4 Languages and Automata - 3.5 Closure Properties - 3.6 Languages That Are Not Context-Free - 3.7 Chomsky Normal Form - 3.8 Determinism - Exercises
• 4 Turing Machines - 4.1 Definition of a Turing Machine - 4.2 Computations by Turing Machines viii CONTENTS
  • 4.3 Extensions of Turing Machines - 4.3.1 Multiple Tapes - 4.3.2 Multiple Heads - 4.3.3 Two-Dimensional Tapes - 4.3.4 Random Access Turing Machines
  • 4.4 Nondeterministic Turing Machines
  • 4.5 Turing Enumerable Languages
  • Exercises
• 5 Undecidability - 5.1 The Church-Turing Thesis - 5.2 Universal Turing Machines - 5.3 The Halting Problem - 5.4 Undecidable Problems - Exercises
• 6 Computational Complexity - 6.1 The Definition and the Class P - 6.2 The Class AýP - 6.3 AHP-Completeness - Exercises
• References
• List of Symbols
• Index - 1.1 A Function List of Figures
  • 2.1 A Finite Automaton
  • 2.2 State Transition Diagram
  • 2.3 Recognizing Two Consecutive a's
  • 2.4 Recognizing the Complement
  • 2.5 Mystery Automaton
  • 2.6 Mystery Automaton
  • 2.7 Recognizing an Even Number of a's and Odd Number of b's
  • 2.8 The Finite State Diagram of a Newspaper Vending Machine.
  • 2.9 Combining Two Automata
  • 2.10 Accepting All a's or All b's
  • 2.11 Start and End with a b
  • 2.12 With Extra States
  • 2.13 Multiple Possible Transitions
  • 2.14 Replacement Transition
• 2.15 A New Connection
• 2.16 An e-Move
• 2.17 An e-Move Eliminated
• 2.18 Segment of a Nondeterministic Finite Automaton
• 2.19 Corresponding Deterministic Segment
• 2.20 Untransformed Automaton
• 2.21 Transformed Automaton
• 2.22 Recognizing Strings Containing abbab
• 2.23 Deterministic Version of Figure 2.22
• 2.24 Automaton A
• 2.25 Automaton B
• 2.26 The Union of A and B
• 2.27 Concatenation of L and M
• 2.28 Kleene Star of A
• 2.29 Singleton Automata
• 2.30 Transformation Step 1 of
• 2.31 Transformation Step 2 of
• 2.32 Transformation Step 3 of x LIST OF FIGURES
• 2.33 Transformation Step 4 of
• 2.34 Transformation Step 5 of
• 2.35 An Expression Diagram
• 2.36 Initial Automaton
• 2.37 Converted Automaton
• 2.38 Automaton with One Accepting State
• 2.39 Steps 1, 2 and
• 2.40 Example Input for Algorithm B
• 2.41 Node 2 Deleted
• 2.42 Node 3 deleted
• 2.43 Final Expression Diagram
• 2.44 Automaton with an Unreachable State
• 2.45 Without the Unreachable State - and q. 2.46 Distinguishability of r and r' Related to the Distinguishability of q
  • 2.47 A Minimal Deterministic Finite Automaton
  • 3.1 Parse Trees for John bought car
  • 3.2 Parse Tree for the First Derivation of aa
  • 3.3 Parse Tree for the Second Derivation of aa
  • 3.4 Pushdown Automaton
  • 3.5 Operation of the Automaton Accepting abbbaaaabb
  • 3.6 Computation of Automaton A on ababbaba
  • 3.7 Schematic Parse Tree for w
  • 3.8 Schematic Parse Tree for w = uvxyz
  • 4.1 A Turing Machine
  • 4.2 A Sequence of Turing Machines
  • 4.3 Branching Turing Machines
  • 4.4 A Loop
  • 4.5 Writing b over a
  • 4.6 Moving Right and Writing a
  • 4.7 Erasing the Input
  • 4.8 Finding the First Blank on the Right
  • 4.9 The Copying Machine Cp
  • 4.10 Computing the Function f(w) = wa
  • 4.11 The Shifting Machine Shl
  • 4.12 Multitape Turing Machine
  • 4.13 A Multitrack Tape
  • 4.14 Simulating k Tapes by 2k Tracks
  • 4.15 Initial Configuration of Tracks
  • 4.16 Two-Dimensional Tape with Numbered Cells
  • 4.17 Simulation of a Two-Dimensional Tape in One Dimension.
• 4.18 Random Access Thring Machine LIST OF FIGURES xi
• 4.19 Commands of RAM Machine
• 4.20 Dovetailing
• 4.21 The Tree of Nondeterministic Computations
  • 5.1 U after Initialization on (M )(w)
  • 5.2 The Diagonal Function
• 5.3 The Program CONTR
• 6.1 An Undirected Weighted Graph
• 6.2 A Graph with a Hamiltonian Cycle
• 6.3 A Clique of Size
• 6.4 Algorithm A
• 6.5 The Graph Derived from a Boolean Formula f

Preface

The theory of computing provides students with a background (^) in the fundamentals of computing with which to achieve (^) a deeper understanding of contemporary com- puting systems. Computers are evolving and (^) developing at a dizzying rate. Yet, the fundamentals (^) of pattern matching and programming language design and im- plementation have remained unchanged. In this book, we present a perspective (^) on computing that will always apply (^) since it addresses only the fundamental issues. In this way, mastery of the topics in this book (^) will give the reader a perspective from which to understand all computers, not just the ones in use today. We cover the automata and regular languages that serve as the basis for pattern- matching algorithms, communication protocols, control (^) mechanisms, sequential cir- cuit design, and a host of other ubiquitous applications. The basic principles behind the parsing of computer languages are also presented as well as a simple, yet gen- eral, model of all computation. This leads to some important distinctions. Some problems of interest turn out to be unsolvable. That is, not only can they not be solved by today's computers, (^) but they will also never be solved by any future com- puter. Some problems that are solvable are nonetheless so (^) difficult that they are called intractable. These intractable problems cannot (^) be solved efficiently by any current computer. Furthermore, (^) advances in the design of computers will not make a significant (^) difference. The collection of intractable problems includes several that must be solved in some way every day. The "solutions" to these problems are generally (^) approxima- tions. Such problems are ubiquitous in everyday life. Consider the shirt you are wearing. The cloth with which it was made was delivered to the clothing manufac- turer in a bolt of cloth. The bolt of cloth is a large roll, perhaps two meters (^) wide and hundreds of meters in length. The manufacturer must cut this cloth into pieces of different sizes for different size shirts and for other garments that use the same material. Some of the material will naturally be wasted - scraps will be left over since the pieces that are cut do not form perfect rectangles, (^) two meters per side. How to cut (^) this cloth so as to minimize the leftover useless scraps is a provably intractable problem. Imagine how much cloth could be wasted in an attempt to find the best solution! Such problems occur frequently in computing. (^) It is vital for any computer professional to be able to recognize an intractable or unsolvable problem when confronted with one. Our intent is to (^) impart such knowledge to the

xiii

Theory of Computing

A Gentle Introduction

Chapter 1

Introduction

4 CHAPTER^ 1.^ INTRODUCTION

the computation, the original data, is the coins you put in the slot and buttons(s)

you press to select the product. The coins are^ moved^ to^ a^ holding^ area and^ some

electronic impulses are generated. The coins have been altered into electronic im-

pulses that are now moving through wires. The data now^ contain information about

how many coins you entered and which^ denominations.^ These^ impulses are^ trans-

formed into signals that trigger the gate that lets the product^ you^ selected^ move

to the area of the machine where it can be collected. So, the impulses encoding

your selection and the coins entered are changed into a product, and perhaps some

change in another area of the machine.

When you use an automatic teller machine, another sequence of events happens

that looks like a computation. You insert your card and then use a keypad to

input more data. Sometimes the keypad is organized around a display and you

must make several selections. As the display changes, so does the meaning of each

button. Nonetheless, you are still inputting data to the machine. After a while,

you get some cash as output and maybe a transaction receipt as well. So, according

to our working definition, a computation has taken place. A more detailed analysis

reveals even more computations. After you have entered all the data, such as pin

codes and transaction type, the bank machine contacts another computer holding

your account information. This connection may be complicated depending on how

far away the bank machine you are using is from the location of the computer

holding the needed information. It is not unusual for such connections to cross

state and even country borders. The computer with the account information does

what is typically considered a computation and sends some information, like an

authorization number, back to the bank machine you are using. Few data are

being transfered between the computer and the bank machine, but the amount of

withdrawal and account number information is transformed into an authorization

to dispense cash.

Another computation that has layers and layers happens every time you use a

computer. At the outermost level, the keystrokes and mouse movements you enter

are transformed into the display on the screen. At the next level, the keystrokes

activate some program which runs on the computer. One level deeper, the program

is meticulously telling the computer to get data from memory and move them into

the processing chip where they are transformed into commands for the display

chip(s). There is another level, what happens inside the chip, but we won't get that

detailed. The basic idea of computation as movement and transformation of data

should be clear.

1.3 The Contents of This Book

In Chapter 2, we study finite automata. These simple devices serve as the basis

for all pattern-matching algorithms. All the World Wide Web search engines use

pattern matching as a host of other applications. Interacting automata, not con-

sidered in this text, are the basis for all communication protocols used in computer

1.4. MATHEMATICAL PRELIMINARIES 5

networks of all kinds.

In Chapter 3 we study context-free languages which are used to describe all

contemporary programming languages. Topics such as parsing are discussed. A

fully general model of computers is presented in Chapter 4. The limitations of this

model are revealed in Chapter 5. Finally, in Chapter 6, we consider the limitations

of feasible computation.

1.4 Mathematical Preliminaries

This is a technical book. We have tried to make the presentation as intuitive as

possible, often deviating from the traditional presentation of iterating definition,

theorem, and proof. However, we must rely on some mathematical notation to

make our presentation clear and precise. We anticipate that every student using

this book will have seen all the concepts in this section before. They are presented

mainly for review and to establish notational conventions.

Computers typically deal with numbers and strings. We will do so as well. The

only numbers that we will consider are the natural numbers, 0, 1, 2, ... which

we denote by N. Actually, the natural numbers can be pulled, more or less, from

thin air by the operation of collection. For example, even though we start with

no numbers, we can still collect what we have. Visually, this collection of nothing

appears as { }. Call this representation zero. Now we have something, so we can collect our zero: {{ }}. Call this one. Now we have two things to collect: {{ }, {{ }}}. This last collection, call it two, is the collection of two items:

"* The collection of nothing, and

"*The collection of the collection of nothing.

Now we have three objects to collect and the process continues, defining represen-

tations for all the natural numbers in terms of the operation of collection. Please

note that this definition of natural numbers is an inductive definition. In this

style of definition a base object is specified (in this case, the empty object) and an

operation to produce new objects from previously defined objects is specified (in

this case, collection).

Once we have natural numbers, we can collect those as well forming sets. For

example, the set of prime numbers less than 10 is denoted by {2, 3, 7}. Set member-

ship is denoted by the symbol "e," so 2 e {2, 3, 7}. Nonmembership is denoted by

"V." For example, 4 ý {2, 3, 7}. Larger sets, like all the numbers between 1 and 100

are denoted as {1, 2, ... , 1001. If all the elements of one set are contained in some

other set, then we say the first set is a subset of the second set and use the symbol

"_C" to denote this. So, for example, {2, 3, 7} f {1, 2,.. ., 100}. For a finite set of

numbers, the largest element in set is called the max and is denoted by max{...}.

Infinite sets, like the set of even numbers, can be represented as {0, 2,.. .} or in

closed form as {2nln E N}.

1.4. MATHEMATICAL PRELIMINARIES 7

for a word and a blank symbol, or some other punctuation symbol, like the period

at the end of a sentence. If we admit symbols that do not appear but cause other

effects, like newline and newpage, we can describe this entire book as a sequence

of symbols. Notice that standard terminology gives up notions of levels. Sequences

of bits form words, sequences of words form sentences, and sequences of sentences

form books. However, this book really is just a sequence of suitable chosen symbols

(about a half million of them). Some of these symbols we haven't talked about,

they format the mathematical symbols, and the like.

Rather than use a jumble of terminology, we^ will^ simply^ discuss^ strings^ which

are just finite sequences of symbols where each symbol is chosen from a fixed alpha-

bet. Sometimes we call strings words. An alphabet is just some finite set of symbols,

like the binary alphabet {0, 1} or the lowercase Latin alphabet {a, b, c, ... , z}. The

empty word, the unique string with no symbols, is denoted by e. The length of a

word w is the number of symbols in the word and is denoted by Iw1. Since the empty

word has no symbols, lel (^) = 0. If w is a word, wR denotes the reverse of w. So, if

w is 011, then wR is 110. A word w such that w = wR is called a palindrome. A

set of words is called a language. These terms are meant to conjure images of their

standard usage. We will use them to discuss the nature of computation, where the

input and output are given as strings of symbols. Even the display on your com-

puter screen is a sequence of symbols (a string) to a particular device (the monitor).

We often have need to manipulate sets of numbers and strings. Here we review

some common operations on sets. If S is a set, then S denotes the complement of

S. The complement of a set is always taken with respect to some larger, often only

implicitly specified, set. For example, if E is the set of even numbers, then E is the

set of all numbers that are not even, that is the odd numbers. If S is a set of words,

then S is the set of all words that can be formed using the same alphabet as was

used to construct S that are not in S. So for example, for the alphabet {a, b}, if S

is the set of words with only a's, {a, aa, aaa, .... } then S is the set of words with a's

and at least one b, maybe more.

Often, we need to count and sort things. If you are trying to place n items into

m slots and n > m, then one slot will have more than one item in it, no matter

what you do. This is called the pigeonhole principle.

Relations between objects are often crucial. Mathematically, a relation is rep-

resented as a subset of some Cartesian product. For example, "<" is a relation on

N x N. It contains all the pairs of natural numbers (x, y) such that x < y. The

"less than" relation is so common we have a special symbol for it. In less common

cases, when we have no special symbol, we give the relation explicitly as a subset

of a Cartesian product. Doing this formally with the "<" relation, we would define

R = I(x,y)Ix E N,y E N and x < y}

Then (2,3) E R and (3,2) ý R.

There are several common properties that a relation may or may not have. For

example, if R is a relation on S x S, then R is reflexive if (a, a) G R for each

8 CHAPTER 1. INTRODUCTION

a E S. For example, let S be the set {a, b, c}. Then any subset of S x S is a relation. Consider Rý = {(a, a), (a, b), (b, b)}. Then R is NOT a reflexive relation. However, it U {(c, c)} is reflexive. We^ say^ that^ R^ is^ symmetric^ if^ (a,^ b)^ E^ R whenever (b, a) c R. Using the same relation R, we see that it is not symmetric either, but RfU {(b, a)} is. A transitive relation (^) is one such that if (a, b) E R and (b, c) c R, then (a, c) E R. Notice that R is transitive, but R U {(b, c)} is not. This is because (a, b) E R U {(b, c)}, (b, c) G ht U {(b, c)}, but (a, c) V R U {(b, c)}. An equivalence relation is a reflexive, symmetric, and transitive relation. Equivalence relations give rise to equivalence classes where each such class contains all and only the members of the underlying set that relate to each other. For example, unless you are fortunate to be reading this book outside in a sunny place, you are using some sort of artificial light source to see the words on this page. The power of the artificial lights, whether they are incandescent, fluorescent, or halogen, is usually measured in watts. Define a relation of the set of light bulbs, of all types, sizes and shapes, by saying that two light bulbs relate to each other if they are of the same power (wattage). This relation is reflexive, since every bulb has the same power as itself. It is also symmetric, (^) since if bulb A has the same wattage as bulb B, then bulb B has the same wattage as bulb A. Finally, this relation is transitive, since if bulb A has the same wattage as bulb B and bulb B has the same wattage as bulb C, then bulbs A and C must have (^) the same wattage. So, equivalence classes arise, where, for example, there is one equivalence class for 40-watt bulbs. It contains all the 40-watt bulbs of all types, shapes, and sizes, but no bulbs of any other wattage. Suppose R is a relation over a set S such that for each a E S there is at most one b E S such that (a, b) E R. Thus we (^) say that R is a function. We will use the more familiar notation f : R _--_* S to indicate a function that takes inputs from the set R and returns elements from the set S. More formally, f C R x S and for each x E R there is at most one y G S such that f(x) = y, that is, (x, y) is in the relation. See Figure 1.1.

Domain Range

Figure 1.1 A (^) Function

A function from R to S can be considered as a mapping from R to S, or as a transformation of elements of R into elements of S. We call the set R the domain of the function f. When the relation that forms the function f is not explicitly formalized and we need to talk about its domain, we will write Dom(f). A function f is one-to-one, also called a bijection, if for each y there is exactly one x E Dom(f) such that f(x) = y. Bijective functions have an inverse. Suppose that f is a bijective function. Then the inverse function, f- 1 , is such that f- 1 (y) = x if and