Baixe Magnus Lie Hetland Python Algorithms Mastering Basic Algorithms in the Python Language e outras Notas de estudo em PDF para Engenharia Elétrica, somente na Docsity!
THE EXPERT’S VOICE®^ IN OPEN SOURCE
Python
Algorithms
Mastering Basic Algorithms in the
Python Language
Magnus Lie Hetland
Learn to implement classic algorithms and design new problem-solving algorithms using Python
Python Algorithms: Mastering Basic Algorithms in the Python Language Copyright © 2010 by Magnus Lie Hetland All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN-13 (pbk): 978-1-4302-3237- ISBN-13 (electronic): 978-1-4302-3238- Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1 Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. President and Publisher: Paul Manning Lead Editor: Frank Pohlmann Development Editor: Douglas Pundick Technical Reviewer: Alex Martelli Editorial Board: Steve Anglin, Mark Beckner, Ewan Buckingham, Gary Cornell, Jonathan Gennick, Jonathan Hassell, Michelle Lowman, Matthew Moodie, Duncan Parkes, Jeffrey Pepper, Frank Pohlmann, Douglas Pundick, Ben Renow-Clarke, Dominic Shakeshaft, Matt Wade, Tom Welsh Coordinating Editor: Adam Heath Compositor: Mary Sudul Indexer: Brenda Miller Artist: April Milne Cover Designer: Anna Ishchenko Photo Credit: Kai T. Dragland Distributed to the book trade worldwide by Springer Science+Business Media, LLC., 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders- [email protected], or visit www.springeronline.com. For information on translations, please e-mail [email protected], or visit www.apress.com. Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Special Bulk Sales–eBook Licensing web page at www.apress.com/info/bulksales. The information in this book is distributed on an “as is” basis, without warranty. Although every precaution has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in this work. The source code for this book is available to readers at www.apress.com
For my students. May your quest for knowledge be richly rewarded.
■ CONTENTS
v
Contents at a Glance
Contents...................................................................................................................vi About the Author ...................................................................................................xiii About the Technical Reviewer ............................................................................... xiv Acknowledgments .................................................................................................. xv Preface .................................................................................................................. xvi ■ Chapter 1: Introduction........................................................................................ ■ Chapter 2: The Basics .......................................................................................... ■ Chapter 3: Counting 101 .................................................................................... ■ Chapter 4: Induction and Recursion … and Reduction...................................... ■ Chapter 5: Traversal: The Skeleton Key of Algorithmics ................................. ■ Chapter 6: Divide, Combine, and Conquer........................................................ ■ Chapter 7: Greed Is Good? Prove It!................................................................. ■ Chapter 8: Tangled Dependencies and Memoization ....................................... ■ Chapter 9: From A to B with Edsger and Friends............................................. ■ Chapter 10: Matchings, Cuts, and Flows ......................................................... ■ Chapter 11: Hard Problems and (Limited) Sloppiness ..................................... ■ Appendix A: Pedal to the Metal: Accelerating Python ..................................... ■ Appendix B: List of Problems and Algorithms ................................................. ■ Appendix C: Graph Terminology....................................................................... ■ Appendix D: Hints for Exercises....................................................................... ■ Index ................................................................................................................
■ CONTENTS
vi
Contents
Contents at a Glance.................................................................................................v About the Author ...................................................................................................xiii About the Technical Reviewer ............................................................................... xiv Acknowledgments .................................................................................................. xv Preface .................................................................................................................. xvi ■ Chapter 1: Introduction........................................................................................ What’s All This, Then? ..................................................................................................... Why Are You Here? .......................................................................................................... Some Prerequisites ......................................................................................................... What’s in This Book ......................................................................................................... Summary ......................................................................................................................... If You’re Curious … ......................................................................................................... Exercises ......................................................................................................................... References....................................................................................................................... ■ Chapter 2: The Basics .......................................................................................... Some Core Ideas in Computing ....................................................................................... Asymptotic Notation ...................................................................................................... It’s Greek to Me! ................................................................................................................................... Rules of the Road ................................................................................................................................. Taking the Asymptotics for a Spin........................................................................................................ Three Important Cases ......................................................................................................................... Empirical Evaluation of Algorithms.......................................................................................................
■ CONTENTS
■ CONTENTS
■ CONTENTS
■ CONTENTS
- Implementing Graphs and Trees.................................................................................... vii
- Adjacency Lists and the Like ................................................................................................................
- Adjacency Matrices ..............................................................................................................................
- Implementing Trees..............................................................................................................................
- A Multitude of Representations ............................................................................................................
- Beware of Black Boxes..................................................................................................
- Hidden Squares ....................................................................................................................................
- The Trouble with Floats ........................................................................................................................
- Summary .......................................................................................................................
- If You’re Curious … .......................................................................................................
- Exercises .......................................................................................................................
- References.....................................................................................................................
- ■ Chapter 3: Counting 101 ....................................................................................
- The Skinny on Sums ......................................................................................................
- More Greek ...........................................................................................................................................
- Working with Sums ..............................................................................................................................
- A Tale of Two Tournaments ...........................................................................................
- Shaking Hands......................................................................................................................................
- The Hare and the Tortoise ....................................................................................................................
- Subsets, Permutations, and Combinations....................................................................
- Recursion and Recurrences...........................................................................................
- Doing It by Hand ...................................................................................................................................
- A Few Important Examples...................................................................................................................
- Guessing and Checking ........................................................................................................................
- The Master Theorem: A Cookie-Cutter Solution ...................................................................................
- So What Was All That About? ........................................................................................
- Summary .......................................................................................................................
- If You’re Curious … .......................................................................................................
- Exercises .......................................................................................................................
- References..................................................................................................................... viii
- ■ Chapter 4: Induction and Recursion … and Reduction......................................
- Oh, That’s Easy!.............................................................................................................
- One, Two, Many .............................................................................................................
- Mirror, Mirror .................................................................................................................
- Designing with Induction (and Recursion) .....................................................................
- Finding a Maximum Permutation .........................................................................................................
- The Celebrity Problem ..........................................................................................................................
- Topological Sorting...............................................................................................................................
- Stronger Assumptions ...................................................................................................
- Invariants and Correctness............................................................................................
- Relaxation and Gradual Improvement............................................................................
- Reduction + Contraposition = Hardness Proof ..............................................................
- Problem Solving Advice .................................................................................................
- Summary .......................................................................................................................
- If You’re Curious … .......................................................................................................
- Exercises .......................................................................................................................
- References.....................................................................................................................
- ■ Chapter 5: Traversal: The Skeleton Key of Algorithmics .................................
- A Walk in the Park .......................................................................................................
- No Cycles Allowed ..............................................................................................................................
- How to Stop Walking in Circles ..........................................................................................................
- Go Deep! ......................................................................................................................
- Depth-First Timestamps and Topological Sorting (Again) ..................................................................
- Infinite Mazes and Shortest (Unweighted) Paths.........................................................
- Strongly Connected Components ................................................................................
- Summary .....................................................................................................................
- If You’re Curious … ..................................................................................................... ix
- Exercises .....................................................................................................................
- References...................................................................................................................
- ■ Chapter 6: Divide, Combine, and Conquer........................................................
- Tree-Shaped Problems: All About the Balance............................................................
- The Canonical D&C Algorithm......................................................................................
- Searching by Halves ....................................................................................................
- Traversing Search Trees … with Pruning ..........................................................................................
- Selection.............................................................................................................................................
- Sorting by Halves.........................................................................................................
- How Fast Can We Sort? ......................................................................................................................
- Three More Examples ..................................................................................................
- Closest Pair.........................................................................................................................................
- Convex Hull.........................................................................................................................................
- Greatest Slice .....................................................................................................................................
- Tree Balance … and Balancing...................................................................................
- Summary .....................................................................................................................
- If You’re Curious … .....................................................................................................
- Exercises .....................................................................................................................
- References...................................................................................................................
- ■ Chapter 7: Greed Is Good? Prove It!.................................................................
- Staying Safe, Step by Step ..........................................................................................
- The Knapsack Problem ................................................................................................
- Fractional Knapsack ...........................................................................................................................
- Integer Knapsack................................................................................................................................
- Huffman’s Algorithm....................................................................................................
- The Algorithm .....................................................................................................................................
- The First Greedy Choice......................................................................................................................
- Going the Rest of the Way x
- Optimal Merging
- Minimum spanning trees.
- The Shortest Edge
- What About the Rest?
- Kruskal’s Algorithm
- Prim’s Algorithm...................................................................................................................................
- Greed Works. But When?.
- Keeping Up with the Best
- No Worse Than Perfect.........................................................................................................................
- Staying Safe
- Summary
- If You’re Curious …
- Exercises
- References...................................................................................................................
- ■ Chapter 8: Tangled Dependencies and Memoization .......................................
- Don’t Repeat Yourself
- Shortest Paths in Directed Acyclic Graphs
- Longest Increasing Subsequence.
- Sequence Comparison.
- The Knapsack Strikes Back
- Binary Sequence Partitioning
- Summary
- If You’re Curious …
- Exercises
- References...................................................................................................................
- ■ Chapter 9: From A to B with Edsger and Friends............................................. xi
- Propagating Knowledge...............................................................................................
- Relaxing like Crazy ......................................................................................................
- Finding the Hidden DAG...............................................................................................
- All Against All...............................................................................................................
- Far-Fetched Subproblems ...........................................................................................
- Meeting in the Middle..................................................................................................
- Knowing Where You’re Going ......................................................................................
- Summary .....................................................................................................................
- If You’re Curious … .....................................................................................................
- Exercises .....................................................................................................................
- References...................................................................................................................
- ■ Chapter 10: Matchings, Cuts, and Flows .........................................................
- Bipartite Matching .......................................................................................................
- Disjoint Paths...............................................................................................................
- Maximum Flow ............................................................................................................
- Minimum Cut ...............................................................................................................
- Cheapest Flow and the Assignment Problem ..............................................................
- Some Applications .......................................................................................................
- Summary .....................................................................................................................
- If You’re Curious … .....................................................................................................
- Exercises .....................................................................................................................
- References...................................................................................................................
- ■ Chapter 11: Hard Problems and (Limited) Sloppiness .....................................
- Reduction Redux..........................................................................................................
- Not in Kansas Anymore?..............................................................................................
- Meanwhile, Back in Kansas …....................................................................................
- But Where Do You Start? And Where Do You Go from There?..................................... xii
- A Ménagerie of Monsters.............................................................................................
- Return of the Knapsack ......................................................................................................................
- Cliques and Colorings.........................................................................................................................
- Paths and Circuits ..............................................................................................................................
- When the Going Gets Tough, the Smart Get Sloppy.....................................................
- Desperately Seeking Solutions ....................................................................................
- And the Moral of the Story Is …..................................................................................
- Summary .....................................................................................................................
- If You’re Curious … .....................................................................................................
- Exercises .....................................................................................................................
- References...................................................................................................................
- ■ Appendix A: Pedal to the Metal: Accelerating Python .....................................
- ■ Appendix B: List of Problems and Algorithms .................................................
- ■ Appendix C: Graph Terminology.......................................................................
- ■ Appendix D: Hints for Exercises.......................................................................
- Index.....................................................................................................................
■ CONTENTS
xiv
About the Technical Reviewer
■ Alex Martelli was born and grew up in Italy and holds a Laurea in Ingeneria Elettronica from the Universitá di Bologna. He wrote Python in a Nutshell and coedited the Python Cookbook. He’s a member of the PSF and won the 2002 Activators’ Choice Award and the 2006 Frank Willison Award for contributions tothe Python community. He currently lives in California and works as senior staff engineer for Google. His detailed profile is at www.google.com/profiles/aleaxit; a summary bio is at http://en.wikipedia.org/wiki/Alex_Martelli.
■ INTRODUCTION
xv
Acknowledgments
Thanks to everyone who contributed to this book, either directly or indirectly. This certainly includes my algorithm mentors, Arne Halaas and Bjørn Olstad, as well as the entire crew at Apress and my brilliant tech editor, Alex. Thanks to Nils Grimsmo, Jon Marius Venstad, Ole Edsberg, Rolv Seehuus, and JorgRødsjø for useful input; to my parents, Kjersti Lie and Tor M. Hetland, and my sister, Anne Lie-Hetland, for their interest and support; and to my uncle Axel, for checking my French. Finally, a big thank-you to the Python Software Foundation for their permission to reproduce parts of the Python standard library and to Randall Munroe for letting me include some of his wonderful XKCD comics.
C H A P T E R 1
1
Introduction
_1. Write down the problem.
- Think real hard.
- Write down the solution._ “The Feynman Algorithm” as described by Murray Gell-Mann
Consider the following problem. You are to visit all the cities, towns, and villages of, say, Sweden and then return to your starting point. This might take a while (there are 24 978 locations to visit, after all), soyou want to minimize your route. You plan on visiting each location exactly once, following the shortest route possible. As a programmer, you certainly don’t want to plot the route by hand. Rather, you try to write some code that will plan your trip for you. For some reason, however, you can’t seem to get it right. A straightforward program works well for a smaller number of towns and cities but seems to run foreveron the actual problem, and improving the program turns out to be surprisingly hard. How come? Actually, in 2004, a team of five researchers 1 found such a tour of Sweden, after a number of other research teams had tried and failed. The five-man team used cutting-edge software with lots of clever optimizations and tricks of the trade, running on a cluster of 96 Xeon 2.6 GHz workstations. Theirsoftware ran from March 2003 until May 2004, before it finally printed out the optimal solution. Taking various interruptions into account, the team estimated that the total CPU time spent was about 85 years! Consider a similar problem: You want to get from Kashgar, in the westernmost regions of China, to Ningbo, on the east coast, following the shortest route possible. Now, China has 3 583 715 km ofroadways and 77 834 km of railways, with millions of intersections to consider and a virtually unfathomable number of possible routes to follow. It might seem that this problem is related to the previous one, yet this shortest path problem is one solved routinely, with no appreciable delay, by GPS software and online map services. If you give those two cities to your favorite map service, you shouldget the shortest route in mere moments. What’s going on here? You will learn more about both of these problems later in the book; the first one is called the traveling salesman (or salesrep ) problem and is covered in Chapter 11, while so-called shortest path problems are primarily dealt with in Chapter 9. I also hope you will gain a rather deep insight into whyone problem seems like such a hard nut to crack while the other admits several well-known, efficient solutions. More importantly, you will learn something about how to deal with algorithmic and computational problems in general, either solving them efficiently, using one of the several techniques and algorithms you encounter in this book, or showing that they are too hard and that approximatesolutions may be all you can hope for. This chapter briefly describes what the book is about—what you
(^1) David Applegate, Robert Bixby, Vašek Chvátal, William Cook, and Keld Helsgaun
CHAPTER 1 ■ INTRODUCTION
2
can expect and what is expected of you. It also outlines the specific contents of the various chapters to come in case you want to skip around.
What’s All This, Then? This is a book about algorithmic problem solving for Python programmers. Just like books on, say,object-oriented patterns, the problems it deals with are of a general nature—as are the solutions. Your task as an algorist will, in many cases, be more than simply to implement or execute an existing algorithm, as you would, for example, in solving an algebra problem. Instead, you are expected to come up with new algorithms—new general solutions to hitherto unseen, general problems. In this book, you are going to learn principles for constructing such solutions.This may not be your typical algorithm book, though. Most of the authoritative books on the subject (such as the Knuth’s classics or the industry-standard textbook by Cormen et al.) have a heavy formal and theoretical slant, even though some of them (such as the one by Kleinberg and Tardos) lean more inthe direction of readability. Instead of trying to replace any of these excellent books, I’d like to supplement them. Building on my experience from teaching algorithms, I try to explain as clearly as possible how the algorithms work and what common principles underlie many of them. For a programmer, these explanations are probably enough. Chances are you’ll be able to understand why thealgorithms are correct and how to adapt them to new problems you may come to face. If, however, you need the full depth of the more formalistic and encyclopedic textbooks, I hope the foundation you get in this book will help you understand the theorems and proofs you encounter there. kind, where theThere is another genre of algorithm books as well: the “(Data Structures and) Algorithms in blank is the author’s favorite programming language. There are quite a few of these^ blank ” (especially for blank = Java, it seems), but many of them focus on relatively basic data structures, to the detriment of the more meaty stuff. This is understandable if the book is designed to be used in a basic course on data structures, for example, but for a Python programmer, learning about singly and doublylinked lists may not be all that exciting (although you will hear a bit about those in the next chapter). And even though techniques such as hashing are highly important, you get hash tables for free in the form of Python dictionaries; there’s no need to implement them from scratch. Instead, I focus on more high- level algorithms. Many important concepts that are available as black-box implementations either in thePython language itself or in the standard library (such as sorting, searching, and hashing) are explained more briefly, in special “black box” sidebars throughout the text. There is, of course, another factor that separates this book from those in the “Algorithms in Java/C/C++/C#” genre, namely, that thelanguage-independent books (such as those by Knuth, blank is Python. This places the book one step closer to the 2 Cormen et al., and Kleinberg and Tardos, for example), which often use pseudocode , the kind of fake programming language that is designed to be readable rather than executable. One of Python’s distinguishing features is its readability; it is, more or less, executable pseudocode. Even if you’ve never programmed in Python, you could probably decipherthe meaning of most basic Python programs. The code in this book is designed to be readable exactly in this fashion—you need not be a Python expert to understand the examples (although you might need to look up some built-in functions and the like). And if you want to pretend the examples are actually pseudocode, feel free to do so. To sum up …
(^2) Knuth is also well-known for using assembly code for an abstract computer of his own design.
