Writing Network Servers in Python: Understanding Socket Programming and MIME for Email, Notas de estudo de Engenharia Elétrica

Baixe Writing Network Servers in Python: Understanding Socket Programming and MIME for Email e outras Notas de estudo em PDF para Engenharia Elétrica, somente na Docsity!

Praise for Foundations of Python Network Programming:

"This may be the easiest book review I've ever written. If you program

in Python and you want to write Internet applications, go buy

Foundations of Python Network Programming by John Goerzen."

-Ron Dippold, Slashdot contributor (www.slashdot.org)

"Foundations of Python Networking Programming is highly recom-

mended if you already have a familiarity with networking concepts

and basic Python programming. This is a second level book that

should be used to extend your knowledge and programming skills

after first learning the basics of Python with something like Dive

Into Python."

-Harold McFarland, Midwest Book Review (www.midwestbookreview.com)

"This book makes an excellent addition to your Python bookshelf"

-Jack Herrington, Code Generation (www.codegeneration.net)

"I am happy to have this book on my shelf"

• Tommi Virtanen, Tv's cobweb (http:/ /tv.debian.net)

Foundations of Python Network Programming

Copyright © 2004 by John Goerzen Originally published by Apress in 2004 All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN 978-1-59059-371-4 ISBN 978-1-4302-0752-8 (eBook) DOI 10.1007/978-1-4302-0752- Trademarked names may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, we use the names ooly in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. Lead Editor: Iason Gilmore Technical Reviewer: Magnus tie Hetland Editorial Board: Steve Anglin, Dan Appleman, Ewan Buckingham, Gary Cornell, Tony Davis, Iohn Franklin, Iason Gilmore, Chris Mills, Steve Rycroft, Dominic Shakeshaft, Iim Sumser, Karen Watterson, Gavin Wray, Iohn Zukowski Project Manager: Beth Christmas Copy Edit Manager: Nicole LeClerc Copy Editor: Mark Nigara Production Manager: Karl Brooks Production Editor: Ellie Fountain Compositor: Susan Glinert Stevens Proofreader: Elizabeth Berry Indexer: Kevin Broccoli Cover Designer: Kurt Krames Manufacturing Manager: Tom Debolski Distributed to the book trade in the United States by Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY 10010 and outside the United States by Springer-Verlag GmbH & Co. KG, Tiergartenstr. 17,69112 Heidelberg, Germany. In the United States: phone 1-800-SPRINGER, e-mail orders@springer-ny _corn, or visit http:/ /'tMt/. springer-ny. corn. Outside the United States: fax +496221 345229, e-mail [email protected], or visit http://'tMt/ • springer. de.

The information in this book is distributed on an "as is" basis, without warranty. Although every precaution has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in this work.

Tomywife Terah: Thank you for tolerating the tall grass,

late evenings,andshort weekends while I worked on this book.

Thank you^ for supportingmeevery stepofthe way with

encouragement, love,andchocolate cake.

Contents at a Glance

vi

Contents

About the Author ..........................................................................................^ xv

About the Technical Reviewer ........................................................ xvii

Acknowledgments ........................................................................................... xix

Introduction ..................................................................................................^ xxi

Part One Low-Level Networking .............................................^!

Chapter 1 Introduction to

Client/Server Networking ........................................ 3

Understanding TCP Basics ................................................................................ 3 Using the Client/Server Model .....................................................................^^6 Understanding User Datagram Protocol ......................................................^7 Understanding Physical Transports and Ethernet ................................ 9 Networking inPython ......................................................................................... 9 Summary ................................................................................................................... 17

Chapter 2 Network Clients .............................................................^19

Understanding Sockets^ .....................................................................................^^19 Creating Sockets ................................................................................................^20 Communicating with Sockets ..........................................................................^^23 Handling Errors ..................................................................................................^^23 Using User Datagram Protocol ......................................................................^^31 Summary ...................................................................................................................^^34

vii

• Chapter Part One
• Chapter
• Chapter
• Chapter
• Chapter
• Chapter Part Two
• Chapter
• Chapter
• Chapter Part Three
• Chapter
• Chapter
• Chapter
• Chapter Part Four
• Chapter
• Chapter - Low- Level Networking - Introduction to Client/Server Networking - Network Clients - Network Servers - Domain Name System - Advanced Network Operations - Web Services - Web Client Access - Parsing HTML and XHTML - XML and XML-RPC - E-mail Services - and Decoding E-Mail Composition - Simple Message Transport Protocol - POP - IMAP - General-Purpose Client Protocols - FTP - Database Clients - SSL
• Chapter Part Five
• Chapter
• Chapter
• Chapter
• Chapter Part Six
• Chapter - Server-Side Frameworks
  • SocketServer
  • SimpleXMLRPCServer
  • CGI
  • mod_python
  • Multitasking
  • Forking
  • Threading
• Chapter 22 Asynchronous Communication
• Index - Chapter 3 Network Servers - Preparing for Connections - Accepting Connections - Handling Errors - Using User Datagram Protocol - Using inetd or xinetd - Logging with syslog - Avoiding Deadlock - Summary - Chapter 4 Domain Name System
  • Making DNS Queries
  • Using Operating System Lookup Services
  • Using PyDNS for Advanced Lookups
    • Summary
    • Chapter 5 Advanced Network Operations
  • Half-Open Sockets
    • Timeouts
    • Transmitting Strings
  • Understanding Network Byte Order
  • Using Broadcast Data
  • Working with IPv6
  • Binding to Speci fie Addresses
  • Using Event Notification with poll() or select()
  • Summary
  • Part Two Web Services
  • Chapter 6 Web Client Access
  • Fetching Web Pages
• Authenticating
  • Submitting Form Data
• Handling Errors
• Using Non-HTTP Protocols
• Summary - Chapter 7 Parsing HTML and XHTML Contents - Understanding Basic HTML Parsing - Handling Real-World HTML - A Working Example - Summary - Chapter 8 XML and XML-RPC - Understanding XML Documents - Using DOM - Using XML-RPC - Summary - Part Three E-mail Services l - Chapter 9 E-Mail Composition and Decoding - Understanding Traditional Messages - Composing Traditional Messages - Parsing Traditional Messages
  • Understanding MIME
  • Composing MIME Attachments
  • Composing MIME Alternatives
  • Composing Non-English Headers
  • Composing Nested Multiparts
  • Parsing MIME Messages
  • Summary
  • Chapter 10 Simple Message Transport Protocol
  • Introducing the SMTP Library
  • Error Handling and Conversation Debugging
  • Getting Information from EHLO
  • Using Secure Sockets Layer and Transport Layer Security
• Authenticating
  • SMTP Tips
  • Summary
  • Chapter 11 POP Contents
  • Connecting and Authenticating
  • Obtaining Mailbox Information
  • Downloading Messages
  • Deleting Messages
  • Summary
  • Chapter 12 IMAP
  • Understanding IMAP in Python
  • Introducing IMAP in Twisted
  • Understanding Twisted Basics
  • Scanning the Folder List
  • Examining Folders
  • Basic Downloading
  • Flagging and Deleting Messages
  • Retrieving Message Parts
  • Finding Messages
• Adding Messages
  • Creating and Deleting Folders
• Moving Messages Between Folders
  • Summary
  • Part Four General-Purpose Client Protocols
• Chapter 13 FTP
• Understanding FTP
• Using FTP in Python
• Downloading ASCII Files
• Downloading Binary Files
• Uploading Data
• Handling Errors
• Scanning Directories
• Downloading Recursively
• Manipulating Server Files and Directories
• Summary - Chapter 14 Database Clients Contents - SQL and Networking - SQL in Python - Connecting - Executing Commands - Transactions - Repeating Commands - Retrieving Data - Reading Metadata - Using Data Types - Summary - Chapter 15 SSL
  • Understanding Network Vulnerabilities
    • Reducing Vulnerabilities with SSL
  • Understanding SSL in Python
  • Using Built-in SSL
  • Using OpenSSL
    • Verifying Server Certificates with OpenSSL
    • Summary
    • Part Five Server-Side Frameworks
  • Chapter 16 SocketServer
  • Using BaseHTTPServer
  • SimpleHTTPServer
  • CGIHTTPServer
  • Implementing New Protocols
  • IPv6
  • Summary
  • Chapter 17 SimpleXMLRPCServer
  • SimpleXMLRPCServer Basics
  • Serving Functions
  • Exploiting Class Features
• Using DocXMLRPCServer
• Using CGIXMLRPCRequestHandler
  • Supporting Multicall Functions
  • Summary
    • Chapter 18 CGI Contents
    • Setting Up CGI
    • Understanding CGI
    • Understanding CGI in Python
    • Retrieving Environment Information
    • Getting Input
    • Escaping Special Characters
    • Handling Multiple Inputs per Field
    • Uploading Files
    • Using Cookies
    • Summary
    • Chapter 19 mod_python
    • Understanding the Need for mod_python
    • Installing and Configuring mod_python
  • Understanding mod_python Basics
    • Dispatching Requests
    • Handling Input
    • Escaping
  • Understanding Interpreter Instances
    • Prebuilt Handlers in mod_python
    • Summary
    • Part Six Multi tasking
  • Chapter 20 Forking
  • Understanding Processes
  • Understanding fork()
  • Forking First Steps
  • Forking Servers
    • Locking
  • Error Handling
  • Summary
  • Chapter 21 Threading
  • Threading in Python
• Writing Threaded Servers
• Writing Threaded Clients
  • Summary

Contents

Chapter 22 Asynchronous Communication .............................. 469

Deciding Whether or Not to Use

Asynchronous Communication ................................................................. 470

Using Asynchronous Communication ........................................................... 471

Advanced Server-Side Use ............................................................................ 476

Monitoring Multiple Master Sockets ...................................................... 480

Using Twisted for Servers .......................................................................... 485

Summary ................................................................................................................. 489

Index ................................................................................................................... 491

xiii

About the

Technical

Magnus Lie Hedand is an associate professor of algorithms at the Norwegian University of Science and Technology (NTNU). He has been using Python since 1997 and is the author of the popular online tutorials "Instant Hacking" and "Instant Python." His publications include Practical Python (Apress, 2004) as well as several scientific papers.

Reviewer

xvii

Acknowledgments

THIS BOOK HAS BENEFITED tremendously from the insight, experience, dedication, and encouragement of many people. Without them, this project would not have happened. I'd like to especially thank the following people:

• Magnus Lie Hetland, technical reviewer. Magnus has a level of Python knowledge that I hope to acquire someday. He has a keen eye for detail, and a great knack for explaining things in a clear way. He found more bugs than I ever thought possible, and I thank him for it.
• Ellie Fountain, production editor. Ellie was extremely patient with my struggles to find a machine capable of running the software I needed to review your files.
• Beth Christmas, project manager. Just when I was losing track of which project to work on next, an e-mail from Beth would pop up, conveniently summarizing exactly what the next steps should be. She did a phenomenal job managing the schedule for this project. I have no idea how she made things always work out, even when I was slower than expected.
• Jason Gilmore, lead editor. Jason's comments on each chapter made the text far stronger than it was originally. By the time the book was half done, I'd read some text and think, "Jason would tell me to do that differently." His words of encouragement helped get me through the frantic days when things were happening at the last minute.
• Mark Nigara, copy editor. Thanks to Mark's reviews, the spelling is correct, the text flows well, and the book is easy to read. Reading over Mark's edits, I was often amazed at how much improvement he could bring to the text.

Many people contributed to the technologies used in this book. I'd like to highlight a few of them:

• To Guido van Rossum: Thank you for providing us with a language so versatile, yet so readable.
• To Richard M. Stallman: Thank you for showing us all the value of collabo- ration and sharing of technology.

xix

Introduction

TwENTY-FIVE YEARS AGO, the world was a different place. Talking to someone across the ocean was, formostpeople,anexceptionally rare experience. Sending a letter could take weeks. Listening to a shortwave newscast from a foreignlandrequired technical skill, patience,andprecise atmospheric conditions. Today, we think nothing of receivingane-mail from South Korea, checking the weatherinCalifornia,andreadingtheday's headlines inGermany-allinlessthan five minutes. Files zip across the global electronic network, allowing us to do every- thing from managing investment accounts to seeing pictures of distant relatives. And yet, despite allthathashappened in the last 25 years, (^) the Internet is stillin its infancy. It's anewtechnology, still growing. I wrote this book because the Internet is exciting. In the past few years, we've seen the rise of anentire industry that did not exist before. It's a place where inventors thrive. Andthatis what I hope you get from this book. I want this to be your lab manual-yourguide for inventing things that make the Internet better.

Organization

This book is divided into five parts. The firstpartexplains how the Internet works. You'll learn about the fundamentals oflnternet communication,and the examples provide you with the basic tools that you'll use to assemble your programs. Part Two covers web-based services and Part Three covers e-mail services. Many new technologies are a form of web or e-mail communication, and these parts explain how to write programs that take advantage of them. Part Four covers other technologies, such as databases and file transfer, that are often used behind the scenes of network programs. Your users may never know you use them, but they're still key components of your toolbox. Part Five shows you how to write servers, the applications that answer requests and give out information. You may never need to write a server, but if you're designing a new protocol from scratch, you'll need the techniques here. Finally, Part Six shows you how to do more than one thing at a time. Some network programmers never need to use these techniques. Others, such as server designers, couldn't survive without them.

xxi

Introduction

xxii

Assumptions

If you're reading this book, I assume that you already know Python. If not, I recommend the Apress book Practical Python by Magnus Lie Hetland. I also assume that you have Python 2.3 or above installed on your system with networking support compiled in. If you don't already have Python installed, check first with your operating system's packaging or installation system; a package is available for many systems. Otherwise, source code and installers can be down- loaded directly from www. python. org. Finally, I assume that you have a network connection configured on your system (even if you didn't personally configure it). You don't need to have any knowledge of Internet protocols to read this book.

Examples

There are many examples in these pages. You can download all of them from the Apress website at www. apress. com. I encourage you to try the examples included here, and to experiment with them. Many are suitable as a starting point for your own programs, and almost every one is a fully contained, ready-to-run program. You can see for yourself how things work. I'll often show you an example of an interaction with a program. When you see such an example, text in bold represents keyboard input. When you see commands, the $ character represents your operating system's command prompt. Networking relies heavily on the network support in your operating system. The operating system provides all of the basic infrastructure necessary for communicating with other computers. Here are some notes related to specific operating systems.

TIP If you have a problem with any example in this book, check these notes or the text in the chapter for an explanation. Some examples cannot run on all operating systems.

Linux, FreeBSD, Mac OS X, Solaris, and Other UNIX Platforms

Each example program assumes that you have the Python interpreter, named python, installed on your path. They also assume you have the /usr/bin/env program available. The first line of each executable Python script begins with#!,