¡Descarga App engine - python backend y más Monografías, Ensayos en PDF de Programación Informática solo en Docsity!
Dan Sanderson
Programming
Google
App Engine
with Python
BUILD & RUN SCALABLE PYTHON APPS
ON GOOGLE'S INFRASTRUCTURE
www.allitebooks.com
WEB PROGR AMMING
Programming Google App Engine with Python
ISBN: 978-1-491-90025-
US $44.99 CAN $51.
“The^ first^ edition^ of^ Dan's
book was a crucial
resource for years when
our team was learning
App Engine. This latest
edition is even better—
it's the most complete
guide available for those
building on App Engine,
and it'll be useful to
newcomers and old
hats alike.” —Ben Kamens lead dev at Khan Academy
“Without^ a^ doubt,^ the
most comprehensive,
well-written, and up-to-
date book written about
App Engine. I think it's
fair to say that if you've
read this book cover-
to-cover, you're now an
expert on App Engine.” —Kevin Gibbs former tech lead and manager, Google App Engine
Twitter: @oreillymedia facebook.com/oreilly
This practical guide shows intermediate and advanced web and mobile app developers how to build highly scalable Python applications in the cloud with Google App Engine. The flagship of Google's Cloud Platform, App Engine hosts your app on infrastructure that grows automatically with your traffic, minimizing up-front costs and accommodating unexpected visitors. You’ll learn hands-on how to perform common development tasks with App Engine services and development tools, including deployment and maintenance.
App Engine's Python support includes a fast Python 2.7 interpreter, the standard library, and a WSGI-based runtime environment. Choose from many popular web application frameworks, including Django and Flask.
■ Get a hands-on introduction to App Engine's tools and features, using an example application ■ Simulate App Engine on your development machine with tools from Google Cloud SDK ■ Structure your app into individually addressable modules, each with its own scaling configuration ■ Exploit the power of the scalable Cloud Datastore, using queries, transactions, and data modeling with the ndb library ■ Use Cloud SQL for standard relational databases with App Engine applications ■ Learn how to deploy, manage, and inspect your application on Google infrastructure
Dan Sanderson is a software engineer at Google Inc. He has worked in the web industry for over fifteen years as a software engineer and technical writer for Google, Amazon.com, and the Walt Disney Internet Group.
www.allitebooks.com
978-1-491-90025-
[LSI]
Programming Google App Engine with Python by Dan Sanderson
Copyright © 2015 Dan Sanderson. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://safaribooksonline.com). For more information, contact our corporate/ institutional sales department: 800-998-9938 or [email protected].
Editors: Meghan Blanchette and Brian Anderson Acquisition Editor: Mike Loukides Production Editors: Colleen Lobner and Kara Ebrahim Copyeditor: Jasmine Kwityn
Proofreader: Charles Roumeliotis Indexer: Judy McConville Interior Designer: David Futato Cover Designer: Ellie Volckhausen Illustrator: Rebecca Demarest
June 2015: First Edition
Revision History for the First Edition 2015-06-17: First Release
See http://oreilly.com/catalog/errata.csp?isbn=9781491900253 for release details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Programming Google App Engine with Python, the cover image of a four-lined snake, and related trade dress are trademarks of O’Reilly Media, Inc.
While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.
www.allitebooks.com
Table of Contents
iii
Preface
On the Internet, popularity is swift and fleeting. A mention of your website on a pop‐ ular news site can bring 300,000 potential customers your way at once, all expecting to find out who you are and what you have to offer. But if you’re a small company just starting out, your hardware and software aren’t likely to be able to handle that kind of traffic. You’ve sensibly built your site to handle the 30,000 visits per hour you’re actually expecting in your first six months. Under heavy load, such a system would be incapable of showing even your company logo to the 270,000 others that showed up to look around. And those potential customers are not likely to come back after the traffic has subsided.
The answer is not to spend time and money building a system to serve millions of visitors on the first day, when those same systems are only expected to serve mere thousands per day for the subsequent months. If you delay your launch to build big, you miss the opportunity to improve your product by using feedback from your cus‐ tomers. Building big early risks building something your customers don’t want.
Historically, small companies haven’t had access to large systems of servers on day one. The best they could do was to build small and hope that meltdowns wouldn’t damage their reputation as they try to grow. The lucky ones found their audience, got another round of funding, and halted feature development to rebuild their product for larger capacity. The unlucky ones, well, didn’t.
These days, there are other options. Large Internet companies such as Amazon.com, Google, and Microsoft are leasing parts of their high-capacity systems by using a pay- per-use model. Your website is served from those large systems, which are plenty capable of handling sudden surges in traffic and ongoing success. And because you pay only for what you use, there is no up-front investment that goes to waste when traffic is low. As your customer base grows, the costs grow proportionally.
Google’s offering, collectively known as Google Cloud Platform, consists of a suite of high-powered services and tools: virtual machines in a variety of sizes, multiple forms of reliable data storage, configurable networking, automatic scaling infrastructure,
xi
and even the big data analysis tools that power Google’s products. But Google Cloud Platform does more than provide access to Google’s infrastructure. It encapsulates best practices for application architecture that have been honed by Google engineers for their own products.
The centerpiece of Google Cloud Platform is Google App Engine, an application hosting service that grows automatically. App Engine runs your application so that each user who accesses it gets the same experience as every other user, whether there are dozens of simultaneous users or thousands. Your application code focuses on each individual user’s experience. App Engine takes care of large-scale computing tasks— such as load balancing, data replication, and fault tolerance—automatically.
The scalable model really kicks in at the point where a traditional system would out‐ grow its first database server. With such a system, adding load-balanced web servers and caching layers can get you pretty far, but when your application needs to write data to more than one place, you face a difficult problem. This problem is made more difficult when development up to that point has relied on features of database soft‐ ware that were never intended for data distributed across multiple machines. By thinking about your data in terms of Cloud Platform’s model up front, you save your‐ self from having to rebuild the whole thing later.
Often overlooked as an advantage, App Engine’s execution model helps to distribute computation as well as data. App Engine excels at allocating computing resources to small tasks quickly. This was originally designed for handling web requests from users, where generating a response for the client is the top priority. Combining this execution model with Cloud Platform’s task queue service, medium-to-large compu‐ tational tasks can be broken into chunks that are executed in parallel. Tasks are retried until they succeed, making tasks resilient in the face of service failures. The execution model encourages designs optimized for the parallelization and robustness provided by the platform.
Running on Google’s infrastructure means you never have to set up a server, replace a failed hard drive, or troubleshoot a network card. You don’t have to be woken up in the middle of the night by a screaming pager because an ISP hiccup confused a ser‐ vice alarm. And with automatic scaling, you don’t have to scramble to set up new hardware as traffic increases.
Google Cloud Platform and App Engine let you focus on your application’s function‐ ality and user experience. You can launch early, enjoy the flood of attention, retain customers, and start improving your product with the help of your users. Your app grows with the size of your audience—up to Google-sized proportions—without hav‐ ing to rebuild for a new architecture. Meanwhile, your competitors are still putting out fires and configuring databases.
xii | Preface
either shut down machines when no longer needed or leave them running in tradi‐ tional or custom configurations.
App Engine and Compute Engine take different approaches to provide different capa‐ bilities. But these technologies are already starting to blend. In early 2014, Google announced Managed VMs, a new way to run VM-based code in an App Engine-like way. (This feature is not fully available as I write this, but check the Google Cloud Platform website for updates.) Overall, you’re able to adopt as much of the platform as you need to accomplish your goals, investing in flexibility when needed, and letting the platform’s automaticity handle the rest.
This book is being written at a turning point in App Engine’s history. Services that were originally built for App Engine are being generalized for Cloud Platform, and given REST APIs so you can call them from off the platform as well. App Engine development tools are being expanded, with a new universal Cloud SDK and Cloud Console. We’re even seeing the beginnings of new ways to develop and deploy soft‐ ware, with integrated Git-based source code revision control. As with any book about an evolving technology, what follows is a snapshot, with an emphasis on major con‐ cepts and long-lasting topics.
The focus of this book is building web applications using App Engine and related parts of the platform, especially Cloud Datastore. We’ll discuss services currently exclusive to App Engine, such as those for fetching URLs and sending email. We’ll also discuss techniques for organizing and optimizing your application, using task queues and offline processes, and otherwise getting the most out of Google App Engine.
Using This Book
Programming Google App Engine with Python covers App Engine’s runtime environ‐ ment for the Python programming language. The Python runtime environment pro‐ vides a fast interpreter for the Python language, and includes Python libraries for all of App Engine’s features. It is compatible with many major open source web applica‐ tion frameworks, such as Django and Flask.
App Engine supports three other runtime environments: Java, PHP, and Go. Java sup‐ port includes a complete Java servlet environment, with a JVM capable of running bytecode produced by compilers for Java and other languages. The PHP environment runs a native PHP interpreter with the standard library and many extensions enabled, and is capable of running many off-the-shelf PHP applications such as WordPress and Drupal. With the Go runtime environment, App Engine compiles your Go code on the server and executes it at native CPU speeds.
xiv | Preface
The information contained in this book was formerly presented in a single volume, Programming Google App Engine, which also covered Java. To make it easy to find the information you need for your language, that book has been split into language- specific versions. You are reading the Python version. Programming Google App Engine with Java covers the same material using the Java language, as well as Java- specific topics.
We are considering PHP and Go versions of this book as a future endeavor. For now, the official App Engine documentation is the best resource for using these languages on the platform. If you’re interested in seeing versions of this book for PHP or Go, let us know by sending email to [email protected].
The book is organized so you can jump to the subjects that are most relevant to you. The introductory chapters provide a lay of the land, and get you working with a com‐ plete example that uses several features. Subsequent chapters are arranged by App Engine’s various features, with a focus on efficient data storage and retrieval, commu‐ nication, and distributed computation. Project life cycle topics such as deployment and maintenance are also covered.
Cloud Datastore is a large enough subject that it gets multiple chapters to itself. Start‐ ing with Chapter 6, datastore concepts are introduced alongside Python APIs related to those concepts. Python examples use the ndb data modeling library, provided in the Cloud SDK. Data modeling gets its own chapter, in Chapter 9.
Here’s a quick look at the chapters in this book:
Chapter 1, Introducing Google App Engine A high-level overview of Google App Engine and its components, tools, and major features, as well as an introduction to Google Cloud Platform as a whole.
Chapter 2, Creating an Application An introductory tutorial in Python, including instructions on setting up a devel‐ opment environment, using template engines to build web pages, setting up accounts and domain names, and deploying the application to App Engine. The tutorial application demonstrates the use of several App Engine features—Google Accounts, the datastore, and memcache—to implement a pattern common to many web applications: storing and retrieving user preferences.
Chapter 3, Configuring an Application A description of how App Engine handles incoming requests, and how to config‐ ure this behavior. This introduces App Engine’s architecture, the various features of the frontend, app servers, and static file servers. We explain how the frontend routes requests to the app servers and the static file servers, and manages secure connections and Google Accounts authentication and authorization. This chapter also discusses quotas and limits, and how to raise them by setting a budget.
Preface | xv
chapter also discusses how to use the Remote API, a proxy for building adminis‐ trative tools that run on your local computer but access the live services for your app.
Chapter 11, Using Google Cloud SQL with App Engine Google Cloud SQL provides fully managed MySQL database instances. You can use Cloud SQL as a relational database for your App Engine applications. This chapter walks through an example of creating a SQL instance, setting up a data‐ base, preparing a local development environment, and connecting to Cloud SQL from App Engine. We also discuss prominent features of Cloud SQL such as backups, and exporting and importing data. Cloud SQL complements Cloud Datastore and Cloud Storage as a new choice for persistent storage, and is a pow‐ erful option when you need a relational database.
Chapter 12, The Memory Cache App Engine’s memory cache service (“memcache”), and its Python APIs. Aggres‐ sive caching is essential for high-performance web applications.
Chapter 13, Fetching URLs and Web Resources How to access other resources on the Internet via HTTP by using the URL Fetch service. Python applications can call this service using a direct API as well as via Python’s standard library.
Chapter 14, Sending and Receiving Email Messages How to use App Engine services to send email. This chapter covers receiving email relayed by App Engine by using request handlers. It also discusses creating and processing messages by using tools in the API.
Chapter 15, Sending and Receiving Instant Messages with XMPP How to use App Engine services to send instant messages to XMPP-compatible services (such as Google Talk), and receive XMPP messages via request handlers. This chapter discusses several major XMPP activities, including managing pres‐ ence.
Chapter 16, Task Queues and Scheduled Tasks How to perform work outside of user requests by using task queues. Task queues perform tasks in parallel by running your code on multiple application servers. You control the processing rate with configuration. Tasks can also be executed on a regular schedule with no user interaction.
Chapter 17, Optimizing Service Calls A summary of optimization techniques, plus detailed information on how to make asynchronous service calls, so your app can continue doing work while services process data in the background. This chapter also describes AppStats, an
Preface | xvii
important tool for visualizing your app’s service call behavior and finding perfor‐ mance bottlenecks.
Chapter 18, The Django Web Application Framework How to use the Django web application framework with the Python runtime environment. This chapter discusses setting up a project by using the Django 1. library included in the runtime environment, and using Django features such as component composition, URL mapping, views, and templating. It also discusses how to use a newer version of Django than what is built into the runtime envi‐ ronment. The chapter introduces WTForms, a web form framework with special features for integrating with App Engine’s ndb data modeling library.
Chapter 19, Managing Request Logs Everything you need to know about logging messages, browsing and searching log data in the Cloud Console, and managing and downloading log data. This chapter also introduces the Logs API, which lets you manage logs programmati‐ cally within the app itself.
Chapter 20, Deploying and Managing Applications How to upload and run your app on App Engine, how to update and test an application using app versions, and how to manage and inspect the running application. This chapter also introduces other maintenance features of the Cloud Console, including billing. The chapter concludes with a list of places to go for help and further reading.
Conventions Used in This Book
The following typographical conventions are used in this book:
Italic Indicates new terms, URLs, email addresses, filenames, and file extensions.
Constant width Used for program listings, as well as within paragraphs to refer to program ele‐ ments such as variable or function names, databases, data types, environment variables, statements, and keywords.
Constant width bold Shows commands or other text that should be typed literally by the user.
Constant width italic Shows text that should be replaced with user-supplied values or by values deter‐ mined by context.
xviii | Preface
