Mailthon Documentation: User's Guide and Developer Reference, Study notes of Philosophy

Download Mailthon Documentation: User's Guide and Developer Reference and more Study notes Philosophy in PDF only on Docsity!

Mailthon Documentation

Release 0.1.

Eeo Jun

August 22, 2016

ii

Welcome to Mailthon’s documentation. Mailthon is an email library for Python that aims to be highly extensible and composable. The docs are divided into different parts. I suggest that you start with Installation and then head to the Quickstart. If you want to dive into the internals, go for the API section.

Contents 1

CHAPTER 1

User’s Guide

This part of the documentation, mostly consisting of prose, begins with background information of Mailthon, and focuses on getting started with the library.

1.1 Foreword

Read this before getting started with Mailthon. It answers (hopefully) some questions about the goals of the project and whether you should consider using it or not.

1.1.1 Philosophy

Mailthon’s philosophy is that composability and extensibility stem from an elegant and simple codebase. Also, one of Mailthon’s aims is to not include as much magic as practically possible, but remain pragmatic at the same time. For example Mailthon does not have magical HTML/plaintext inference of the content- that is left to the developer to decide. Another value is the sanity of the API- whether something is named intuitively, or whether something should work like it does. The focus is on having as little surprises as possible.

1.1.2 Code over Configuration

While some minimum configuration can be done via “configuration”, most of the time Mailthon strives to encourage code-over-configuration. This is because code is always more powerful than configuration, which is one of the driving ideals behind middleware classes. Also, Mailthon does not rely on any configuration files, nor does it try to parse any files. Any configuration should be made explicit in the code.

Continue to the Installation or the Quickstart.

1.2 Installation

Mailthon does not have any external dependencies apart from the Python standard library. Mailthon is however, only tested against Python 2.6 or newer, so make sure you have an up-to-date Python installation. Mailthon also supports Python 3.

1.2.1 Installing a stable version

Usually you want to install the latest stable version. Pulling the package from PIP is what you want. I recommend using a virtualenv but that is not required to install Mailthon:

$ pip install Mailthon

Before you do that however you should check if you have PIP installed, which is a package manager for python. If you don’t, just download the get-pip.py script and run it.

1.2.2 Living on the edge

If you want to work on Mailthon or download the latest version, it is recommended that clone from the git repository, because you can always check out the latest version of the codebase and keep your local copy in sync with the latest goodies (Virtualenv is recommended):

$ git clone http://github.com/eugene-eeo/mailthon.git $ cd mailthon $ virtualenv venv New python executable in venv/bin/python Installing setuptools, pip..............done. $. venv/bin/activate $ python setup.py develop

This will pull in the latest codebase and activate the git head as the current version inside the virtualenv. All you have to do is run git pull origin to update to the latest version.

1.3 Quickstart

This page gives a good introduction to Mailthon, and assumes that you already have Mailthon installed. Head over to the Installation section if you do not.

1.3.1 Comparison

A comparison between using the standard library smtplib module and Mailthon:

from smtplib import SMTP from email.mime.multipart import MIMEMultipart from email.mime.text import MIMEText from email.mime.image import MIMEImage

text = MIMEText('

Hi!

', 'html') image = MIMEImage(open('cats.gif', 'rb').read())

mime = MIMEMultipart() mime['Sender'] = 'Me ' mime['To'] = 'Them ' mime.attach(text) mime.attach(image)

smtp = SMTP(host='smtp.server.com', port=587) smtp.ehlo() smtp.starttls() smtp.login('username', 'password')

4 Chapter 1. User’s Guide

receivers=['[email protected]'], subject='Hello World!', content='Hi!', )

However take note that the email() function makes an email with a HTML enclosure, not the plaintext one like what we’ve just created. An alternative to building the headers by hand is to use the mailthon.headers module:

import mailthon.headers as headers

Envelope( headers=[ headers.sender('sender '), headers.to('[email protected]'), headers.subject('Hello World!'), ], enclosure=[], )

1.3.3 Creating a Postman

Mailthon uses rather quirky real-life names to represent abstract email concepts. What better name for something that delivers an envelope than a postman? To create a postman that is configured for a GMail server:

from mailthon.postman import Postman from mailthon.middleware import TLS, Auth

postman = Postman( host='mail.google.com', port=587, middlewares=[ TLS(force=True), Auth(username='USERNAME', password='PASSWORD') ], )

(Substitute USERNAME and PASSWORD with your credentials, obtained from the Authorizing applications & sites page.) So what did we just do?

1. We created a Postman instance. A Postman handles the sending of emails via some transport, usually that defaults to SMTP. The Postman is created with the correct host and port arguments.
2. We configured the Auth and TLS middleware. They provide authentication and TLS support, respectively. We also forced TLS because we know that the GMail SMTP server only allows us to login if we have TLS enabled (which is also the reason why it is placed before the authentication middleware).

There is again, a simpler function for handling that in the form of the postman() function:

from mailthon import postman as postman_

postman = postman_( host='mail.google.com', port=587, force_tls=True, auth=('USERNAME', 'PASSWORD'), )

6 Chapter 1. User’s Guide

1.3.4 Sending an Envelope

After creating an envelope and a postman, we can then send the envelope to the receivers using the send() method:

response = postman.send(envelope)

Which returns the result of the sending the envelope- whether the server accepted it, whether everything went OK, etc. You can access the response values:

print (response.message) print (response.status_code)

if response.ok: print ("OK! :)")

You might want to continue reading about Mailthon’s architecture in the In Depth Guide, or dive into the internals in the API section.

1.4 In Depth Guide

This document describes the API in a stage-by-stage basis. It is useful as a book-like, gentle technical introduction to the higher and lower level APIs. Before reading this guide it is recommended that you browse through the Quickstart as it will give a very high level introduction to Maithon. If you want to look for some method or class, go to the API section.

1.4.1 Envelopes and MIMEs

The Envelope class actually wraps around multipart MIME objects. This means they can be composed of multiple other MIME objects, e.g. plaintext MIMEs, MIMEs containing HTML data, images, etc, which provides the perfect building block for a composable class. Envelopes are made of two parts- like a real life envelope, a “stamp”, or headers, and an enclosure, made of other mime objects:

from mailthon.envelope import Envelope from mailthon.enclosure import PlainText

e = Envelope( headers={'X-Key': 'Value'}, enclosure=[PlainText('something')], )

An interesting thing to take note of is that envelopes can be enclosed within envelopes. Concretely speaking, Envelopes consist of the Headers class and a list of Enclosure objects:

>>> e.headers {'X-Key': u'Value'} >>> e.enclosure []

You might have noticed that the Value string that was set was changed to a Unicode value. Why is that so? This is because internally the Headers class decodes bytes values that we throw at it into Unicode objects, freeing the developer from any headaches about encoding. You can read more about these design decisions at the Design Decisions section.

Now that we’ve looked at the higher level API of the Envelope class, let’s plunge deeper into madness and look into how it generates MIME objects with the mime() method:

1.4. In Depth Guide 7

Which prevents you from shooting yourself in the foot. Or other parts of your body. Also it makes sure that, most of the time, you get the benefit of having the Mailthon infrastructure supporting your back- the main example being free of having to worry about encoding.

1.4.3 Few Sips of SMTP

How in the world, you ask, do you have tricks to make the Enclosure class to behave like an envelope? The Oracle answers, via the runtime modification of attributes which may cause headaches in production; but hey, let’s try them anyways:

enclosure = PlainText('something') enclosure.mail_from = u'[email protected]' enclosure.receivers = [u'[email protected]', u'[email protected]']

def string(self): return self.mime().as_string()

enclosure.string = string

Note that the mail_from and receivers attributes having Unicode values is absolutely necessary, and we’ll see why when we talk about then later when we explore the Postman object. For now, assume that they will be properly encoded by Mailthon. When we pass the enclosure we’ve mutated to a Postman instance, it’ll happily send it off:

>>> r = postman.send(enclosure) >>> assert r.ok

Questioning our identity

Notice the mail_from attribute- it is not named something like sender. Why is that so? It is named such that it is synonymous with the SMTP MAIL FROM command. This is what is sent by a vanila (without any middleware) Postman instance in a typical SMTP session:

HELO relay.example.org MAIL FROM: RCPT TO: RCPT TO: DATA

QUIT

Note the highlighted line- the address passed to the MAIL FROM command is the ‘true’ sender. For example you begin your letter with something along the lines of “From XXX”. The postman doesn’t care about whatever you wrote in there. He may, however write down your name somewhere for bookeeping reasons. The address passed to the MAIL FROM command is, essentially, your ‘true’ name. More info about this can be obtained by reading RFC 2821.

Usually you are doing the sane thing- you are sending from the same email address that you are claiming to send from (i.e. the one you set in the headers argument to the Envelope class). But if you wish to do so, you can change the ‘real’ address. There are two ways to do it:

from mailthon.headers import sender

envelope = Envelope( headers=[sender('Fake ')], enclosure=[], mail_from=u'[email protected]',

1.4. In Depth Guide 9

) envelope.mail_from = u'[email protected]'

However if you want the inferred sender (the one that was obtained from the headers) you can still do so via the sender attribute. You can read more about the behaviour of the mail_from attribute.

The headless MIME

In an ideal world, the SMTP protocol speaks Unicode and we can all throw poop emojis around at each other while pretending to get our work done. But that is sadly not the case. SMTP is a protocol which only understands bytes, and was invented way back in 1982 when nobody cared about characters outside the English alphabet.

As a result, the simple ASCII encoding stuck and was used as the de-facto standard for emails and most other protocols. However, SMTP, given that it does only operate in bytes, does allow you to simply do:

Subject: (Hello)

But some clients will not be able to read it if they are expected something encoded in ASCII, and suddenly get some UTF-8 value, and is likely to end up with Mojibake.

Instead, we must specify the encoding, and then rewrite all of the code points of the string so that it is ASCII-encoded. So your beautiful characters end up looking like:

>>> from email.header import Header >>> Header(u' (Hello)', 'utf-8').encode() '=?utf-8?b?5ZOI5ZKvIChIZWxsbyk=?='

Not very nice, nor human readable. So rather than having you manually encode everything, Mailthon insists on having everything in Unicode. This makes everything a lot easier- extracting and encoding addresses, equality comparisions, etc. So the job of the Headers class (specifically, the UnicodeDict class) is to handle all this for you:

>>> from email.message import Message >>> from mailthon.headers import Headers >>> headers = Headers({ ... 'Subject': u'y is not exact', ... }) >>> mime = Message() >>> headers.prepare(mime) >>> mime.as_string() 'Subject: =?utf-8?q?=E2=88=82y_is_not_exact?=\n\n'

For the record, it’s actually the Message class that does all the heavy lifting- for space saving and efficiency reasons, Mailthon simply supplies it with the Unicode string and it determines whether to encode with ASCII or UTF-8.

IDNA and Friends

Turns out that there is now a format for encoding domain names with non-ASCII characters in them, specified in RFC 3490 and usually referred to as IDN or IDNA. For a real life example: é.com. This gives us a pleasant surprise if we try to encode everything with UTF-8, the silver bullet to our Unicode encoding woes:

>>> u'é'.encode('utf8') '\xc3\xa9' >>> u'é'.encode('idna') 'xn--9ca'

A short detour on the format of email addresses- they are made up of two parts, separated by the first occurence of the ‘@’ symbol.

10 Chapter 1. User’s Guide

def ehlo(self): self.connection_started = True

def sendmail(self, sender, receipients, string): self.check_conn() return {}

def quit(self): self.connection_started = False

Next all we need to do is replace the tranport attribute with the class object that we’ve just created. Although this is not recommended as I recommend subclassing to change the transport being used we will do it anyways:

postman.transport = MyTransport

The response_cls attribute will contain a custom response class. We will create our own response class as well:

class Response (object): def init(self, rejected, status): self.rejected = rejected self.status_code = status

@property def ok(self): return self.status_code == 200 and not self.rejected

If you haven’t noticed, the init method of our custom response class matches perfectly with the return values of the sendmail and noop methods from the MyTransport class, respectively. They are called by the Postman class like so:

def deliver(self, conn, envelope): rejected = conn.sendmail(...) return self.response_cls(rejected, conn.noop())

Now we just have to change the response class on the postman object we’ve created. Once again I recommend subclassing to change these attributes but for this experiment we’ll change them in runtime:

>>> postman.response_cls = Response

Putting it all together

Next we’ll send an envelope “across the wire” using our mutated postman object with our custom transport and response classes:

>>> r = postman.send(envelope) >>> assert r.ok

But that doesn’t give us very much knowlegde of what happens underneath the hood. The send() method is simply a veneer over the lower level connection() and deliver() methods. Let’s recreate the send method:

>>> with postman.connection() as conn: ... print (conn.connection_started) ... r = postman.deliver(conn, envelope) ... print (r) ... True <main.Response object at 0x...>

12 Chapter 1. User’s Guide

Basically what the connection() context manager does is that it manages the (usually SMTP) session for you. It is roughly implemented as:

@contextmanager def connection(self): conn = self.transport(self.host, self.port) try : conn.ehlo() yield conn finally : conn.quit()

Which closes the connection regardless of whether the sending operation is a success. This is important to prevent excessive memory and file-descriptor usage from the open sockets. You can verify that the connection as closed:

>>> conn.connection_started False

Which is changed to False due the the context manager calling the quit method once the block of code within the with statement has finished executing. If you would like to find out how all of this is implemented you can take a look at the source code.

1.4.5 Middlewares and Middlemen

One of the more powerful features of Mailthon is the ability to add middleware- which are basically functions that allow for certain features, e.g. TLS, Auth which provide for TLS and authentication, respectively. Let’s make our own middleware to see how all of this is done:

def my_middleware(must_have=()): def func(conn): for item in must_have: assert hasattr(conn, item) return func

Then we need to put our middleware in what’s known as a middleware stack. It is basically a list of callables which will be invoked with the transport object. Using our Postman class:

postman.use(my_middleware(['quit']))

Which will add the closure into the middleware stack and assert that the transport object has the quit attribute/method. More powerful middleware can certainly be programmed via classes, the recommended way if you want to make extensible middlewares is to subclass from the Middleware class:

from mailthon.middleware import Middleware

class MyMiddleware (Middle): def call(self, conn): pass

The registered middlewares will be called by the connection() method to set up the connection. If any exception is raised, the connection is automatically closed.

1.4. In Depth Guide 13

CHAPTER 2

Developer Reference

If you are looking for specific classes, methods or functions, or curious about why certain things work the way they work, this part of the documentation is for you.

2.1 API

2.1.1 Postman Object

class mailthon.postman. Postman (host, port, middlewares=(), options=None) Encapsulates a connection to a server and knows how to send MIME emails over a certain transport. When subclassing, change the transport and response_cls class variables to tweak the transport used and the response class, respectively. Parameters

• host – The address to a server.
• port – Port to connect to.
• middlewares – An iterable of middleware that will be used by the Postman.
• options – Dictionary of options to be passed to the underlying transport. transport The transport class to be used. This defaults to the smtplib.SMTP class, but can be any callable that accepts a host, port, and the additional arguments specified in the options parameter. The recom- mended way to change this is via subclassing: class HTTPPostman (Postman): transport = HTTPMailingProtocol

Although you can change it during runtime for special purposes such as testing. The transport class must also support the ehlo, noop, quit, and sendmail methods. For reference:

• smtplib.SMTP.sendmail()
• smtplib.SMTP.noop()
• smtplib.SMTP.ehlo()
• smtplib.SMTP.quit() response_cls The response class to be used. Defaults to the SendmailResponse class. It must be a callable which has the same type signature, and is recommended to be changed only by subclasses:

class HTTPPostman (Postman): response_cls = MyResponse

It will be called with the response returned by the noop() method, or the noop method of your own transport class. connection (*args, **kwds) A context manager that returns a connection to the server using some transport, defaulting to SMTP. The transport will be called with the server address, port, and options that have been passed to the constructor, in that order. deliver (conn, envelope) Deliver an envelope using a given connection conn, and return the response object. Does not close the connection. send (envelope) Sends an envelope and return a response object. use (middleware) Use a certain callable middleware, i.e. append it to the list of middlewares, and return it so it can be used as a decorator. Note that the middleware is added to the end of the middlewares list, so it will be called last.

2.1.2 Envelope Object

class mailthon.envelope. Envelope (headers, enclosure, mail_from=None) Encapsulates the concept of an Envelope- there can be multiple stamps (headers) and multiple “things” inside the enclosure. Parameters

• headers – An iterable/mapping of headers.
• enclosure – A list of enclosure objects. mail_from Dictates the sender argument being passed to the SMTP.sendmail method. This is different from the sender property as it is the real sender, but the sender property is merely what appears on the email. If it is not set, the real sender is then inferred from the headers. mime () Returns a mime object. Internally this generates a MIMEMultipart object, attaches the enclosures, then prepares it using the internal headers object. receivers Returns a list of receiver addresses. sender Returns the sender of the envelope, obtained from the headers. string () Returns the MIME object as a string- i.e., calls the as_string method of the generated MIME object.

2.1.3 Enclosure Objects

class mailthon.enclosure. Enclosure (headers=()) Base class for Enclosure objects to inherit from. An enclosure is a part of the enclosure in a real envelope- it contains part of the content to be sent.

16 Chapter 2. Developer Reference