behave Documentation, Lecture notes of Software Development

Download behave Documentation and more Lecture notes Software Development in PDF only on Docsity!

behave Documentation

Release 1.2.

Benno Rice, Richard Jones and Jens Engel

ii

behave is behaviour-driven development, Python style.

Behavior-driven development (or BDD) is an agile software development technique that encourages collaboration

between developers, QA and non-technical or business participants in a software project. We have a page further

describing this philosophy.

behave uses tests written in a natural language style, backed up by Python code.

Once you’ve installed behave, we recommend reading the

• tutorial first and then
• feature test setup,
• behave API and
• related software (things that you can combine with behave)
• finally: how to use and configure the behave tool.

There is also a comparison with the other tools available.

Contents 1

CHAPTER 1

Contents

1.1 Installation

1.1.1 Using pip (or... )

Category Stable version

Precondition pip (or setuptools) is installed

Execute the following command to install behave with pip:

pip install behave

To update an already installed behave version, use:

pip install -U behave

As an alternative, you can also use easy_install to install behave:

easy_install behave # CASE: New installation. easy_install -U behave # CASE: Upgrade existing installation.

Hint: See also pip related information for installing Python packages.

1.1.2 Using a Source Distribution

After unpacking the behave source distribution, enter the newly created directory “behave-” and run:

python setup.py install

1.1.3 Using the Github Repository

Category Bleading edge

Precondition pip is installed

Run the following command to install the newest version from the Github repository:

pip install git+https://github.com/behave/behave

To install a tagged version from the Github repository, use:

pip install git+https://github.com/behave/behave@

where is the placeholder for an existing tag.

1.2 Tutorial

First, install behave.

Now make a directory called “features”. In that directory create a file called “tutorial.feature” containing:

Feature: showing off behave

Scenario: run a simple test Given we have behave installed When we implement a test Then behave will test it for us!

Make a new directory called “features/steps”. In that directory create a file called “tutorial.py” containing:

from behave import (^) *

@given ('we have behave installed') def step_impl(context): pass

@when ('we implement a test') def step_impl(context): assert True is not False

@then ('behave will test it for us!') def step_impl(context): assert context.failed is False

Run behave:

% behave Feature: showing off behave # features/tutorial.feature:

Scenario: run a simple test # features/tutorial.feature: Given we have behave installed # features/steps/tutorial.py: When we implement a test # features/steps/tutorial.py: Then behave will test it for us! # features/steps/tutorial.py:

1 feature passed, 0 failed, 0 skipped 1 scenario passed, 0 failed, 0 skipped 3 steps passed, 0 failed, 0 skipped, 0 undefined

Now, continue reading to learn how to make the most of behave.

1.2.1 Features

behave operates on directories containing:

1. feature files written by your Business Analyst / Sponsor / whoever with your behaviour scenarios in it, and

2. a “steps” directory with Python step implementations for the scenarios.

4 Chapter 1. Contents

(continued from previous page) Then the ninja should run for his life And fall off a cliff

In this case behave will look for a step definition for "Then fall off a cliff".

Scenario Outlines

Sometimes a scenario should be run with a number of variables giving a set of known states, actions to take and

expected outcomes, all using the same basic actions. You may use a Scenario Outline to achieve this:

Scenario Outline: Blenders Given I put in a blender, when I switch the blender on then it should transform into

Examples: Amphibians | thing | other thing | | Red Tree Frog | mush |

Examples: Consumer Electronics | thing | other thing | | iPhone | toxic waste | | Galaxy Nexus | toxic waste |

behave will run the scenario once for each (non-heading) line appearing in the example data tables.

Step Data

Sometimes it’s useful to associate a table of data with your step.

Any text block following a step wrapped in """ lines will be associated with the step. For example:

Scenario: some scenario Given a sample text loaded into the frobulator """ Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. """ When we activate the frobulator Then we will find it similar to English

The text is available to the Python step code as the “.text” attribute in the Context variable passed into each step

function.

You may also associate a table of data with a step by simply entering it, indented, following the step. This can be

useful for loading specific required data into a model.

Scenario: some scenario Given a set of specific users | name | department | | Barry | Beer Cans | | Pudey | Silly Walks | | Two-Lumps | Silly Walks |

When we count the number of people in each department Then we will find two people in "Silly Walks" But we will find one person in "Beer Cans"

The table is available to the Python step code as the “.table” attribute in the Context variable passed into each

step function. The table for the example above could be accessed like so:

6 Chapter 1. Contents

@given ('a set of specific users') def step_impl(context): for row in context.table: model.add_user(name=row['name'], department=row['department'])

There’s a variety of ways to access the table data - see the Table API documentation for the full details.

1.2.3 Python Step Implementations

Steps used in the scenarios are implemented in Python files in the “steps” directory. You can call these whatever

you like as long as they use the python *.py file extension. You don’t need to tell behave which ones to use - it’ll

use all of them.

The full detail of the Python side of behave is in the API documentation.

Steps are identified using decorators which match the predicate from the feature file: given, when, then and step

(variants with Title case are also available if that’s your preference.) The decorator accepts a string containing the

rest of the phrase used in the scenario step it belongs to.

Given a Scenario:

Scenario: Search for an account Given I search for a valid account Then I will see the account details

Step code implementing the two steps here might look like (using selenium webdriver and some other helpers):

@given ('I search for a valid account') def step_impl(context): context.browser.get('http://localhost:8000/index') form = get_element(context.browser, tag='form') get_element(form, name="msisdn").send_keys('61415551234') form.submit()

@then ('I will see the account details') def step_impl(context): elements = find_elements(context.browser, id='no-account') eq_(elements, [], 'account not found') h = get_element(context.browser, id='account-head') ok_(h.text.startswith("Account 61415551234"), 'Heading %r has wrong text' % h.text)

The step decorator matches the step to any step type, “given”, “when” or “then”. The “and” and “but” step

types are renamed internally to take the preceding step’s keyword (so an “and” following a “given” will become a

“given” internally and use a given decorated step).

Note: Step function names do not need to have a unique symbol name, because the text matching selects the

step function from the step registry before it is called as anonymous function. Hence, when behave prints out the

missing step implementations in a test run, it uses “step_impl” for all functions by default.

If you find you’d like your step implementation to invoke another step you may do so with the Context method

execute_steps().

This function allows you to, for example:

@when ('I do the same thing as before') def step_impl(context): context.execute_steps(''' when I press the big red button (continues on next page)

1.2. Tutorial 7

Context

You’ll have noticed the “context” variable that’s passed around. It’s a clever place where you and behave can store

information to share around. It runs at three levels, automatically managed by behave.

When behave launches into a new feature or scenario it adds a new layer to the context, allowing the new activity

level to add new values, or overwrite ones previously defined, for the duration of that activity. These can be

thought of as scopes.

You can define values in your environmental controls file which may be set at the feature level and then overridden

for some scenarios. Changes made at the scenario level won’t permanently affect the value set at the feature level.

You may also use it to share values between steps. For example, in some steps you define you might have:

@given ('I request a new widget for an account via SOAP') def step_impl(context): client = Client("http://127.0.0.1:8000/soap/") context.response = client.Allocate(customer_first='Firstname', customer_last='Lastname', colour='red')

@then ('I should receive an OK SOAP response') def step_impl(context): eq_(context.response['ok'], 1)

There’s also some values added to the context by behave itself:

table This holds any table data associated with a step.

text This holds any multi-line text associated with a step.

failed This is set at the root of the context when any step fails. It is sometimes useful to use this combined with

the --stop command-line option to prevent some mis-behaving resource from being cleaned up in an

after_feature() or similar (for example, a web browser being driven by Selenium.)

The context variable in all cases is an instance of behave.runner.Context.

1.2.4 Environmental Controls

The environment.py module may define code to run before and after certain events during your testing:

before_step(context, step), after_step(context, step) These run before and after every step.

before_scenario(context, scenario), after_scenario(context, scenario) These run before and after each sce-

nario is run.

before_feature(context, feature), after_feature(context, feature) These run before and after each feature file is

exercised.

before_tag(context, tag), after_tag(context, tag) These run before and after a section tagged with the given

name. They are invoked for each tag encountered in the order they’re found in the feature file. See control-

ling things with tags.

before_all(context), after_all(context) These run before and after the whole shooting match.

The feature, scenario and step objects represent the information parsed from the feature file. They have a number

of attributes:

keyword “Feature”, “Scenario”, “Given”, etc.

name The name of the step (the text after the keyword.)

tags A list of the tags attached to the section or step. See controlling things with tags.

filename and line The file name (or “”) and line number of the statement.

A common use-case for environmental controls might be to set up a web server and browser to run all your tests

in. For example:

1.2. Tutorial 9

-- FILE: features/environment.py

from behave import fixture, use_fixture from behave4my_project.fixtures import wsgi_server from selenium import webdriver

@fixture def selenium_browser_chrome(context):

-- HINT: @behave.fixture is similar to @contextlib.contextmanager

context.browser = webdriver.Chrome() yield context.browser

-- CLEANUP-FIXTURE PART:

context.browser.quit()

def before_all(context): use_fixture(wsgi_server, context, port=8000) use_fixture(selenium_browser_chrome, context)

-- HINT: CLEANUP-FIXTURE is performed after after_all() hook is called.

def before_feature(context, feature): model.init(environment='test')

-- FILE: behave4my_project/fixtures.py

ALTERNATIVE: Place fixture in "features/environment.py" (but reuse is harder)

from behave import fixture import threading from wsgiref import simple_server from my_application import model from my_application import web_app

@fixture def wsgi_server(context, port=8000): context.server = simple_server.WSGIServer(('', port)) context.server.set_app(web_app.main(environment='test')) context.thread = threading.Thread(target=context.server.serve_forever) context.thread.start() yield context.server

-- CLEANUP-FIXTURE PART:

context.server.shutdown() context.thread.join()

Of course, if you wish, you could have a new browser for each feature, or to retain the database state between

features or even initialise the database for each scenario.

1.2.5 Controlling Things With Tags

You may also “tag” parts of your feature file. At the simplest level this allows behave to selectively check parts of

your feature set.

Given a feature file with:

Feature: Fight or flight In order to increase the ninja survival rate, As a ninja commander I want my ninjas to decide whether to take on an opponent based on their skill levels

@slow Scenario: Weaker opponent Given the ninja has a third level black-belt When attacked by a samurai (continues on next page)

10 Chapter 1. Contents

4. only runs scenarios tagged with “@wip”

5. stops at the first error

1.2.7 Fixtures

Fixtures simplify the setup/cleanup tasks that are often needed during test execution.

-- FILE: behave4my_project/fixtures.py (or in: features/environment.py)

from behave import fixture from somewhere.browser.firefox import FirefoxBrowser

-- FIXTURE: Use generator-function

@fixture def browser_firefox(context, timeout=30, **kwargs):

-- SETUP-FIXTURE PART:

context.browser = FirefoxBrowser(timeout, **kwargs) yield context.browser

-- CLEANUP-FIXTURE PART:

context.browser.shutdown()

See Fixtures for more information.

1.2.8 Debug-on-Error (in Case of Step Failures)

A “debug on error/failure” functionality can easily be provided, by using the after_step() hook. The debug-

ger is started when a step fails.

It is in general a good idea to enable this functionality only when needed (in interactive mode). The functionality

is enabled (in this example) by using the user-specific configuration data. A user can:

• provide a userdata define on command-line
• store a value in the “behave.userdata” section of behave’s configuration file

-- FILE: features/environment.py

USE: behave -D BEHAVE_DEBUG_ON_ERROR (to enable debug-on-error)

USE: behave -D BEHAVE_DEBUG_ON_ERROR=yes (to enable debug-on-error)

USE: behave -D BEHAVE_DEBUG_ON_ERROR=no (to disable debug-on-error)

BEHAVE_DEBUG_ON_ERROR = False

def setup_debug_on_error(userdata): global BEHAVE_DEBUG_ON_ERROR BEHAVE_DEBUG_ON_ERROR = userdata.getbool("BEHAVE_DEBUG_ON_ERROR")

def before_all(context): setup_debug_on_error(context.config.userdata)

def after_step(context, step): if BEHAVE_DEBUG_ON_ERROR and step.status == "failed":

-- ENTER DEBUGGER: Zoom in on failure location.

NOTE: Use IPython debugger, same for pdb (basic python debugger).

import ipdb ipdb.post_mortem(step.exc_traceback)

1.3 Behavior Driven Development

Behavior-driven development (or BDD) is an agile software development technique that encourages collaboration

between developers, QA and non-technical or business participants in a software project. It was originally named

12 Chapter 1. Contents

in 2003 by Dan North as a response to test-driven development (TDD), including acceptance test or customer test

driven development practices as found in extreme programming. It has evolved over the last few years.

On the “Agile specifications, BDD and Testing eXchange” in November 2009 in London, Dan North gave the

following definition of BDD:

BDD is a second-generation, outside–in, pull-based, multiple-stakeholder, multiple-scale, high-

automation, agile methodology. It describes a cycle of interactions with well-defined outputs, re-

sulting in the delivery of working, tested software that matters.

BDD focuses on obtaining a clear understanding of desired software behavior through discussion with stakehold-

ers. It extends TDD by writing test cases in a natural language that non-programmers can read. Behavior-driven

developers use their native language in combination with the ubiquitous language of domain-driven design to

describe the purpose and benefit of their code. This allows the developers to focus on why the code should be

created, rather than the technical details, and minimizes translation between the technical language in which the

code is written and the domain language spoken by the business, users, stakeholders, project management, etc.

1.3.1 BDD practices

The practices of BDD include:

• Establishing the goals of different stakeholders required for a vision to be implemented
• Drawing out features which will achieve those goals using feature injection
• Involving stakeholders in the implementation process through outside–in software development
• Using examples to describe the behavior of the application, or of units of code
• Automating those examples to provide quick feedback and regression testing
• Using ‘should’ when describing the behavior of software to help clarify responsibility and allow the soft-

ware’s functionality to be questioned

• Using ‘ensure’ when describing responsibilities of software to differentiate outcomes in the scope of the

code in question from side-effects of other elements of code.

• Using mocks to stand-in for collaborating modules of code which have not yet been written

1.3.2 Outside–in

BDD is driven by business value; that is, the benefit to the business which accrues once the application is in

production. The only way in which this benefit can be realized is through the user interface(s) to the application,

usually (but not always) a GUI.

In the same way, each piece of code, starting with the UI, can be considered a stakeholder of the other modules of

code which it uses. Each element of code provides some aspect of behavior which, in collaboration with the other

elements, provides the application behavior.

The first piece of production code that BDD developers implement is the UI. Developers can then benefit from

quick feedback as to whether the UI looks and behaves appropriately. Through code, and using principles of good

design and refactoring, developers discover collaborators of the UI, and of every unit of code thereafter. This helps

them adhere to the principle of YAGNI, since each piece of production code is required either by the business, or

by another piece of code already written.

1.3.3 The Gherkin language

The requirements of a retail application might be, “Refunded or exchanged items should be returned to stock.” In

BDD, a developer or QA engineer might clarify the requirements by breaking this down into specific examples.

The language of the examples below is called Gherkin and is used by behave as well as many other tools.

1.3. Behavior Driven Development 13

Sometimes the difference between the context, events and outcomes is made more explicit. For instance:

class TestWindow (object): def test_window_close(self):

Given

window = gui.Window("My Window") frame = gui.Frame(window)

When

window.close()

Then

assert_( not frame.isVisible())

However the example is phrased, the effect describes the behavior of the code in question. For instance, from the

examples above one can derive:

• lists should know when they are empty
• window.close() should cause contents to stop being visible

The description is intended to be useful if the test fails, and to provide documentation of the code’s behavior. Once

the examples have been written they are then run and the code implemented to make them work in the same way

as TDD. The examples then become part of the suite of regression tests.

1.3.5 Using mocks

BDD proponents claim that the use of “should” and “ensureThat” in BDD examples encourages developers to

question whether the responsibilities they’re assigning to their classes are appropriate, or whether they can be

delegated or moved to another class entirely. Practitioners use an object which is simpler than the collaborating

code, and provides the same interface but more predictable behavior. This is injected into the code which needs it,

and examples of that code’s behavior are written using this object instead of the production version.

These objects can either be created by hand, or created using a mocking framework such as mock.

Questioning responsibilities in this way, and using mocks to fulfill the required roles of collaborating classes,

encourages the use of Role-based Interfaces. It also helps to keep the classes small and loosely coupled.

1.3.6 Acknowledgement

This text is partially taken from the wikipedia text on Behavior Driven Development with modifications where

appropriate to be more specific to behave and Python.

1.4 Feature Testing Setup

1.4.1 Feature Testing Layout

behave works with three types of files:

1. feature files written by your Business Analyst / Sponsor / whoever with your behaviour scenarios in it, and

2. a “steps” directory with Python step implementations for the scenarios.

3. optionally some environmental controls (code to run before and after steps, scenarios, features or the whole

shooting match).

These files are typically stored in a directory called “features”. The minimum requirement for a features directory

is:

1.4. Feature Testing Setup 15

features/ features/everything.feature features/steps/ features/steps/steps.py

A more complex directory might look like:

features/ features/signup.feature features/login.feature features/account_details.feature features/environment.py features/steps/ features/steps/website.py features/steps/utils.py

Layout Variations

behave has some flexibility built in. It will actually try quite hard to find feature specifications. When launched

you may pass on the command line:

nothing In the absence of any information behave will attempt to load your features from a subdirectory called

“features” in the directory you launched behave.

a features directory path This is the path to a features directory laid out as described above. It may be called

anything by must contain at least one “name.feature” file and a directory called “steps”. The “environ-

ment.py” file, if present, must be in the same directory that contains the “steps” directory (not in the “steps”

directory).

the path to a “name.feature” file This tells behave where to find the feature file. To find the steps directory

behave will look in the directory containing the feature file. If it is not present, behave will look in the parent

directory, and then its parent, and so on until it hits the root of the filesystem. The “environment.py” file, if

present, must be in the same directory that contains the “steps” directory (not in the “steps” directory).

a directory containing your feature files Similar to the approach above, you’re identifying the directory where

your “name.feature” files are, and if the “steps” directory is not in the same place then behave will search

for it just like above. This allows you to have a layout like:

tests/ tests/environment.py tests/features/signup.feature tests/features/login.feature tests/features/account_details.feature tests/steps/ tests/steps/website.py tests/steps/utils.py

Note that with this approach, if you want to execute behave without having to explicitly specify the directory

(first option) you can set the paths setting in your configuration file (e.g. paths=tests).

If you’re having trouble setting things up and want to see what behave is doing in attempting to find your features

use the “-v” (verbose) command-line switch.

1.4.2 Gherkin: Feature Testing Language

behave features are written using a language called Gherkin (with some modifications) and are named

“name.feature”.

These files should be written using natural language - ideally by the non-technical business participants in the

software project. Feature files serve two purposes – documentation and automated tests.

16 Chapter 1. Contents