Pytest is a mature and full-featured testing framework, from small tests to large scale functional tests for applications and libraries alike.
Pytest is simple to get started with. To write tests, you don't need classes; you can write simple functions that start with test and use Python's built-in assert statement:
from fibo import fibonacci
def test_fibo():
assert fibonacci(4) == 3
That's it. You import your code, write a function, and use plain assert calls to ensure they are working as you expect: no need to make a subclass and use various self.assert* methods to do your testing. And the beautiful thing is that it also provides helpful output when an assertion fails:
λ pytest test_fibo2.py -q
F [100%]
============================= FAILURES ==============================
_____________________________ test_fibo _____________________________
def test_fibo():
> assert fibonacci(4) == 3
E assert 5 == 3
E + where 5 = fibonacci(4)
test_fibo2.py:4: AssertionError
1 failed in 0.03 seconds
Notice that the values involved in the expression and the code around it are displayed to make it easier to understand the error.
Pytest not only makes it simple to write tests, it has many command-line options that increase productivity, such as running just the last failing tests, or running a specific group of tests by name or because they're specially marked.
Creating and managing testing resources is an important aspect that is often overlooked in tutorials or overviews of testing frameworks. Tests for real-world applications usually need complex setups, such as starting a background worker, filling up a database, or initializing a GUI. Using pytest, those complex test resources can be managed by a powerful mechanism called fixtures. Fixtures are simple to use but very powerful at the same time, and many people refer to them as pytest's killer feature. They will be shown in detail in Chapter 4, Fixtures.
Customization is important, and pytest goes a step further by defining a very powerful plugin system. Plugins can change several aspects of the test run, from how tests are executed to providing new fixtures and capabilities to make it easy to test many types of applications and frameworks. There are plugins that execute tests in a random order each time to ensure tests are not changing global state that might affect other tests, plugins that repeat failing tests a number of times to weed out flaky behavior, plugins that show failures as they appear instead of only at the end of the run, and plugins that execute tests across many CPUs to speed up the suite. There are also plugins that are useful when testing Django, Flask, Twisted, and Qt applications, further plugins for the acceptance testing of web applications using Selenium. The number of external plugins is really staggering: at the time of writing, there are over 500 pytest plugins available to be installed and used right away (http://plugincompat.herokuapp.com/).
To summarize pytest:
- You use plain assert statements to write your checks, with detailed reporting
- pytest has automatic test discovery
- It has fixtures to manage test resources
- It has many, many plugins to expand its built-in capabilities and help test a huge number of frameworks and applications
- It runs unittest based test suites out of the box and without any modifications, so you can gradually migrate existing test suites
For these reasons, many consider pytest to be a Pythonic approach to writing tests in Python. It makes it easy to write simple tests and is powerful enough to write very complex functional tests. Perhaps more importantly, though, pytest makes testing fun.
Writing automated tests, and enjoying their many benefits, will become natural with pytest.