pytest: an overview for new users
Introduction
pytest is a testing framework for Python. Like Python, it uses a highly readable syntax. It can both be used for small tests, and also for complex functional testing. This article aims to give an overview of pytest and highlight some of its features.
assert Statements
Assert helper functions are commonly used by testing frameworks. What makes them different in pytest is the syntax. Instead of using different helper functions, like assertTrue(x), assertEqual(a, b), assertIn(a, b), in pytest, you simply use assert followed by any expression. In addition, pytest will generate traceback information with details about why the assertion failed.
Here are the most common assert statements for pytest with examples:
- Equal to / not equal to
assert "Scott" == "Scott" # Test will pass assert "Scott" == "Steve" # Test will fail assert "Scott" != "Steve" # Test will pass assert "Scott" != "Scott" # Test will fail
- Greater than or less than [value]
assert 10 > 2 # Test will pass assert 10 < 2 # Test will fail
- Modulo is equal to [value]
assert 10 % 2 == 0 # Test will pass assert 10 % 3 == 2 # Test will fail
- in and not in [iterable]
favourite_colours = ['blue', 'black', 'orange'] assert "blue" in favourite_colours # Test will pass assert "blue" not in favourite_colours # Test will fail
- type() is [value]
assert type("name") is str # Test will pass
assert type("name") is not str # Test will fail
- isinstance
assert isinstance(10, int) # Test will pass assert isinstance(10, str) # Test will fail
- Boolean is [Boolean Type]
true = 10 == 10 assert true is True # Test will pass assert true is False # Test will fail
Exceptions
There are cases where we want our code to raise an exception. Examples are a ValueError exception when trying to multiply by zero, or DivsionByZero exception when trying to divide by zero. For these cases, you can use a context manager called raises. The test will only pass if the provided code causes the given exception.
Inside the test definition, we add
with pytest.raises([Name of the expected exception]): [expression to be evaluated]
A working example for a calculator app with a Class Calculator and a function multiply(self, *args) looks like this:
def test_multiply_by_zero_raises_exception():
calculator = Calculator()
with pytest.raises(ValueError):
calculator.multiply(3, 0)
Marking tests
You can either run all tests in the current directory, or specify a single test with the '-k' option followed by the test name in quotes, e.g.
pytest -k "test_multiply_by_zero_raises_exception"
But what if you want to run more than one, but less than all tests in one go? Both creating subdirectories or adding common prefixes or suffixes might not always be the best solution.
pytest allows you to add markers to one or more tests. Simply add the decorator @pytest.mark.name_of_marker to the top of the test definition. A test can have multiple markers, and a marker can be used for multiple tests.
All markers should be listed in the pytest.ini, tox.ini, or setup.cfg file, and each marker should be added in a new line. The format is
markers =
{name of marker_1}
{name of marker_2}
To run all tests with the same marker, use the option -m followed by the name of the marker, e.g.
pytest -m newly_added
It is also possible to combine several markers with and, or, or not, e.g.
pytest -m newly_added and not api_test
This would run all tests with the marker newly_added, but exclude tests with the marker api_test.
Expecting tests to fail
If a test is expected to fail, it can be marked with the decorator @pytest.mark.xfail(). It is good practice to give a reason as a parameter to make it clear why the test is expected to fail. Possible reasons are a bug that has not yet been fixed or the feature not being supported in certain versions.
Here is an example of a test expected to fail, with a reason given as argument:
@pytest.mark.django_db
@pytest.mark.xfail(
reason='actor will not be created without required data'
)
def test_actor_form_is_not_valid_without_required_data():
"""Verify the form is not valid without required data"""
data = {
'name': "somebody"
}
form = ActorForm(data=data)
form.save()
assert True is form.is_valid()
If a test marked with the xfail decorator does indeed fail, the outcome will be presented as xfail in the verbose version, or as x in the short version. Should it pass despite the expectation, the outcome will be presented as XPASS or X. Note the uppercase in these cases.
Parametrized testing
One of the best features of pytest, imho, is parametrized testing. It allows us to run a test with multiple sets of data. To use parametrized testing, we need another decorator. @pytest.mark.parametrize(), which takes two parameters. First, we pass the name for the data as a string. Then, we pass a list of values as the second argument. It is possible to pass multiple strings as a comma-separated list for the names to create tuples from the name and the value. Finally, we pass the first argument as an argument for the test definition.
Here is an example to check whether several templates render as expected in a Django app:
@pytest.mark.parametrize("templates", [
("home"), ("profile"), ("tickets")
])
@pytest.mark.django_db
def test_home_page_url_works(client, templates):
url = reverse(templates)
response = client.get(url)
assert response.status_code == 200
Fixtures
In pytest, fixtures are functions that are run before the actual test functions. In addition, they can also be run after the test functions. Possible use cases are getting a data set for the test to work with. This is similar to fixtures in Django that load initial data into the database. Another use case is getting the system into a specific state before the test is run.
Using fixtures to store data for testing has the advantage that we can see where a test using a fixture failed. In case of an AssertionError or Exception in the test function, pytest will report the error and point to the line of code containing the error. The test will be reported as a failure. If the AssertionError or any exception happens in the fixture, the test is reported as ERROR instead of failure, and the line containing the error is pointed out. This makes it easy to know why and where an error or failure occurred.
Fixtures can be used as arguments for other fixtures. Below is a simple example of a fixture to create user data, and a second fixture that uses this data to create a test user. It is also possible to pass multiple fixtures as arguments.
import pytest
from django.contrib.auth.models import User
@pytest.fixture
def user_data():
return {
'email': 'user_email',
'username': 'user_name',
'password': 'user_password12345'
}
@pytest.fixture
def create_test_user(user_data):
test_user = User.objects.create_user(**user_data)
test_user.set_password(user_data.get('password'))
return test_user
And here is an example of a test that uses the data from the user_data fixture:
import pytest
@pytest.mark.django_db
def test_profile_page_not_accessible_for_unauthenticated_users(
client, user_data
):
# user_data passed to show that the user needs to be logged in
"""Verify that the page returns an error message for
unauthenticated users"""
url = reverse("profile")
response = client.get(url)
assert response.status_code == 302
conftest.py
Fixtures can be put into individual test files, which are only available for tests inside the file. To make a fixture available for all tests inside a project, put the fixtures into the conftest.py file. This file does not need to be imported, as it will be read automatically by pytest.
In addition to fixtures, hook functions can be put into the conftest.py file to allow modifications through plugins. But this is outside the scope of this article.
Parametrizing fixtures
Fixtures in pytest can be parametrized. The params argument is added to the pytest.fixture decorator, and then accessed inside the fixture by using pytest's request fixture:
@pytest.fixture(params=["my_password", "G7W1%prt?", "1234567"])
def passwords(request):
return request.param
To use this parametrized fixture in a test function, we pass it as an argument:
def test_password_length(passwords):
assert len(passwords) > 6
A test function that uses a parametrized fixture will be called multiple times.
Further information
This article aimed to give a brief introduction into pytest and some of its features. If you want to learn more, take a look at the full documentation.
