Unit Testing in Python with Pytest

2020-07-28 12:00:14 | #programming #python

Unit Testing is a software testing method, with the purpose of testing the smallest testable components of your program.

For example, you may want to ensure that your program's add or subtract functions are returning accurate values. Simple functions, with one job to perform, are best not just for clean code, but testable code. It's much easier to maintain and debug fuctions that in a small amount of input, and return a simple, verifiable output.

You want to eliminate state, and reduce dependencies, if you want to reduce bugs. In other words, you don't want to be creating a global variable that gets modified by multiple functions over time. This adds complexity to your application because now you have to keep track of those modifications and combinations of modifications across your dependencies. It's better to have sets of localized variables and simple functions. Each function should perform a job, without the higher level program being concerned with how. And with unit testing, we can verify that each function is performing its job as expected, under a variety of different scenarios.

How to Set Up a Python Project

How to Create a Python Project with Windows 10 PowerShell 2.0+

cd ~
New-Item -ItemType "directory" -Path ".\python-pytest-example"
cd python-boilerplate
New-Item -ItemType "file" -Path . -Name "main.py"
New-Item -ItemType "file" -Path . -Name "test_main.py"
virtualenv venv
.\venv\Scripts\activate
pip install pytest

To verify that the virtual environment is active, make sure (venv) is in the PowerShell command prompt. For example, (venv) PS C:\Users\username\python-pytest-example>

How to Create a Python Project with Linux Ubuntu 14.04+ or macOS

cd ~
mkdir python-pytest-example
cd python-pytest-example
virtualenv -p python3 venv
source venv/bin/activate
pip install pytest
touch main.py
touch test_main.py

To verify that the virtual environment is active, make sure (venv) is in the terminal command prompt.

This will create the following files and folders, activate the virtual environment, and install pytest into that virtual environment.

▾ python-pytest-example/
  ▸ venv/
  main.py
  test_main.py

Example Code for How to Add and Subtract with Python

Filename: main.py

The following code defines some simple addition and subtraction functions.

def add(a, b):
    return a + b


def subtract(a, b):
    return a - b


if __name__ == '__main__':
    print(add(1, 2))

Example Code for Pytest Unit Tests

Using pytest, we can verify that the functions are returning the correct values. We do this with assert statements. The format for a basic assertion assert x == y. If x equals y, the test will pass. So in the following example, we're asserting, using the add function, (1 + 2) == 2. Obviously, 3 does not equal 2, so this test should fail.

Filename: test_main.py

from main import add, subtract


def test_add():
    assert add(1, 2) == 2


def test_subtract():
    assert subtract(2, 1) == 1

Running the test with pytest test_main.py should produce the following report:

======================== test session starts ========================
platform linux -- Python 3.8.2, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: /home/username/repos/code-samples/unit-testing-in-python-with-pytest-and-nosetests
collected 2 items

test_main.py F.                                               [100%]

======================== FAILURES ==================================
________________________ test_add __________________________________

    def test_add():
>       assert add(1, 2) == 2
E       assert 3 == 2
E        +  where 3 = add(1, 2)

test_main.py:5: AssertionError
================= short test summary info ==========================
FAILED test_main.py::test_add - assert 3 == 2
=============== 1 failed, 1 passed in 0.03s ========================

If we want our test to pass, we need to assert with the correct value.

from main import add, subtract


def test_add():
    assert add(1, 2) == 3


def test_subtract():
    assert subtract(2, 1) == 1

Now when you run the test with pytest test_main.py, you should get the following output:

======================== test session starts ========================
platform linux -- Python 3.8.2, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: /home/username/repos/code-samples/unit-testing-in-python-with-pytest-and-nosetests
collected 2 items                                                                

test_main.py ..                                               [100%]

==================== 2 passed in 0.01s =============================

Skipping Tests in Python with Pytest

Filename: test_main.py

Sometimes, we won't need all tests to run and we'll want to skip tests to speed up debugging. The pytest module comes with a decorator that does just that. Just import pytest and decorate the function you want to skip with @pytest.mark.skip. Update your test_main.py like so:

import pytest
from main import add, subtract


def test_add():
    assert add(1, 2) == 3


@pytest.mark.skip
def test_subtract():
    assert subtract(2, 1) == 1

After running your test with pytest test_main.py, you should see the following report, indicating which tests have been skipped:

======================= test session starts =============================
platform linux -- Python 3.8.2, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: /home/username/repos/code-samples/unit-testing-in-python-with-pytest-and-nosetests
collected 2 items                                                             

test_main.py .s                                               [100%]

========================== 1 passed, 1 skipped in 0.01s =============================
  

Custom Markers for Your Unit Tests with Pytest

Pytest also gives us the ability to group and mark which tests we want to run. First, we have to define our custom markers in a conftest.py file. Your project folder should now look like the following:

▾ python-pytest-example/
  ▸ venv/
  conftest.py
  main.py
  test_main.py

Example conftest.py File with Custom Markers

Filename: conftest.py

def pytest_configure(config):
    config.addinivalue_line("markers", "simple: marker for simple tests")
    config.addinivalue_line("markers", "complex: marker for complex tests")

We're going to want to update a few other files, as well.

Filename: main.py

def add(a, b):
    return a + b


def subtract(a, b):
    return a - b


def multiply(a, b):
    return a * b


def divide(a, b):
    return a / b


if __name__ == '__main__':
    print(add(1, 2))

Filename: test_main.py

import pytest
from main import add, subtract, multiply, divide


@pytest.mark.simple
def test_add():
    assert add(1, 2) == 3


@pytest.mark.simple
def test_subtract():
    assert subtract(2, 1) == 1


@pytest.mark.complex
def test_multiply():
    assert multiply(2, 2) == 4


@pytest.mark.complex
def test_divide():
    assert divide(25, 5) == 5

Now, when you run the test, you can indicate which group you want to run in the command line. Run pytest -m simple main.py and you'll see how only the tests marked "simple" run, whereas pytest -m complex main.py only runs the complex tests. Feel free to define your own custom markers, and check the pytest docs for more examples.

You may get the following error while running the python test_main.py command:

============== warnings summary =================
test_main.py:11
/home/username/code-samples/test_main.py:11: PytestUnknownMarkWarning: Unknown pytest.mark.simple - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
  @pytest.mark.simple

Pytest either can't find your conftest.py file or your custom marker isn't configured, properly. Scroll up to the Example conftest.py File with Custom Markers section for a proper configuration.

Pytest Fixtures

Fixtures allow us to predefine a set of objects that our tests can rely on. In this case, our fixture is simply going to return some data. This is handy for when you want to predefine, for example, the JSON data that a database would return, without having to set up the database in your test environment. In our example, we're returning a tuple, and unpacking it with * before passing it into our add and subtract functions.

Filename: test_main.py

import pytest
from main import add, subtract, multiply, divide


@pytest.fixture
def data():
    return (3, 4)


@pytest.mark.simple
def test_add(data):
    assert add(*data) == 7


@pytest.mark.simple
def test_subtract(data):
    assert subtract(*data) == -1


@pytest.mark.complex
def test_multiply():
    assert multiply(2, 2) == 4


@pytest.mark.complex
def test_divide():
    assert divide(25, 5) == 5

Rerun your test with pytest test_main.py

Pytest Parameterized Tests

Another way of predefining our test data is through the parameterize decorator. Update your test_main.py like so:

Filename: test_main.py

import pytest
from main import add, subtract, multiply, divide


@pytest.fixture
def data():
    return (3, 4)


@pytest.mark.simple
def test_add(data):
    assert add(*data) == 7


@pytest.mark.simple
def test_subtract(data):
    assert subtract(*data) == -1


@pytest.mark.complex
@pytest.mark.parametrize("data, expected", [((3, 3), 9), ((5, 5), 25), ((6, 6), 36)])
def test_multiply(data, expected):
    assert multiply(*data) == expected


@pytest.mark.complex
def test_divide():
    assert divide(25, 5) == 5

Mocking in Pytest

Pytest mocks allow us to fake the behavior and return value of an existing function, for testing purposes. For example, sometimes we don't actually need to connect to a database, or authenticate a user's session for test purposes. But we can't go in an modify the code to turn off database connections or authentication. So we use a mock function to override the existing function, let the function get called as it normally would, and return a value that helps us move forward to what the test needs to focus on.

First, we have to update some files again. We're going to create anauth.py file with some simple authentication logic and require the divide function to check if a user is logged in before performing the calculation. This isn't a realistic scenario, but it does teach you the basic principles. Your project folder should now look like the following:

▾ python-pytest-example/
  ▸ venv/
  auth.py
  conftest.py
  main.py
  test_main.py

Example Code for auth.py

Filename: auth.py

This just sets session to None and returns True or False based on if a session exists. For this example, it will always return False, because we don't have any real authentication logic.

session = None


def is_logged_in():
    # Logic for checking if user session exists
    return session is not None

Example Code for main.py

Filename: main.py

As you can see, we've updated the divide function to rely on auth to see if the user is logged in. is_logged_in will always return False, so divide will never run the calculation, returning Unauthorized instead.

import auth


def add(a, b):
    return a + b


def subtract(a, b):
    return a - b


def multiply(a, b):
    return a * b


def divide(a, b):
    if auth.is_logged_in():
        return a / b
    else:
        return 'Unauthorized'


if __name__ == '__main__':
    print(add(1, 2))

Example Code for test_main.py

Filename: test_main.py

After making the following updates, you will notice we now have two scenarios for divide. The first scenario expects 'Unauthorized' to be returned. This is OK because we don't have any authentication logic and we don't expect the user to be logged in. Or maybe the user entered in the wrong credentials. But what if we are in a scenario where we don't care about testing authentication? What if we want to skip all the authentication parts without touching the main.py code, and we just want to see if divide is working properly?

Lines 33-36: defines a test case where we can mock/override the auth.is_logged_in function, temporarily, for this test case, and force it to always return True so that we can move on to testing the divide logic. Run the test again with pytest test_main.py

import pytest
from main import add, subtract, multiply, divide
import auth


@pytest.fixture
def data():
    return (3, 4)


@pytest.mark.simple
def test_add(data):
    assert add(*data) == 7


@pytest.mark.simple
def test_subtract(data):
    assert subtract(*data) == -1


@pytest.mark.complex
@pytest.mark.parametrize("data, expected", [((3, 3), 9), ((5, 5), 25), ((6, 6), 36)])
def test_multiply(data, expected):
    assert multiply(*data) == expected


@pytest.mark.complex
def test_divide_unauthorized():
    assert divide(25, 5) == 'Unauthorized'


@pytest.mark.complex
def test_divide(mocker):
    mocker.patch.object(auth, 'is_logged_in')
    auth.is_logged_in.return_value = True
    assert divide(25, 5) == 5

You may get the following error while running the python test_main.py command:

============== warnings summary =================
E       fixture 'mocker' not found
>       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, datos, doctest_namespace, monkeypatch, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

This indicates that pytest-mock is missing. Install it with pip install pytest-mock

We now have a set of test scenarios that explains what pytest is capable of. Notice that you can even stack decorators, like in lines 21-22. That concludes this lesson on unit testing with pytest. Feel free to apply these concepts to your own real world scenarios.

Comments

You must log in to comment. Don't have an account? Sign up for free.

Subscribe to Our Newsletter

Would you like to receive free whitepapers and other IT news? Just leave your email address below. You may opt out at any time.



Tell Us About Your Project









Contact Us

Do you have a specific IT problem that needs solving or just have a general IT question? Use the contact form to get in touch with us and an IT professional will be with you, momentarily.

Hire Us

We offer web development, enterprise software development, QA & testing, google analytics, domains and hosting, databases, security, IT consulting, and other IT-related services.

Free IT Tutorials

Head over to our tutorials section to learn all about working with various IT solutions.

Contact