reStructuredText

The following directives are supported:

.. code-block:: python
.. code:: python
.. codeblock-name: <name>
.. literalinclude::

Any code directive, such as .. code-block:: python, .. code:: python, .. literalinclude:: or literal blocks with a preceding .. codeblock-name: <name>, will be collected and executed automatically, if your pytest configuration allows that.

Usage examples

Standalone code blocks

`code-block` directive

Note

Note that :name: value has a test_ prefix.

Filename: README.rst

.. code-block:: python
   :name: test_basic_example

   import math

   result = math.pow(3, 2)
   assert result == 9

`literalinclude` directive

Filename: README.rst

.. literalinclude:: examples/python/basic_example.py
    :name: test_li_basic_example

`codeblock-name` directive

You can also use a literal block with a preceding name comment:

Filename: README.rst

.. codeblock-name: test_grouping_example_literal_block
This is a literal block::

   y = 5
   print(y * 2)

Grouping multiple `code-block` directives

It’s possible to split one logical test into multiple blocks. They will be tested under the first :name: specified. Note the .. continue:: directive.

Note

Note that continue directive of the test_grouping_example_part_2 and test_grouping_example_part_3 refers to the test_grouping_example.

Filename: README.rst

.. code-block:: python
   :name: test_grouping_example

   x = 1

Some intervening text.

.. continue: test_grouping_example
.. code-block:: python
   :name: test_grouping_example_part_2

   y = x + 1  # Uses x from the first snippet
   assert y == 2

Some intervening text.

.. continue: test_grouping_example
.. code-block:: python
   :name: test_grouping_example_part_3

   print(y)  # Uses y from the previous snippet

The above mentioned three snippets will run as a single test.

Async

You can use top-level await in your code blocks. The code will be automatically wrapped in an async function.

Filename: README.rst

.. code-block:: python
    :name: test_async_example

    import asyncio

    result = await asyncio.sleep(0.1, result=42)
    assert result == 42

Adding pytest markers to `code-block` and `literalinclude` directives

It’s possible to add custom pytest markers to your code-block or literalinclude directives. That allows adding custom logic and mocking in your conftest.py.

In the example below, django_db marker is added to the code-block directive.

Note

Note the pytestmark directive django_db marker.

Filename: README.rst

.. pytestmark: django_db
.. code-block:: python
    :name: test_django

    from django.contrib.auth.models import User

    user = User.objects.first()

In the example below, django_db marker is added to the literalinclude directive.

Filename: README.rst

.. pytestmark: django_db
.. literalinclude:: examples/python/django_example.py
    :name: test_li_django_example

Requesting pytest fixtures for `code-block` and `literalinclude` directives

It’s possible to request existing or custom pytest fixtures in code-block or literalinclude directives. That allows adding custom logic and mocking in conftest.py.

In the example below, tmp_path fixture is requested for the code-block directive.

Note

Note the pytestfixture directive tmp_path fixture.

Filename: README.rst

.. pytestfixture: tmp_path
.. code-block:: python
    :name: test_path

    d = tmp_path / "sub"
    d.mkdir()  # Create the directory
    assert d.is_dir()  # Verify it was created and is a directory

In the example below, tmp_path fixture is requested for the literalinclude directive.

Filename: README.rst

.. pytestfixture: tmp_path
.. literalinclude:: examples/python/tmp_path_example.py
    :name: test_li_tmp_path_example

Multiple pytestfixture directives are supported. Add one on each line.

Note

When combining pytestfixture and continue directives together, request pytest-fixtures only in the first code-block, as they will automatically become available in all continuing blocks.

Custom pytest-fixtures are supported as well. Just define them in your conftest.py file.

Customisation/hooks

Tests can be extended and fine-tuned using pytest’s standard hook system.

Below is an example workflow:

Add custom pytest markers to the code-block or literalinclude (fakepy, aws, openai).
Implement pytest hooks in conftest.py to react to those markers.

Add custom pytest markers

Add `fakepy` marker

The example code below will generate a PDF file with random text using fake.py library. Note, that a fakepy marker is added to the code-block.

In the Implement pytest hooks section, you will see what can be done with the markers.

Note

Note the pytestmark directive fakepy marker.

Filename: README.rst

.. pytestmark: fakepy
.. code-block:: python
    :name: test_create_pdf_file

    from fake import FAKER

    FAKER.pdf_file()

In the example code below, a fakepy marker is added to the literalinclude block.

Filename: README.rst

.. pytestmark: fakepy
.. literalinclude:: examples/python/create_pdf_file_example.py
    :name: test_li_create_pdf_file

Add `aws` marker

Sample boto3 code to create a bucket on AWS S3.

Note

Note the pytestmark directive aws marker.

Filename: README.rst

.. pytestmark: aws
.. code-block:: python
    :name: test_create_bucket

    import boto3

    s3 = boto3.client("s3", region_name="us-east-1")
    s3.create_bucket(Bucket="my-bucket")
    assert "my-bucket" in [b["Name"] for b in s3.list_buckets()["Buckets"]]

Add `openai` marker

Sample openai code to ask LLM to tell a joke. Note, that next to a custom openai marker, xfail marker is used, which allows underlying code to fail, without marking entire test suite as failed.

Note

Note the pytestmark directive xfail and openai markers.

Filename: README.rst

.. pytestmark: xfail
.. pytestmark: openai
.. code-block:: python
    :name: test_tell_me_a_joke

    from openai import OpenAI

    client = OpenAI()
    completion = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "developer", "content": "You are a famous comedian."},
            {"role": "user", "content": "Tell me a joke."},
        ],
    )

    assert isinstance(completion.choices[0].message.content, str)

Implement pytest hooks

In the example below:

moto is used to mock AWS S3 service for all tests marked as aws.
Environment variable OPENAI_BASE_URL is set to http://localhost:11434/v1 (assuming you have Ollama running) for all tests marked as openai.
FILE_REGISTRY.clean_up() is executed at the end of each test marked as fakepy.

Filename: conftest.py

import os
from contextlib import suppress

import pytest

from fake import FILE_REGISTRY
from moto import mock_aws
from pytest_codeblock.constants import CODEBLOCK_MARK

# Modify test item during collection
def pytest_collection_modifyitems(config, items):
    for item in items:
        if item.get_closest_marker(CODEBLOCK_MARK):
            # All `pytest-codeblock` tests are automatically assigned
            # a `codeblock` marker, which can be used for customisation.
            # In the example below we add an additional `documentation`
            # marker to `pytest-codeblock` tests.
            item.add_marker(pytest.mark.documentation)
        if item.get_closest_marker("aws"):
            # Apply `mock_aws` to all tests marked as `aws`
            item.obj = mock_aws(item.obj)


# Setup before test runs
def pytest_runtest_setup(item):
    if item.get_closest_marker("openai"):
        # Send all OpenAI requests to locally running Ollama for all
        # tests marked as `openai`. The tests would x-pass on environments
        # where Ollama is up and running (assuming, you have created an
        # alias for gpt-4o using one of the available models) and would
        # x-fail on environments, where Ollama isn't runnig.
        os.environ.setdefault("OPENAI_API_KEY", "ollama")
        os.environ.setdefault("OPENAI_BASE_URL", "http://localhost:11434/v1")


# Teardown after the test ends
def pytest_runtest_teardown(item, nextitem):
    # Run file clean up on all tests marked as `fakepy`
    if item.get_closest_marker("fakepy"):
        FILE_REGISTRY.clean_up()

View llms.txt version

reStructuredText

Usage examples

Standalone code blocks

code-block directive

literalinclude directive

codeblock-name directive

Grouping multiple code-block directives

Async

Adding pytest markers to code-block and literalinclude directives

Requesting pytest fixtures for code-block and literalinclude directives

Customisation/hooks

Add custom pytest markers

Add fakepy marker

Add aws marker

Add openai marker

Implement pytest hooks

`code-block` directive

`literalinclude` directive

`codeblock-name` directive

Grouping multiple `code-block` directives

Adding pytest markers to `code-block` and `literalinclude` directives

Requesting pytest fixtures for `code-block` and `literalinclude` directives

Add `fakepy` marker

Add `aws` marker

Add `openai` marker