Markdown

Usage examples

Any fenced code block with a recognized Python language tag (e.g., python, py) will be collected and executed automatically, if your pytest configuration allows that.

Standalone code blocks

Note

Note that name value has a test_ prefix.

Filename: README.md

```python name=test_basic_example
import math

result = math.pow(3, 2)
assert result == 9
```

---

Grouping multiple code blocks

It’s possible to split one logical test into multiple blocks. They will be tested under the first name specified. Note the <!-- continue: test_group_new_syntax --> directive.

Note

Note that continue directive of the test_grouping_example_part_2 and test_grouping_example_part_3 refers to the test_grouping_example.

Filename: README.md

```python name=test_grouping_example
x = 1
```

Some intervening text.

<!-- continue: test_grouping_example -->
```python name=test_grouping_example_part_2
y = x + 1  # Uses x from the first snippet
assert y == 2
```

Some intervening text.

<!-- continue: test_grouping_example -->
```python name=test_grouping_example_part_3

print(y)  # Uses y from the previous snippet
```

The above mentioned three snippets will run as a single test.

---

Async

You can use top-level await in your code blocks. The code will be automatically wrapped in an async function.

Filename: README.md

```python name=test_async_example
import asyncio

result = await asyncio.sleep(0.1, result=42)
assert result == 42
```

---

Adding pytest markers to code blocks

It’s possible to add custom pytest markers to your code blocks. That allows adding custom logic and mocking in your conftest.py.

In the example below, django_db marker is added to the code block.

Note

Note the pytestmark directive django_db marker.

Filename: README.md

<!-- pytestmark: django_db -->
```python name=test_django
from django.contrib.auth.models import User

user = User.objects.first()
```

Requesting pytest fixtures for code blocks

It’s possible to request existing or custom pytest fixtures for code blocks. That allows adding custom logic and mocking in conftest.py.

In the example below, tmp_path fixture is requested for the code block.

Note

Note the pytestfixture directive tmp_path fixture.

Filename: README.md

<!-- pytestfixture: tmp_path -->
```python name=test_path
d = tmp_path / "sub"
d.mkdir()  # Create the directory
assert d.is_dir()  # Verify it was created and is a directory
```

---

Multiple pytestfixture directives are supported. Add one on each line.

Note

When combining pytestfixture and continue directives together, request pytest-fixtures only in the first code-block, as they will automatically become available in all continuing blocks.

Custom pytest-fixtures are supported as well. Just define them in your conftest.py file.

Customisation/hooks

Tests can be extended and fine-tuned using pytest’s standard hook system.

Below is an example workflow:

1. Add custom pytest markers to the code blocks (fakepy, aws, openai).

2. Implement pytest hooks in conftest.py to react to those markers.

Add custom pytest markers

Add fakepy marker

The example code below will generate a PDF file with random text using fake.py library. Note, that a fakepy marker is added to the code block.

In the Implement pytest hooks section, you will see what can be done with the markers.

Note

Note the pytestmark directive fakepy marker.

Filename: README.md

<!-- pytestmark: fakepy -->
```python name=test_create_pdf_file
from fake import FAKER

FAKER.pdf_file()
```

Add aws marker

Sample boto3 code to create a bucket on AWS S3.

Note

Note the pytestmark directive aws marker.

Filename: README.md

<!-- pytestmark: aws -->
```python name=test_create_bucket
import boto3

s3 = boto3.client("s3", region_name="us-east-1")
s3.create_bucket(Bucket="my-bucket")
assert "my-bucket" in [b["Name"] for b in s3.list_buckets()["Buckets"]]
```

Add openai marker

Sample openai code to ask LLM to tell a joke. Note, that next to a custom openai marker, xfail marker is used, which allows underlying code to fail, without marking entire test suite as failed.

Note

Note the pytestmark directive xfail and openai markers.

Filename: README.md

<!-- pytestmark: xfail -->
<!-- pytestmark: openai -->
```python name=test_tell_me_a_joke
from openai import OpenAI

client = OpenAI()
completion = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "developer", "content": "You are a famous comedian."},
        {"role": "user", "content": "Tell me a joke."},
    ],
)

assert isinstance(completion.choices[0].message.content, str)
```

---

Implement pytest hooks

In the example below:

• moto is used to mock AWS S3 service for all tests marked as aws.

• Environment variable OPENAI_BASE_URL is set to http://localhost:11434/v1 (assuming you have Ollama running) for all tests marked as openai.

• FILE_REGISTRY.clean_up() is executed at the end of each test marked as fakepy.

Filename: conftest.py

import os
from contextlib import suppress

import pytest

from fake import FILE_REGISTRY
from moto import mock_aws
from pytest_codeblock.constants import CODEBLOCK_MARK

# Modify test item during collection
def pytest_collection_modifyitems(config, items):
    for item in items:
        if item.get_closest_marker(CODEBLOCK_MARK):
            # All `pytest-codeblock` tests are automatically assigned
            # a `codeblock` marker, which can be used for customisation.
            # In the example below we add an additional `documentation`
            # marker to `pytest-codeblock` tests.
            item.add_marker(pytest.mark.documentation)
        if item.get_closest_marker("aws"):
            # Apply `mock_aws` to all tests marked as `aws`
            item.obj = mock_aws(item.obj)


# Setup before test runs
def pytest_runtest_setup(item):
    if item.get_closest_marker("openai"):
        # Send all OpenAI requests to locally running Ollama for all
        # tests marked as `openai`. The tests would x-pass on environments
        # where Ollama is up and running (assuming, you have created an
        # alias for gpt-4o using one of the available models) and would
        # x-fail on environments, where Ollama isn't runnig.
        os.environ.setdefault("OPENAI_API_KEY", "ollama")
        os.environ.setdefault("OPENAI_BASE_URL", "http://localhost:11434/v1")


# Teardown after the test ends
def pytest_runtest_teardown(item, nextitem):
    # Run file clean up on all tests marked as `fakepy`
    if item.get_closest_marker("fakepy"):
        FILE_REGISTRY.clean_up()

View llms.txt version