pytest-codeblock
Test your documentation code blocks.
pytest-codeblock is a Pytest plugin that discovers Python code examples in your reStructuredText and Markdown documentation files and runs them as part of your test suite. This ensures your docs stay correct and up-to-date.
Features
reStructuredText and Markdown support: Automatically find and test code blocks in reStructuredText (
.rst) and Markdown (.md) files. The only requirement here is that your code blocks shall have a name starting withtest_. Async code snippets are supported as well.Grouping: Split a single example across multiple code blocks; the plugin concatenates them into one test.
Pytest markers support: Add existing or custom pytest markers to the code blocks and hook into the tests life-cycle using
conftest.py.Pytest fixtures support: Request existing or custom pytest fixtures for the code blocks.
Prerequisites
Python 3.9+
pytest is the only required dependency
Documentation
Documentation is available on Read the Docs.
For reStructuredText, see a dedicated reStructuredText docs.
For Markdown, see a dedicated Markdown docs.
Both reStructuredText docs and Markdown docs have extensive documentation on pytest markers and corresponding
conftest.pyhooks.For guidelines on contributing check the Contributor guidelines.
Installation
Install with pip:
pip install pytest-codeblock
Or install with uv:
uv pip install pytest-codeblock
Configuration
No configuration needed. All your .rst and .md files shall be picked automatically.
Usage
reStructruredText usage
Any code directive, such as .. code-block:: python, .. code:: python,
or literal blocks with a preceding .. codeblock-name: <name>, will be
collected and executed automatically by pytest.
code-block directive example
Note
Note that :name: value has a test_ prefix.
Filename: README.rst
.. code-block:: python
:name: test_basic_example
import math
result = math.pow(3, 2)
assert result == 9
literalinclude directive example
Note
Note that :name: value has a test_ prefix.
Filename: README.rst
.. literalinclude:: examples/python/basic_example.py
:name: test_li_basic_example
See a dedicated reStructuredText docs for more.
Markdown usage
Any fenced code block with a recognized Python language tag (e.g., python,
py) will be collected and executed automatically by pytest.
Note
Note that name value has a test_ prefix.
Filename: README.md
```python name=test_basic_example
import math
result = math.pow(3, 2)
assert result == 9
```
See a dedicated Markdown docs for more.
Tests
Run the tests with pytest:
pytest
Troubleshooting
If something doesn’t work, try to add this to your pyproject.toml:
Filename: pyproject.toml
[tool.pytest.ini_options]
testpaths = [
"**/*.rst",
"**/*.md",
]
Writing documentation
Keep the following hierarchy.
=====
title
=====
header
======
sub-header
----------
sub-sub-header
~~~~~~~~~~~~~~
sub-sub-sub-header
^^^^^^^^^^^^^^^^^^
sub-sub-sub-sub-header
++++++++++++++++++++++
sub-sub-sub-sub-sub-header
**************************
License
MIT
Support
For security issues contact me at the e-mail given in the Author section.
For overall issues, go to GitHub.
reStructuredText
The following directives are supported:
.. code-block:: python.. code:: python.. codeblock-name: <name>.. literalinclude::
Any code directive, such as .. code-block:: python, .. code:: python,
.. literalinclude:: or literal blocks with a
preceding .. codeblock-name: <name>, will be collected and executed
automatically, if your pytest configuration allows
that.
Usage examples
Standalone code blocks
code-block directive
Note
Note that :name: value has a test_ prefix.
Filename: README.rst
.. code-block:: python
:name: test_basic_example
import math
result = math.pow(3, 2)
assert result == 9
literalinclude directive
Filename: README.rst
.. literalinclude:: examples/python/basic_example.py
:name: test_li_basic_example
codeblock-name directive
You can also use a literal block with a preceding name comment:
Filename: README.rst
.. codeblock-name: test_grouping_example_literal_block
This is a literal block::
y = 5
print(y * 2)
Grouping multiple code-block directives
It’s possible to split one logical test into multiple blocks.
They will be tested under the first :name: specified.
Note the .. continue:: directive.
Note
Note that continue directive of
the test_grouping_example_part_2
and test_grouping_example_part_3 refers to
the test_grouping_example.
Filename: README.rst
.. code-block:: python
:name: test_grouping_example
x = 1
Some intervening text.
.. continue: test_grouping_example
.. code-block:: python
:name: test_grouping_example_part_2
y = x + 1 # Uses x from the first snippet
assert y == 2
Some intervening text.
.. continue: test_grouping_example
.. code-block:: python
:name: test_grouping_example_part_3
print(y) # Uses y from the previous snippet
The above mentioned three snippets will run as a single test.
Async
You can use top-level await in your code blocks. The code will be automatically wrapped in an async function.
Filename: README.rst
.. code-block:: python
:name: test_async_example
import asyncio
result = await asyncio.sleep(0.1, result=42)
assert result == 42
Adding pytest markers to code-block and literalinclude directives
It’s possible to add custom pytest markers to your code-block
or literalinclude directives. That allows adding custom logic and mocking
in your conftest.py.
In the example below, django_db marker is added to the code-block
directive.
Note
Note the pytestmark directive django_db marker.
Filename: README.rst
.. pytestmark: django_db
.. code-block:: python
:name: test_django
from django.contrib.auth.models import User
user = User.objects.first()
In the example below, django_db marker is added to the literalinclude
directive.
Filename: README.rst
.. pytestmark: django_db
.. literalinclude:: examples/python/django_example.py
:name: test_li_django_example
Requesting pytest fixtures for code-block and literalinclude directives
It’s possible to request existing or custom pytest fixtures in code-block
or literalinclude directives. That allows adding custom logic and mocking
in conftest.py.
In the example below, tmp_path fixture is requested for the code-block
directive.
Note
Note the pytestfixture directive tmp_path fixture.
Filename: README.rst
.. pytestfixture: tmp_path
.. code-block:: python
:name: test_path
d = tmp_path / "sub"
d.mkdir() # Create the directory
assert d.is_dir() # Verify it was created and is a directory
In the example below, tmp_path fixture is requested for the literalinclude
directive.
Filename: README.rst
.. pytestfixture: tmp_path
.. literalinclude:: examples/python/tmp_path_example.py
:name: test_li_tmp_path_example
Multiple pytestfixture directives are supported. Add one on each line.
Note
When combining pytestfixture and continue directives together,
request pytest-fixtures only in the first code-block, as they will
automatically become available in all continuing blocks.
Custom pytest-fixtures are supported as well. Just define them in
your conftest.py file.
Customisation/hooks
Tests can be extended and fine-tuned using pytest’s standard hook system.
Below is an example workflow:
Add custom pytest markers to the
code-blockorliteralinclude(fakepy,aws,openai).Implement pytest hooks in
conftest.pyto react to those markers.
Add custom pytest markers
Add fakepy marker
The example code below will generate a PDF file with random text
using fake.py library. Note, that a fakepy marker is added to
the code-block.
In the `Implement pytest hooks`_ section, you will see what can be done with the markers.
Note
Note the pytestmark directive fakepy marker.
Filename: README.rst
.. pytestmark: fakepy
.. code-block:: python
:name: test_create_pdf_file
from fake import FAKER
FAKER.pdf_file()
In the example code below, a fakepy marker is added to
the literalinclude block.
Filename: README.rst
.. pytestmark: fakepy
.. literalinclude:: examples/python/create_pdf_file_example.py
:name: test_li_create_pdf_file
Add aws marker
Sample boto3 code to create a bucket on AWS S3.
Note
Note the pytestmark directive aws marker.
Filename: README.rst
.. pytestmark: aws
.. code-block:: python
:name: test_create_bucket
import boto3
s3 = boto3.client("s3", region_name="us-east-1")
s3.create_bucket(Bucket="my-bucket")
assert "my-bucket" in [b["Name"] for b in s3.list_buckets()["Buckets"]]
Add openai marker
Sample openai code to ask LLM to tell a joke. Note, that next to a
custom openai marker, xfail marker is used, which allows underlying
code to fail, without marking entire test suite as failed.
Note
Note the pytestmark directive xfail and openai markers.
Filename: README.rst
.. pytestmark: xfail
.. pytestmark: openai
.. code-block:: python
:name: test_tell_me_a_joke
from openai import OpenAI
client = OpenAI()
completion = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "developer", "content": "You are a famous comedian."},
{"role": "user", "content": "Tell me a joke."},
],
)
assert isinstance(completion.choices[0].message.content, str)
Implement pytest hooks
In the example below:
moto is used to mock AWS S3 service for all tests marked as
aws.Environment variable
OPENAI_BASE_URLis set tohttp://localhost:11434/v1(assuming you have Ollama running) for all tests marked asopenai.FILE_REGISTRY.clean_up()is executed at the end of each test marked asfakepy.
Filename: conftest.py
import os
from contextlib import suppress
import pytest
from fake import FILE_REGISTRY
from moto import mock_aws
from pytest_codeblock.constants import CODEBLOCK_MARK
# Modify test item during collection
def pytest_collection_modifyitems(config, items):
for item in items:
if item.get_closest_marker(CODEBLOCK_MARK):
# All `pytest-codeblock` tests are automatically assigned
# a `codeblock` marker, which can be used for customisation.
# In the example below we add an additional `documentation`
# marker to `pytest-codeblock` tests.
item.add_marker(pytest.mark.documentation)
if item.get_closest_marker("aws"):
# Apply `mock_aws` to all tests marked as `aws`
item.obj = mock_aws(item.obj)
# Setup before test runs
def pytest_runtest_setup(item):
if item.get_closest_marker("openai"):
# Send all OpenAI requests to locally running Ollama for all
# tests marked as `openai`. The tests would x-pass on environments
# where Ollama is up and running (assuming, you have created an
# alias for gpt-4o using one of the available models) and would
# x-fail on environments, where Ollama isn't runnig.
os.environ.setdefault("OPENAI_API_KEY", "ollama")
os.environ.setdefault("OPENAI_BASE_URL", "http://localhost:11434/v1")
# Teardown after the test ends
def pytest_runtest_teardown(item, nextitem):
# Run file clean up on all tests marked as `fakepy`
if item.get_closest_marker("fakepy"):
FILE_REGISTRY.clean_up()
Markdown
Usage examples
Any fenced code block with a recognized Python language tag (e.g., python,
py) will be collected and executed automatically, if
your pytest configuration allows that.
Standalone code blocks
Note
Note that name value has a test_ prefix.
Filename: README.md
```python name=test_basic_example
import math
result = math.pow(3, 2)
assert result == 9
```
Grouping multiple code blocks
It’s possible to split one logical test into multiple blocks.
They will be tested under the first name specified.
Note the <!-- continue: test_group_new_syntax --> directive.
Note
Note that continue directive of
the test_grouping_example_part_2
and test_grouping_example_part_3 refers to
the test_grouping_example.
Filename: README.md
```python name=test_grouping_example
x = 1
```
Some intervening text.
<!-- continue: test_grouping_example -->
```python name=test_grouping_example_part_2
y = x + 1 # Uses x from the first snippet
assert y == 2
```
Some intervening text.
<!-- continue: test_grouping_example -->
```python name=test_grouping_example_part_3
print(y) # Uses y from the previous snippet
```
The above mentioned three snippets will run as a single test.
Async
You can use top-level await in your code blocks. The code will be automatically wrapped in an async function.
Filename: README.md
```python name=test_async_example
import asyncio
result = await asyncio.sleep(0.1, result=42)
assert result == 42
```
Adding pytest markers to code blocks
It’s possible to add custom pytest markers to your code blocks. That allows
adding custom logic and mocking in your conftest.py.
In the example below, django_db marker is added to the code block.
Note
Note the pytestmark directive django_db marker.
Filename: README.md
<!-- pytestmark: django_db -->
```python name=test_django
from django.contrib.auth.models import User
user = User.objects.first()
```
Requesting pytest fixtures for code blocks
It’s possible to request existing or custom pytest fixtures for code blocks.
That allows adding custom logic and mocking in conftest.py.
In the example below, tmp_path fixture is requested for the code block.
Note
Note the pytestfixture directive tmp_path fixture.
Filename: README.md
<!-- pytestfixture: tmp_path -->
```python name=test_path
d = tmp_path / "sub"
d.mkdir() # Create the directory
assert d.is_dir() # Verify it was created and is a directory
```
Multiple pytestfixture directives are supported. Add one on each line.
Note
When combining pytestfixture and continue directives together,
request pytest-fixtures only in the first code-block, as they will
automatically become available in all continuing blocks.
Custom pytest-fixtures are supported as well. Just define them in
your conftest.py file.
Customisation/hooks
Tests can be extended and fine-tuned using pytest’s standard hook system.
Below is an example workflow:
Add custom pytest markers to the code blocks (
fakepy,aws,openai).Implement pytest hooks in
conftest.pyto react to those markers.
Add custom pytest markers
Add fakepy marker
The example code below will generate a PDF file with random text
using fake.py library. Note, that a fakepy marker is added to
the code block.
In the `Implement pytest hooks`_ section, you will see what can be done with the markers.
Note
Note the pytestmark directive fakepy marker.
Filename: README.md
<!-- pytestmark: fakepy -->
```python name=test_create_pdf_file
from fake import FAKER
FAKER.pdf_file()
```
Add aws marker
Sample boto3 code to create a bucket on AWS S3.
Note
Note the pytestmark directive aws marker.
Filename: README.md
<!-- pytestmark: aws -->
```python name=test_create_bucket
import boto3
s3 = boto3.client("s3", region_name="us-east-1")
s3.create_bucket(Bucket="my-bucket")
assert "my-bucket" in [b["Name"] for b in s3.list_buckets()["Buckets"]]
```
Add openai marker
Sample openai code to ask LLM to tell a joke. Note, that next to a
custom openai marker, xfail marker is used, which allows underlying
code to fail, without marking entire test suite as failed.
Note
Note the pytestmark directive xfail and openai markers.
Filename: README.md
<!-- pytestmark: xfail -->
<!-- pytestmark: openai -->
```python name=test_tell_me_a_joke
from openai import OpenAI
client = OpenAI()
completion = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "developer", "content": "You are a famous comedian."},
{"role": "user", "content": "Tell me a joke."},
],
)
assert isinstance(completion.choices[0].message.content, str)
```
Implement pytest hooks
In the example below:
moto is used to mock AWS S3 service for all tests marked as
aws.Environment variable
OPENAI_BASE_URLis set tohttp://localhost:11434/v1(assuming you have Ollama running) for all tests marked asopenai.FILE_REGISTRY.clean_up()is executed at the end of each test marked asfakepy.
Filename: conftest.py
import os
from contextlib import suppress
import pytest
from fake import FILE_REGISTRY
from moto import mock_aws
from pytest_codeblock.constants import CODEBLOCK_MARK
# Modify test item during collection
def pytest_collection_modifyitems(config, items):
for item in items:
if item.get_closest_marker(CODEBLOCK_MARK):
# All `pytest-codeblock` tests are automatically assigned
# a `codeblock` marker, which can be used for customisation.
# In the example below we add an additional `documentation`
# marker to `pytest-codeblock` tests.
item.add_marker(pytest.mark.documentation)
if item.get_closest_marker("aws"):
# Apply `mock_aws` to all tests marked as `aws`
item.obj = mock_aws(item.obj)
# Setup before test runs
def pytest_runtest_setup(item):
if item.get_closest_marker("openai"):
# Send all OpenAI requests to locally running Ollama for all
# tests marked as `openai`. The tests would x-pass on environments
# where Ollama is up and running (assuming, you have created an
# alias for gpt-4o using one of the available models) and would
# x-fail on environments, where Ollama isn't runnig.
os.environ.setdefault("OPENAI_API_KEY", "ollama")
os.environ.setdefault("OPENAI_BASE_URL", "http://localhost:11434/v1")
# Teardown after the test ends
def pytest_runtest_teardown(item, nextitem):
# Run file clean up on all tests marked as `fakepy`
if item.get_closest_marker("fakepy"):
FILE_REGISTRY.clean_up()
Package
Indices and tables
Project source-tree
Below is the layout of our project (to 10 levels), followed by the contents of each key file.
Project directory layout
pytest-codeblock/
├── docs
│ ├── _static
│ ├── _templates
│ ├── _implement_pytest_hooks.rst
│ ├── changelog.rst
│ ├── code_of_conduct.rst
│ ├── conf.py
│ ├── conf.py.distrib
│ ├── contributor_guidelines.rst
│ ├── documentation.rst
│ ├── index.rst
│ ├── index.rst.distrib
│ ├── llms.rst
│ ├── make.bat
│ ├── Makefile
│ ├── markdown.rst
│ ├── package.rst
│ ├── requirements.txt
│ ├── restructured_text.rst
│ ├── security.rst
│ └── source_tree.rst
├── examples
│ ├── md_example
│ │ ├── customisation.md
│ │ └── README.md
│ ├── python
│ │ ├── __init__.py
│ │ ├── basic_example.py
│ │ ├── create_bucket_example.py
│ │ ├── create_pdf_file_example.py
│ │ ├── django_example.py
│ │ └── tell_me_a_joke_example.py
│ └── rst_example
│ ├── __pycache__
│ ├── __init__.py
│ ├── customisation.rst
│ ├── django_settings.py
│ └── README.rst
├── scripts
│ └── generate_project_source_tree.py
├── src
│ └── pytest_codeblock
│ ├── __pycache__
│ ├── tests
│ │ ├── __pycache__
│ │ ├── __init__.py
│ │ ├── test_pytest_codeblock.py
│ │ └── tests.rst
│ ├── __init__.py
│ ├── collector.py
│ ├── constants.py
│ ├── md.py
│ ├── rst.py
docs/_implement_pytest_hooks.rst
docs/_implement_pytest_hooks.rst
In the example below:
- `moto`_ is used to mock AWS S3 service for all tests marked as ``aws``.
- Environment variable ``OPENAI_BASE_URL`` is set
to ``http://localhost:11434/v1`` (assuming you have `Ollama`_ running) for
all tests marked as ``openai``.
- ``FILE_REGISTRY.clean_up()`` is executed at the end of each test marked
as ``fakepy``.
*Filename: conftest.py*
.. code-block:: python
import os
from contextlib import suppress
import pytest
from fake import FILE_REGISTRY
from moto import mock_aws
from pytest_codeblock.constants import CODEBLOCK_MARK
# Modify test item during collection
def pytest_collection_modifyitems(config, items):
for item in items:
if item.get_closest_marker(CODEBLOCK_MARK):
# All `pytest-codeblock` tests are automatically assigned
# a `codeblock` marker, which can be used for customisation.
# In the example below we add an additional `documentation`
# marker to `pytest-codeblock` tests.
item.add_marker(pytest.mark.documentation)
if item.get_closest_marker("aws"):
# Apply `mock_aws` to all tests marked as `aws`
item.obj = mock_aws(item.obj)
# Setup before test runs
def pytest_runtest_setup(item):
if item.get_closest_marker("openai"):
# Send all OpenAI requests to locally running Ollama for all
# tests marked as `openai`. The tests would x-pass on environments
# where Ollama is up and running (assuming, you have created an
# alias for gpt-4o using one of the available models) and would
# x-fail on environments, where Ollama isn't runnig.
os.environ.setdefault("OPENAI_API_KEY", "ollama")
os.environ.setdefault("OPENAI_BASE_URL", "http://localhost:11434/v1")
# Teardown after the test ends
def pytest_runtest_teardown(item, nextitem):
# Run file clean up on all tests marked as `fakepy`
if item.get_closest_marker("fakepy"):
FILE_REGISTRY.clean_up()
docs/changelog.rst
docs/changelog.rst
.. include:: ../CHANGELOG.rst
docs/code_of_conduct.rst
docs/code_of_conduct.rst
.. include:: ../CODE_OF_CONDUCT.rst
docs/conf.py
docs/conf.py
# Configuration file for the Sphinx documentation builder.
#
# For the full list of built-in configuration values, see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html
import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join("..", "src")))
# -- Project information -----------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
try:
import pytest_codeblock
version = pytest_codeblock.__version__
project = pytest_codeblock.__title__
copyright = pytest_codeblock.__copyright__
author = pytest_codeblock.__author__
except ImportError:
version = "0.1"
project = "pytest-codeblock"
copyright = "2025, Artur Barseghyan <artur.barseghyan@gmail.com>"
author = "Artur Barseghyan <artur.barseghyan@gmail.com>"
# -- General configuration ---------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
extensions = [
"sphinx.ext.autodoc",
"sphinx.ext.viewcode",
"sphinx.ext.todo",
"sphinx_no_pragma",
"sphinx_llms_txt_link",
]
templates_path = ["_templates"]
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
language = "en"
release = version
# The suffix of source filenames.
source_suffix = {
".rst": "restructuredtext",
}
pygments_style = "sphinx"
# -- Options for HTML output -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
html_theme = "sphinx_rtd_theme"
html_static_path = ["_static"]
# html_extra_path = ["examples"]
prismjs_base = "//cdnjs.cloudflare.com/ajax/libs/prism/1.29.0"
html_css_files = [
f"{prismjs_base}/themes/prism.min.css",
f"{prismjs_base}/plugins/toolbar/prism-toolbar.min.css",
# "https://cdn.jsdelivr.net/gh/barseghyanartur/jsphinx@1.3.4/src/css/sphinx_rtd_theme.css",
"https://cdn.jsdelivr.net/gh/barseghyanartur/jsphinx/src/css/sphinx_rtd_theme.css",
]
html_js_files = [
f"{prismjs_base}/prism.min.js",
f"{prismjs_base}/plugins/autoloader/prism-autoloader.min.js",
f"{prismjs_base}/plugins/toolbar/prism-toolbar.min.js",
f"{prismjs_base}/plugins/copy-to-clipboard/prism-copy-to-clipboard.min.js",
# "https://cdn.jsdelivr.net/gh/barseghyanartur/jsphinx@1.3.4/src/js/download_adapter.js",
"https://cdn.jsdelivr.net/gh/barseghyanartur/jsphinx/src/js/download_adapter.js",
]
# -- Options for todo extension ----------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/extensions/todo.html#configuration
todo_include_todos = True
# -- Options for Epub output ----------------------------------------------
epub_title = project
epub_author = author
epub_publisher = "GitHub"
epub_copyright = copyright
epub_identifier = "https://github.com/barseghyanartur/pytest-codeblock" # URL or ISBN
epub_scheme = "URL" # or "ISBN"
epub_uid = "https://github.com/barseghyanartur/pytest-codeblock"
docs/contributor_guidelines.rst
docs/contributor_guidelines.rst
.. include:: ../CONTRIBUTING.rst
docs/documentation.rst
docs/documentation.rst
Project documentation
=====================
Contents:
.. contents:: Table of Contents
.. toctree::
:maxdepth: 2
index
restructured_text
markdown
security
contributor_guidelines
code_of_conduct
changelog
package
docs/index.rst
docs/index.rst
.. include:: ../README.rst
.. include:: documentation.rst
docs/llms.rst
docs/llms.rst
.. include:: ../README.rst
----
.. include:: restructured_text.rst
----
.. include:: markdown.rst
----
.. include:: package.rst
----
.. include:: source_tree.rst
docs/markdown.rst
docs/markdown.rst
Markdown
========
.. External references
.. _Markdown: https://daringfireball.net/projects/markdown/
.. _pytest: https://docs.pytest.org
.. _Django: https://www.djangoproject.com
.. _pip: https://pypi.org/project/pip/
.. _uv: https://pypi.org/project/uv/
.. _fake.py: https://github.com/barseghyanartur/fake.py
.. _boto3: https://github.com/boto/boto3
.. _moto: https://github.com/getmoto/moto
.. _openai: https://github.com/openai/openai-python
.. _Ollama: https://github.com/ollama/ollama
Usage examples
--------------
Any fenced code block with a recognized Python language tag (e.g., ``python``,
``py``) will be collected and executed automatically, if
your `pytest`_ :ref:`configuration <configuration>` allows that.
Standalone code blocks
~~~~~~~~~~~~~~~~~~~~~~
.. note:: Note that ``name`` value has a ``test_`` prefix.
*Filename: README.md*
.. code-block:: markdown
```python name=test_basic_example
import math
result = math.pow(3, 2)
assert res ult == 9
```
----
Grouping multiple code blocks
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It's possible to split one logical test into multiple blocks.
They will be tested under the first ``name`` specified.
Note the ``<!-- continue: test_group_new_syntax -->`` directive.
.. note:: Note that ``continue`` directive of
the ``test_grouping_example_part_2``
and ``test_grouping_example_part_3`` refers to
the ``test_grouping_example``.
*Filename: README.md*
.. code-block:: markdown
```python name=test_grouping_example
x = 1
```
Some intervening text.
<!-- continue: test_grouping_example -->
```python name=test_grouping_example_part_2
y = x + 1 # Uses x from the first snippet
assert y == 2
```
Some intervening text.
<!-- continue: test_grouping_example -->
```python name=test_grouping_example_part_3
print(y) # Uses y from the previous snippet
```
The above mentioned three snippets will run as a single test.
----
Async
~~~~~
You can use `top-level await` in your code blocks. The code will be
automatically wrapped in an async function.
*Filename: README.md*
.. code-block:: markdown
```python name=test_async_example
import asyncio
result = await asyncio.sleep(0.1, result=42)
assert result == 42
```
----
Adding pytest markers to code blocks
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It's possible to add custom pytest markers to your code blocks. That allows
adding custom logic and mocking in your ``conftest.py``.
In the example below, ``django_db`` marker is added to the code block.
.. note:: Note the ``pytestmark`` directive ``django_db`` marker.
*Filename: README.md*
.. code-block:: markdown
<!-- pytestmark: django_db -->
```python name=test_django
from django.contrib.auth.models import User
user = User.objects.first()
```
Requesting pytest fixtures for code blocks
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It's possible to request existing or custom pytest fixtures for code blocks.
That allows adding custom logic and mocking in ``conftest.py``.
In the example below, ``tmp_path`` fixture is requested for the code block.
.. note:: Note the ``pytestfixture`` directive ``tmp_path`` fixture.
*Filename: README.md*
.. code-block:: markdown
<!-- pytestfixture: tmp_path -->
```python name=test_path
d = tmp_path / "sub"
d.mkdir() # Create the directory
assert d.is_dir() # Ver ify it was created and is a direct ory
```
----
Multiple ``pytestfixture`` directives are supported. Add one on each line.
.. note::
When combining ``pytestfixture`` and ``continue`` directives together,
request pytest-fixtures only in the first ``code-block``, as they will
automatically become available in all continuing blocks.
Custom pytest-fixtures are supported as well. Just define them in
your ``conftest.py`` file.
Customisation/hooks
-------------------
Tests can be extended and fine-tuned using `pytest`_'s standard hook system.
Below is an example workflow:
1. **Add custom pytest markers** to the code
blocks (``fakepy``, ``aws``, ``openai``).
2. **Implement pytest hooks** in ``conftest.py`` to react to those markers.
Add custom pytest markers
~~~~~~~~~~~~~~~~~~~~~~~~~
Add ``fakepy`` marker
^^^^^^^^^^^^^^^^^^^^^
The example code below will generate a PDF file with random text
using `fake.py`_ library. Note, that a ``fakepy`` marker is added to
the code block.
In the `Implement pytest hooks`_ section, you will see what can be done
with the markers.
.. note:: Note the ``pytestmark`` directive ``fakepy`` marker.
*Filename: README.md*
.. code-block:: markdown
<!-- pytestmark: fakepy -->
```python name=test_create_pdf_file
from fake import FAKER
FAKER.pdf_file()
```
Add ``aws`` marker
^^^^^^^^^^^^^^^^^^
Sample `boto3`_ code to create a bucket on AWS S3.
.. note:: Note the ``pytestmark`` directive ``aws`` marker.
*Filename: README.md*
.. code-block:: markdown
<!-- pytestmark: aws -->
```python name=test_create_bucket
import boto3
s3 = boto3.client("s3", region_name="us-east-1")
s3.create_ bucket(Bucket="my-bucket")
assert "my-bucket" in [b["Name"] for b in s3.list_buckets() ["Buckets"]]
```
Add ``openai`` marker
^^^^^^^^^^^^^^^^^^^^^
Sample `openai`_ code to ask LLM to tell a joke. Note, that next to a
custom ``openai`` marker, ``xfail`` marker is used, which allows underlying
code to fail, without marking entire test suite as failed.
.. note:: Note the ``pytestmark`` directive ``xfail`` and ``openai`` markers.
*Filename: README.md*
.. code-block:: markdown
<!-- pytestmark: xfail -->
<!-- pytestmark: openai -->
```python name=test_tell_me_a_joke
from openai import OpenAI
client = OpenAI()
completion = client.chat.completions.create(
model="gpt-4o",
mes sages=[
{" role": "developer", "content": "You are a fam ous comedian."},
{"role": " user", "content": "Tell me a joke."},
],
)
assert isinstance(comple tion.choices[0].message.content, str)
```
----
Implement pytest hooks
~~~~~~~~~~~~~~~~~~~~~~
.. include:: _implement_pytest_hooks.rst
docs/package.rst
docs/package.rst
Package
=======
.. toctree::
:maxdepth: 20
fake
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
docs/restructured_text.rst
docs/restructured_text.rst
reStructuredText
================
.. External references
.. _reStructuredText: https://docutils.sourceforge.io/rst.html
.. _pytest: https://docs.pytest.org
.. _Django: https://www.djangoproject.com
.. _pip: https://pypi.org/project/pip/
.. _uv: https://pypi.org/project/uv/
.. _fake.py: https://github.com/barseghyanartur/fake.py
.. _boto3: https://github.com/boto/boto3
.. _moto: https://github.com/getmoto/moto
.. _openai: https://github.com/openai/openai-python
.. _Ollama: https://github.com/ollama/ollama
The following directives are supported:
- ``.. code-block:: python``
- ``.. code:: python``
- ``.. codeblock-name: <name>``
- ``.. literalinclude::``
Any code directive, such as ``.. code-block:: python``, ``.. code:: python``,
``.. literalinclude::`` or literal blocks with a
preceding ``.. codeblock-name: <name>``, will be collected and executed
automatically, if your `pytest`_ :ref:`configuration <configuration>` allows
that.
Usage examples
--------------
Standalone code blocks
~~~~~~~~~~~~~~~~~~~~~~
``code-block`` directive
^^^^^^^^^^^^^^^^^^^^^^^^
.. note:: Note that ``:name:`` value has a ``test_`` prefix.
*Filename: README.rst*
.. code-block:: rst
.. code-block:: python
:name: test_basic_example
import math
result = math.pow(3, 2)
assert result == 9
----
``literalinclude`` directive
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
*Filename: README.rst*
.. code-block:: rst
.. literalinclude:: examples/python/basic_example.py
:name: test_li_basic_example
----
``codeblock-name`` directive
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
You can also use a literal block with a preceding name comment:
*Filename: README.rst*
.. code-block:: rst
.. codeblock-name: test_grouping_example_literal_block
This is a literal block::
y = 5
print(y * 2)
----
Grouping multiple ``code-block`` directives
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It's possible to split one logical test into multiple blocks.
They will be tested under the first ``:name:`` specified.
Note the ``.. continue::`` directive.
.. note:: Note that ``continue`` directive of
the ``test_grouping_example_part_2``
and ``test_grouping_example_part_3`` refers to
the ``test_grouping_example``.
*Filename: README.rst*
.. code-block:: rst
.. code-block:: python
:name: test_grouping_example
x = 1
Some intervening text.
.. continue: test_grouping_example
.. code-block:: python
:name: test_grouping_example_part_2
y = x + 1 # Uses x from the first snippet
assert y == 2
Some intervening text.
.. continue: test_grouping_example
.. code-block:: python
:name: test_grouping_example_part_3
print(y) # Uses y from the previous snippet
The above mentioned three snippets will run as a single test.
----
Async
~~~~~
You can use `top-level await` in your code blocks. The code will be
automatically wrapped in an async function.
*Filename: README.rst*
.. code-block:: rst
.. code-block:: python
:name: test_async_example
import asyncio
result = await asyncio.sleep(0.1, result=42)
assert result == 42
----
Adding pytest markers to ``code-block`` and ``literalinclude`` directives
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It's possible to add custom pytest markers to your ``code-block``
or ``literalinclude`` directives. That allows adding custom logic and mocking
in your ``conftest.py``.
In the example below, ``django_db`` marker is added to the ``code-block``
directive.
.. note:: Note the ``pytestmark`` directive ``django_db`` marker.
*Filename: README.rst*
.. code-block:: rst
.. pytestmark: django_db
.. code-block:: python
:name: test_django
from django.contrib.auth.models import User
user = User.objects.first()
----
In the example below, ``django_db`` marker is added to the ``literalinclude``
directive.
*Filename: README.rst*
.. code-block:: rst
.. pytestmark: django_db
.. literalinclude:: examples/python/django_example.py
:name: test_li_django_example
----
Requesting pytest fixtures for ``code-block`` and ``literalinclude`` directives
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It's possible to request existing or custom pytest fixtures in ``code-block``
or ``literalinclude`` directives. That allows adding custom logic and mocking
in ``conftest.py``.
In the example below, ``tmp_path`` fixture is requested for the ``code-block``
directive.
.. note:: Note the ``pytestfixture`` directive ``tmp_path`` fixture.
*Filename: README.rst*
.. code-block:: rst
.. pytestfixture: tmp_path
.. code-block:: python
:name: test_path
d = tmp_path / "sub"
d.mkdir() # Create the directory
assert d.is_dir() # Verify it was created and is a directory
----
In the example below, ``tmp_path`` fixture is requested for the ``literalinclude``
directive.
*Filename: README.rst*
.. code-block:: rst
.. pytestfixture: tmp_path
.. literalinclude:: examples/python/tmp_path_example.py
:name: test_li_tmp_path_example
----
Multiple ``pytestfixture`` directives are supported. Add one on each line.
.. note::
When combining ``pytestfixture`` and ``continue`` directives together,
request pytest-fixtures only in the first ``code-block``, as they will
automatically become available in all continuing blocks.
Custom pytest-fixtures are supported as well. Just define them in
your ``conftest.py`` file.
Customisation/hooks
-------------------
Tests can be extended and fine-tuned using `pytest`_'s standard hook system.
Below is an example workflow:
1. **Add custom pytest markers** to the ``code-block``
or ``literalinclude`` (``fakepy``, ``aws``, ``openai``).
2. **Implement pytest hooks** in ``conftest.py`` to react to those markers.
Add custom pytest markers
~~~~~~~~~~~~~~~~~~~~~~~~~
Add ``fakepy`` marker
^^^^^^^^^^^^^^^^^^^^^
The example code below will generate a PDF file with random text
using `fake.py`_ library. Note, that a ``fakepy`` marker is added to
the ``code-block``.
In the `Implement pytest hooks`_ section, you will see what can be done
with the markers.
.. note:: Note the ``pytestmark`` directive ``fakepy`` marker.
*Filename: README.rst*
.. code-block:: rst
.. pytestmark: fakepy
.. code-block:: python
:name: test_create_pdf_file
from fake import FAKER
FAKER.pdf_file()
----
In the example code below, a ``fakepy`` marker is added to
the ``literalinclude`` block.
*Filename: README.rst*
.. code-block:: rst
.. pytestmark: fakepy
.. literalinclude:: examples/python/create_pdf_file_example.py
:name: test_li_create_pdf_file
----
Add ``aws`` marker
^^^^^^^^^^^^^^^^^^
Sample `boto3`_ code to create a bucket on AWS S3.
.. note:: Note the ``pytestmark`` directive ``aws`` marker.
*Filename: README.rst*
.. code-block:: rst
.. pytestmark: aws
.. code-block:: python
:name: test_create_bucket
import boto3
s3 = boto3.client("s3", region_name="us-east-1")
s3.create_bucket(Bucket="my-bucket")
assert "my-bucket" in [b["Name"] for b in s3.list_buckets()["Buckets"]]
----
Add ``openai`` marker
^^^^^^^^^^^^^^^^^^^^^
Sample `openai`_ code to ask LLM to tell a joke. Note, that next to a
custom ``openai`` marker, ``xfail`` marker is used, which allows underlying
code to fail, without marking entire test suite as failed.
.. note:: Note the ``pytestmark`` directive ``xfail`` and ``openai`` markers.
*Filename: README.rst*
.. code-block:: rst
.. pytestmark: xfail
.. pytestmark: openai
.. code-block:: python
:name: test_tell_me_a_joke
from openai import OpenAI
client = OpenAI()
completion = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "developer", "content": "You are a famous comedian."},
{"role": "user", "content": "Tell me a joke."},
],
)
assert isinstance(completion.choices[0].message.content, str)
----
Implement pytest hooks
~~~~~~~~~~~~~~~~~~~~~~
.. include:: _implement_pytest_hooks.rst
docs/security.rst
docs/security.rst
.. include:: ../SECURITY.rst
docs/source_tree.rst
docs/source_tree.rst
Project source-tree
===================
Below is the layout of our project (to 10 levels), followed by
the contents of each key file.
.. code-block:: bash
:caption: Project directory layout
pytest-codeblock/
├── docs
│ ├── _static
│ ├── _templates
│ ├── _implement_pytest_hooks.rst
│ ├── changelog.rst
│ ├── code_of_conduct.rst
│ ├── conf.py
│ ├── conf.py.distrib
│ ├── contributor_guidelines.rst
│ ├── documentation.rst
│ ├── index.rst
│ ├── index.rst.distrib
│ ├── llms.rst
│ ├── make.bat
│ ├── Makefile
│ ├── markdown.rst
│ ├── package.rst
│ ├── requirements.txt
│ ├── restructured_text.rst
│ ├── security.rst
│ └── source_tree.rst
├── examples
│ ├── md_example
│ │ ├── customisation.md
│ │ └── README.md
│ ├── python
│ │ ├── __init__.py
│ │ ├── basic_example.py
│ │ ├── create_bucket_example.py
│ │ ├── create_pdf_file_example.py
│ │ ├── django_example.py
│ │ └── tell_me_a_joke_example.py
│ └── rst_example
│ ├── __pycache__
│ ├── __init__.py
│ ├── customisation.rst
│ ├── django_settings.py
│ └── README.rst
├── scripts
│ └── generate_project_source_tree.py
├── src
│ └── pytest_codeblock
│ ├── __pycache__
│ ├── tests
│ │ ├── __pycache__
│ │ ├── __init__.py
│ │ ├── test_pytest_codeblock.py
│ │ └── tests.rst
│ ├── __init__.py
│ ├── collector.py
│ ├── constants.py
│ ├── md.py
│ ├── rst.py
docs/_implement_pytest_hooks.rst
--------------------------------
.. literalinclude:: _implement_pytest_hooks.rst
:language: rst
:caption: docs/_implement_pytest_hooks.rst
docs/changelog.rst
------------------
.. literalinclude:: changelog.rst
:language: rst
:caption: docs/changelog.rst
docs/code_of_conduct.rst
------------------------
.. literalinclude:: code_of_conduct.rst
:language: rst
:caption: docs/code_of_conduct.rst
docs/conf.py
------------
.. literalinclude:: conf.py
:language: python
:caption: docs/conf.py
docs/contributor_guidelines.rst
-------------------------------
.. literalinclude:: contributor_guidelines.rst
:language: rst
:caption: docs/contributor_guidelines.rst
docs/documentation.rst
----------------------
.. literalinclude:: documentation.rst
:language: rst
:caption: docs/documentation.rst
docs/index.rst
--------------
.. literalinclude:: index.rst
:language: rst
:caption: docs/index.rst
docs/llms.rst
-------------
.. literalinclude:: llms.rst
:language: rst
:caption: docs/llms.rst
docs/markdown.rst
-----------------
.. literalinclude:: markdown.rst
:language: rst
:caption: docs/markdown.rst
docs/package.rst
----------------
.. literalinclude:: package.rst
:language: rst
:caption: docs/package.rst
docs/restructured_text.rst
--------------------------
.. literalinclude:: restructured_text.rst
:language: rst
:caption: docs/restructured_text.rst
docs/security.rst
-----------------
.. literalinclude:: security.rst
:language: rst
:caption: docs/security.rst
docs/source_tree.rst
--------------------
.. literalinclude:: source_tree.rst
:language: rst
:caption: docs/source_tree.rst
examples/md_example/README.md
-----------------------------
.. literalinclude:: ../examples/md_example/README.md
:language: markdown
:caption: examples/md_example/README.md
examples/md_example/customisation.md
------------------------------------
.. literalinclude:: ../examples/md_example/customisation.md
:language: markdown
:caption: examples/md_example/customisation.md
examples/python/__init__.py
---------------------------
.. literalinclude:: ../examples/python/__init__.py
:language: python
:caption: examples/python/__init__.py
examples/python/basic_example.py
--------------------------------
.. literalinclude:: ../examples/python/basic_example.py
:language: python
:caption: examples/python/basic_example.py
examples/python/create_bucket_example.py
----------------------------------------
.. literalinclude:: ../examples/python/create_bucket_example.py
:language: python
:caption: examples/python/create_bucket_example.py
examples/python/create_pdf_file_example.py
------------------------------------------
.. literalinclude:: ../examples/python/create_pdf_file_example.py
:language: python
:caption: examples/python/create_pdf_file_example.py
examples/python/django_example.py
---------------------------------
.. literalinclude:: ../examples/python/django_example.py
:language: python
:caption: examples/python/django_example.py
examples/python/tell_me_a_joke_example.py
-----------------------------------------
.. literalinclude:: ../examples/python/tell_me_a_joke_example.py
:language: python
:caption: examples/python/tell_me_a_joke_example.py
examples/rst_example/README.rst
-------------------------------
.. literalinclude:: ../examples/rst_example/README.rst
:language: rst
:caption: examples/rst_example/README.rst
examples/rst_example/__init__.py
--------------------------------
.. literalinclude:: ../examples/rst_example/__init__.py
:language: python
:caption: examples/rst_example/__init__.py
examples/rst_example/customisation.rst
--------------------------------------
.. literalinclude:: ../examples/rst_example/customisation.rst
:language: rst
:caption: examples/rst_example/customisation.rst
examples/rst_example/django_settings.py
---------------------------------------
.. literalinclude:: ../examples/rst_example/django_settings.py
:language: python
:caption: examples/rst_example/django_settings.py
scripts/generate_project_source_tree.py
---------------------------------------
.. literalinclude:: ../scripts/generate_project_source_tree.py
:language: python
:caption: scripts/generate_project_source_tree.py
src/pytest_codeblock/__init__.py
--------------------------------
.. literalinclude:: ../src/pytest_codeblock/__init__.py
:language: python
:caption: src/pytest_codeblock/__init__.py
src/pytest_codeblock/collector.py
---------------------------------
.. literalinclude:: ../src/pytest_codeblock/collector.py
:language: python
:caption: src/pytest_codeblock/collector.py
src/pytest_codeblock/constants.py
---------------------------------
.. literalinclude:: ../src/pytest_codeblock/constants.py
:language: python
:caption: src/pytest_codeblock/constants.py
src/pytest_codeblock/md.py
--------------------------
.. literalinclude:: ../src/pytest_codeblock/md.py
:language: python
:caption: src/pytest_codeblock/md.py
src/pytest_codeblock/rst.py
---------------------------
.. literalinclude:: ../src/pytest_codeblock/rst.py
:language: python
:caption: src/pytest_codeblock/rst.py
src/pytest_codeblock/tests/__init__.py
--------------------------------------
.. literalinclude:: ../src/pytest_codeblock/tests/__init__.py
:language: python
:caption: src/pytest_codeblock/tests/__init__.py
src/pytest_codeblock/tests/test_pytest_codeblock.py
---------------------------------------------------
.. literalinclude:: ../src/pytest_codeblock/tests/test_pytest_codeblock.py
:language: python
:caption: src/pytest_codeblock/tests/test_pytest_codeblock.py
src/pytest_codeblock/tests/tests.rst
------------------------------------
.. literalinclude:: ../src/pytest_codeblock/tests/tests.rst
:language: rst
:caption: src/pytest_codeblock/tests/tests.rst
examples/md_example/README.md
examples/md_example/README.md
# Markdown example project
This is a minimal example showing how pytest-codeblock will discover and run
only Python snippets whose `name` starts with test_.
## Simple assertion
```python name=test_simple_assert
# A trivial test that always passes
assert 2 + 2 == 4
```
## Multi-part example
It's possible to split one logical test into multiple blocks. All of them
share the same ``name``:
```python name=test_compute_square
import math
```
Some intervening text.
```python name=test_compute_square
result = math.pow(3, 2)
assert result == 9
```
Some intervening text.
```python name=test_compute_square
print(result)
```
## Ignored snippets
Blocks without a `name` or without the `test_` prefix are **not** collected:
```python name=example_not_test
# Name does not start with `test_`, so this is ignored
```
## Non-Python blocks are also ignored
```bash name=test_should_be_ignored
echo "Not Python → skipped"
```
## Custom pytest marks
<!-- pytestmark: django_db -->
```python name=test_django
from django.contrib.auth.models import User
user = User.objects.first()
```
examples/md_example/customisation.md
examples/md_example/customisation.md
# Customisation
Customisation examples.
## External references
- [openai](https://github.com/openai/openai-python)
- [moto](https://docs.getmoto.org)
- [fake.py](https://github.com/barseghyanartur/fake.py)
## `fake.py` example
<!-- pytestmark: fakepy -->
```python name=test_create_pdf_file
from fake import FAKER
FAKER.pdf_file()
```
## `moto` example
<!-- pytestmark: aws -->
```python name=test_create_bucket
import boto3
s3 = boto3.client("s3", region_name="us-east-1")
s3.create_bucket(Bucket="my-bucket")
assert "my-bucket" in [b["Name"] for b in s3.list_buckets()["Buckets"]]
```
## `openai` example
<!-- pytestmark: xfail -->
<!-- pytestmark: openai -->
```python name=test_tell_me_a_joke
from openai import OpenAI
client = OpenAI()
completion = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "developer", "content": "You are a famous comedian."},
{"role": "user", "content": "Tell me a joke."},
],
)
assert isinstance(completion.choices[0].message.content, str)
```
examples/python/__init__.py
examples/python/__init__.py
examples/python/basic_example.py
examples/python/basic_example.py
import math
result = math.pow(3, 2)
assert result == 9
examples/python/create_bucket_example.py
examples/python/create_bucket_example.py
import boto3
s3 = boto3.client("s3", region_name="us-east-1")
s3.create_bucket(Bucket="my-bucket")
assert "my-bucket" in [b["Name"] for b in s3.list_buckets()["Buckets"]]
examples/python/create_pdf_file_example.py
examples/python/create_pdf_file_example.py
from fake import FAKER
file = FAKER.pdf_file()
assert file.data["storage"].exists(str(file))
examples/python/django_example.py
examples/python/django_example.py
from django.contrib.auth.models import User
user = User.objects.first()
examples/python/tell_me_a_joke_example.py
examples/python/tell_me_a_joke_example.py
from openai import OpenAI
client = OpenAI()
completion = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "developer", "content": "You are a famous comedian."},
{"role": "user", "content": "Tell me a joke."},
],
)
assert isinstance(completion.choices[0].message.content, str)
examples/rst_example/README.rst
examples/rst_example/README.rst
reStructuredText example project
================================
This is a minimal example showing how `pytest-codeblock` will discover
and run only Python snippets whose `:name:` starts with `test_`.
Simple assertion
----------------
.. code-block:: python
:name: test_simple_assert
# A trivial test that always passes
assert 2 + 2 == 4
Multi-part example
------------------
It's possible to split one logical test into multiple blocks.
They will be tested under the first ``:name:`` specified.
Note the ``.. continue::`` directive.
.. code-block:: python
:name: test_compute_square
import math
Some intervening text.
.. continue: test_compute_square
.. code-block:: python
:name: test_compute_square_part_2
result = math.pow(3, 2)
assert result == 9
Some intervening text.
.. continue: test_compute_square
.. code-block:: python
:name: test_compute_square_part_3
print(result)
Ignored snippets
----------------
Blocks without a `:name:` or without the `test_` prefix are **not** collected:
.. code-block:: python
# No :name:, so this is ignored
.. code-block:: python
:name: example_not_test
# Name does not start with `test_`, so this is ignored
Non-Python blocks are also ignored
----------------------------------
.. code-block:: bash
:name: test_should_be_ignored
echo "Not Python → skipped"
Custom pytest marks
-------------------
.. pytestmark: django_db
.. code-block:: python
:name: test_django
from django.contrib.auth.models import User
user = User.objects.first()
examples/rst_example/__init__.py
examples/rst_example/__init__.py
examples/rst_example/customisation.rst
examples/rst_example/customisation.rst
Customisation
=============
Customisation examples.
.. External references
.. _openai: https://github.com/openai/openai-python
.. _moto: https://docs.getmoto.org
.. _fake.py: https://github.com/barseghyanartur/fake.py
`fake.py`_ example for `.. code-block::` directive
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: rst
.. pytestmark: fakepy
.. code-block:: python
:name: test_create_pdf_file
from fake import FAKER
file = FAKER.pdf_file()
assert file.data["storage"].exists(str(file))
`moto`_ example for `.. code-block::` directive
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: rst
.. pytestmark: aws
.. code-block:: python
:name: test_create_bucket
import boto3
s3 = boto3.client("s3", region_name="us-east-1")
s3.create_bucket(Bucket="my-bucket")
assert "my-bucket" in [b["Name"] for b in s3.list_buckets()["Buckets"]]
`openai`_ example for `.. code-block::` directive
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: rst
.. pytestmark: xfail
.. pytestmark: openai
.. code-block:: python
:name: test_tell_me_a_joke
from openai import OpenAI
client = OpenAI()
completion = client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "developer", "content": "You are a famous comedian."},
{"role": "user", "content": "Tell me a joke."},
],
)
assert isinstance(completion.choices[0].message.content, str)
`fake.py`_ example for `.. literalinclude::` directive
------------------------------------------------------
.. code-block:: rst
.. pytestmark: fakepy
.. literalinclude:: examples/python/create_pdf_file_example.py
:name: test_li_create_pdf_file
`moto`_ example for `.. literalinclude::` directive
---------------------------------------------------
.. code-block:: rst
.. pytestmark: aws
.. literalinclude:: examples/python/create_bucket_example.py
:name: test_li_create_bucket
`openai`_ example for `.. literalinclude::` directive
-----------------------------------------------------
.. code-block:: rst
.. pytestmark: xfail
.. pytestmark: aws
.. literalinclude:: examples/python/tell_me_a_joke_example.py
:name: test_li_tell_me_a_joke
examples/rst_example/django_settings.py
examples/rst_example/django_settings.py
from pathlib import Path
from django import conf, http, urls
from django.core.handlers import asgi
# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True
INSTALLED_APPS = [
"django.contrib.admin",
"django.contrib.auth",
"django.contrib.contenttypes",
"django.contrib.sessions",
"django.contrib.messages",
"django.contrib.staticfiles",
]
DATABASES = {
"default": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": BASE_DIR / "django_db.sqlite3",
}
}
conf.settings.configure(
ALLOWED_HOSTS="*",
ROOT_URLCONF=__name__,
INSTALLED_APPS=INSTALLED_APPS,
DATABASES=DATABASES,
)
app = asgi.ASGIHandler()
async def root(_request: http.HttpRequest) -> http.JsonResponse:
return http.JsonResponse({"message": "OK"})
urlpatterns = [urls.path("", root)]
scripts/generate_project_source_tree.py
scripts/generate_project_source_tree.py
#!/usr/bin/env python3
import argparse
import fnmatch
import os
from pathlib import Path
def build_tree(
path: Path,
max_depth: int,
ignore_patterns: list,
whitelist_dirs: list,
include_all: bool,
root: Path,
prefix: str = "",
) -> str:
"""
Recursively build an ASCII tree up to max_depth, applying whitelist and
ignore rules.
"""
if max_depth < 0:
return ""
entries = sorted(
path.iterdir(), key=lambda p: (p.is_file(), p.name.lower())
)
lines = []
for i, entry in enumerate(entries):
rel_path = entry.relative_to(root).as_posix()
# Skip ignored patterns
if any(fnmatch.fnmatch(rel_path, pat) for pat in ignore_patterns):
continue
# Enforce whitelist if not including all
if not include_all and whitelist_dirs and not any(
rel_path.startswith(w.rstrip("/")) for w in whitelist_dirs
):
continue
connector = "└── " if i == len(entries) - 1 else "├── "
lines.append(f"{prefix}{connector}{entry.name}")
# Recurse into directories
if entry.is_dir():
extension = " " if i == len(entries) - 1 else "│ "
subtree = build_tree(
entry,
max_depth - 1,
ignore_patterns,
whitelist_dirs,
include_all,
root,
prefix + extension,
)
if subtree:
lines += subtree.splitlines()
return "\n".join(lines)
def detect_language(path: Path) -> str:
"""Map file suffix to Sphinx language."""
mapping = {
".py": "python",
".js": "javascript",
".java": "java",
".md": "markdown",
".yaml": "yaml",
".yml": "yaml",
".json": "json",
".sh": "bash",
".rst": "rst",
}
return mapping.get(path.suffix, "")
def main():
p = argparse.ArgumentParser(
description="Auto-generate a .rst with tree + literalinclude blocks"
)
p.add_argument(
"-p",
"--project-root",
type=Path,
default=Path("."),
help="Path to your project directory",
)
p.add_argument(
"-d",
"--depth",
type=int,
default=10,
help="How many levels deep to print in the tree",
)
p.add_argument(
"-o",
"--output",
type=Path,
default=Path("docs/source_tree.rst"),
help="Where to write the generated .rst",
)
p.add_argument(
"-e",
"--ext",
nargs="+",
default=[".py", ".md", ".js", ".rst"],
help="Which file extensions to include via literalinclude",
)
p.add_argument(
"-i",
"--ignore",
nargs="+",
default=["__pycache__", "*.pyc", "*.py,cover"],
help="Ignore files or dirs matching these glob patterns (relative to "
"project root)",
)
p.add_argument(
"-w",
"--whitelist",
nargs="+",
default=["src", "docs", "examples", "scripts"],
help="Directories (relative to project root) to include "
"unless --include-all is given",
)
p.add_argument(
"--include-all",
action="store_true",
help="Include all files regardless of whitelist",
)
args = p.parse_args()
root = args.project_root.resolve()
ignore_patterns = args.ignore
whitelist_dirs = args.whitelist
include_all = args.include_all
output = args.output.resolve()
output_dir = output.parent.resolve()
# Header + tree
header = f"""Project source-tree
===================
Below is the layout of our project (to {args.depth} levels), followed by
the contents of each key file.
.. code-block:: bash
:caption: Project directory layout
{root.name}/
"""
tree = build_tree(
root,
args.depth,
ignore_patterns,
whitelist_dirs,
include_all,
root,
prefix=" ",
)
out = [header, tree, ""]
# Walk and collect files
for filepath in sorted(root.rglob("*")):
if not filepath.is_file() or filepath.suffix not in args.ext:
continue
rel_path = filepath.relative_to(root).as_posix()
# Skip ignored
if any(fnmatch.fnmatch(rel_path, pat) for pat in ignore_patterns):
continue
# Enforce whitelist
if (
not include_all
and whitelist_dirs
and not any(
rel_path.startswith(w.rstrip("/")) for w in whitelist_dirs)
):
continue
# Compute include path relative to output_dir
include_path = os.path.relpath(filepath, output_dir).replace(
os.sep, "/"
)
title = rel_path
underline = "-" * len(title)
lang = detect_language(filepath)
out += [
title,
underline,
"",
f".. literalinclude:: {include_path}",
f" :language: {lang}" if lang else "",
f" :caption: {rel_path}",
# " :linenos:",
"",
]
# Write output
args.output.write_text("\n".join(line for line in out if line is not None))
print(f"Wrote {args.output}")
if __name__ == "__main__":
main()
src/pytest_codeblock/__init__.py
src/pytest_codeblock/__init__.py
from pathlib import Path
from .md import MarkdownFile
from .rst import RSTFile
__title__ = "pytest-codeblock"
__version__ = "0.3.5"
__author__ = "Artur Barseghyan <artur.barseghyan@gmail.com>"
__copyright__ = "2025-2026 Artur Barseghyan"
__license__ = "MIT"
__all__ = (
"pytest_collect_file",
)
def pytest_collect_file(parent, path):
"""Collect .md and .rst files for codeblock tests."""
# Determine file extension (works for py.path or pathlib.Path)
file_name = str(path).lower()
if file_name.endswith((".md", ".markdown")):
# Use the MarkdownFile collector for Markdown files
return MarkdownFile.from_parent(parent=parent, path=Path(path))
if file_name.endswith(".rst"):
# Use the RSTFile collector for reStructuredText files
return RSTFile.from_parent(parent=parent, path=Path(path))
return None
src/pytest_codeblock/collector.py
src/pytest_codeblock/collector.py
from dataclasses import dataclass, field
from typing import Optional
__author__ = "Artur Barseghyan <artur.barseghyan@gmail.com>"
__copyright__ = "2025-2026 Artur Barseghyan"
__license__ = "MIT"
__all__ = (
"CodeSnippet",
"group_snippets",
)
@dataclass
class CodeSnippet:
"""Data container for an extracted code snippet."""
code: str # The code content
line: int # Starting line number in the source
name: Optional[str] = None # Identifier for grouping (None if anonymous)
marks: list[str] = field(default_factory=list)
# Collected pytest marks (e.g. ['django_db']), parsed from doc comments
fixtures: list[str] = field(default_factory=list)
# Collected pytest fixtures (e.g. ['tmp_path']), parsed from doc comments
def group_snippets(snippets: list[CodeSnippet]) -> list[CodeSnippet]:
"""
Merge snippets with the same name into one CodeSnippet,
concatenating their code and accumulating marks.
Unnamed snippets get unique auto-names.
"""
combined: list[CodeSnippet] = []
seen: dict[str, CodeSnippet] = {}
anon_count = 0
for sn in snippets:
key = sn.name
if not key:
anon_count += 1
key = f"codeblock{anon_count}"
if key in seen:
seen_sn = seen[key]
seen_sn.code += "\n" + sn.code
seen_sn.marks.extend(sn.marks)
seen_sn.fixtures.extend(sn.fixtures)
else:
sn.marks = list(sn.marks) # copy
sn.fixtures = list(sn.fixtures) # copy
seen[key] = sn
combined.append(sn)
return combined
src/pytest_codeblock/constants.py
src/pytest_codeblock/constants.py
__author__ = "Artur Barseghyan <artur.barseghyan@gmail.com>"
__copyright__ = "2025-2026 Artur Barseghyan"
__license__ = "MIT"
__all__ = (
"CODEBLOCK_MARK",
"DJANGO_DB_MARKS",
"TEST_PREFIX",
)
DJANGO_DB_MARKS = {
"django_db",
"db",
"transactional_db",
}
TEST_PREFIX = "test_"
CODEBLOCK_MARK = "codeblock"
src/pytest_codeblock/md.py
src/pytest_codeblock/md.py
import asyncio
import inspect
import re
import textwrap
import traceback
from typing import Optional
import pytest
from .collector import CodeSnippet, group_snippets
from .constants import CODEBLOCK_MARK, DJANGO_DB_MARKS, TEST_PREFIX
from .helpers import contains_top_level_await, wrap_async_code
__author__ = "Artur Barseghyan <artur.barseghyan@gmail.com>"
__copyright__ = "2025-2026 Artur Barseghyan"
__license__ = "MIT"
__all__ = (
"MarkdownFile",
"parse_markdown",
)
def parse_markdown(text: str) -> list[CodeSnippet]:
"""
Parse Markdown text and extract Python code snippets as CodeSnippet
objects.
Supports:
- <!-- pytestmark: <mark> --> comments immediately before a code fence
- <!-- codeblock-name: <name> --> comments for naming
- <!-- continue: <name> --> comments for grouping with a named snippet
- Fenced code blocks with ```python (and optional name=<name> in the
info string)
Captures each snippet's name, code, starting line, and any pytest marks.
"""
snippets: list[CodeSnippet] = []
lines = text.splitlines()
pending_name: Optional[str] = None
pending_continue: Optional[str] = None
pending_marks: list[str] = [CODEBLOCK_MARK]
pending_fixtures: list[str] = []
in_block = False
fence = ""
block_indent = 0
code_buffer: list[str] = []
snippet_name: Optional[str] = None
start_line = 0
for idx, line in enumerate(lines, start=1):
stripped = line.strip()
if not in_block:
# Check for pytest mark comment
if stripped.startswith("<!--") and "pytestmark:" in stripped:
m = re.match(r"<!--\s*pytestmark:\s*(\w+)\s*-->", stripped)
if m:
pending_marks.append(m.group(1))
continue
# Check for pytest fixture comment
if stripped.startswith("<!--") and "pytestfixture:" in stripped:
m = re.match(r"<!--\s*pytestfixture:\s*(\w+)\s*-->", stripped)
if m:
pending_fixtures.append(m.group(1))
continue
# Check for continue comment
if stripped.startswith("<!--") and "continue:" in stripped:
m = re.match(r"<!--\s*continue:\s*(\S+)\s*-->", stripped)
if m:
pending_continue = m.group(1)
continue
# Check for name comment
if stripped.startswith("<!--") and "codeblock-name:" in stripped:
m = re.match(
r"<!--\s*codeblock-name:\s*([^ >]+)\s*-->", stripped
)
if m:
pending_name = m.group(1)
continue
# Start of fenced code block?
if line.lstrip().startswith("```"):
indent = len(line) - len(line.lstrip())
m = re.match(r"^`{3,}", line.lstrip())
if not m:
continue
fence = m.group(0)
info = line.lstrip()[len(fence):].strip()
parts = info.split(None, 1)
lang = parts[0].lower() if parts else ""
extra = parts[1] if len(parts) > 1 else ""
if lang in ("python", "py", "python3"):
in_block = True
block_indent = indent
start_line = idx + 1
code_buffer = []
# determine name from info string or pending comment
snippet_name = None
for token in extra.split():
if (
token.startswith("name=")
or token.startswith("name:")
):
snippet_name = (
token.split("=", 1)[-1]
if "=" in token
else token.split(":", 1)[-1]
)
break
if snippet_name is None:
snippet_name = pending_name
# reset pending_name; marks stay until block closes
pending_name = None
continue
else:
# inside a fenced code block
if line.lstrip().startswith(fence):
# end of block
in_block = False
code_text = "\n".join(code_buffer)
# continue overrides snippet_name for grouping
if pending_continue:
final_name = pending_continue
pending_continue = None
else:
final_name = snippet_name
snippets.append(CodeSnippet(
name=final_name,
code=code_text,
line=start_line,
marks=pending_marks.copy(),
fixtures=pending_fixtures.copy(),
))
# reset pending marks after collecting
pending_marks = [CODEBLOCK_MARK] # Reset to default
snippet_name = None
pending_fixtures.clear() # Clear pending fixtures
else:
# collect code lines (dedent by block_indent)
if line.strip() == "":
code_buffer.append("")
else:
if len(line) >= block_indent:
code_buffer.append(line[block_indent:])
else:
code_buffer.append(line.lstrip())
continue
return snippets
class MarkdownFile(pytest.File):
"""
Collector for Markdown files, extracting only `test_`-prefixed code
snippets.
"""
def collect(self):
text = self.path.read_text(encoding="utf-8")
raw = parse_markdown(text)
# keep only snippets named test_*
tests = [
sn for sn in raw if sn.name and sn.name.startswith(TEST_PREFIX)
]
combined = group_snippets(tests)
for sn in combined:
# Bind the values we need so we don't close over `sn` itself
_sn_name = sn.name
_fpath = str(self.path)
# Build list of fixture names requested by this snippet
_fixture_names: list[str] = list(sn.fixtures)
# If snippet is marked as needing DB, also request the `db`
# fixture, unless user already added it explicitly.
if (
DJANGO_DB_MARKS.intersection(sn.marks)
and "db" not in _fixture_names
):
_fixture_names.append("db")
# Generate a real pytest Function so fixtures work
def make_func(
code,
sn_name=_sn_name,
fpath=_fpath,
fixture_names=_fixture_names,
):
# This inner function *actually* has a **fixtures signature,
# but we override __signature__ so pytest passes the right
# fixtures and names.
def test_block(**fixtures):
# Auto-wrap async code
ex_code = code
if contains_top_level_await(code):
ex_code = wrap_async_code(code)
try:
compiled = compile(ex_code, fpath, "exec")
except SyntaxError as err:
raise SyntaxError(
f"Syntax error in "
f"codeblock `{sn_name}` in {fpath}:\n"
f"\n{textwrap.indent(ex_code, prefix=' ')}\n\n"
f"{traceback.format_exc()}"
) from err
try:
# Make fixtures available as top-level names
# inside the executed snippet.
exec(compiled, {"asyncio": asyncio, **dict(fixtures)})
except Exception as err:
raise Exception(
f"Error in "
f"codeblock `{sn_name}` in {fpath}:\n"
f"\n{textwrap.indent(ex_code, prefix=' ')}\n\n"
f"{traceback.format_exc()}"
) from err
# Tell pytest which fixture arguments this test has:
test_block.__signature__ = inspect.Signature(
[
inspect.Parameter(
name,
inspect.Parameter.POSITIONAL_OR_KEYWORD,
)
for name in fixture_names
]
)
return test_block
callobj = make_func(sn.code)
fn = pytest.Function.from_parent(
parent=self,
name=sn.name,
callobj=callobj,
)
# apply any marks (e.g. django_db)
for m in sn.marks:
fn.add_marker(getattr(pytest.mark, m))
yield fn
src/pytest_codeblock/rst.py
src/pytest_codeblock/rst.py
import asyncio
import inspect
import re
import textwrap
import traceback
from pathlib import Path
from typing import Optional, Union
import pytest
from .collector import CodeSnippet, group_snippets
from .constants import CODEBLOCK_MARK, DJANGO_DB_MARKS, TEST_PREFIX
from .helpers import contains_top_level_await, wrap_async_code
__author__ = "Artur Barseghyan <artur.barseghyan@gmail.com>"
__copyright__ = "2025-2026 Artur Barseghyan"
__license__ = "MIT"
__all__ = (
"RSTFile",
"parse_rst",
"resolve_literalinclude_path",
"get_literalinclude_content",
)
def resolve_literalinclude_path(
base_dir: Union[str, Path],
include_path: str,
) -> Optional[str]:
"""
Resolve the full path for a literalinclude directive.
Returns None if the file doesn't exist.
"""
_include_path = Path(include_path)
# If `include_path` is already absolute or relative and exists, done
if _include_path.exists():
return str(_include_path.resolve())
# If base_path is a file, switch to its parent directory
_base_path = Path(base_dir)
if _base_path.is_file():
_base_path = _base_path.parent
try:
full_path = _base_path / include_path
if full_path.exists():
return str(full_path.resolve())
except Exception:
pass
return None
def get_literalinclude_content(path):
try:
with open(path) as f:
return f.read()
except Exception as e:
raise RuntimeError(
f"Failed to read literalinclude file {path}: {e}"
) from e
def parse_rst(text: str, base_dir: Path) -> list[CodeSnippet]:
"""
Parse an RST document into CodeSnippet objects, capturing:
- .. pytestmark: <mark>
- .. continue: <name>
- .. codeblock-name: <name>
- .. code-block:: python
"""
snippets: list[CodeSnippet] = []
lines = text.splitlines()
n = len(lines)
pending_name: Optional[str] = None
pending_marks: list[str] = [CODEBLOCK_MARK]
pending_fixtures: list[str] = []
pending_continue: Optional[str] = None
i = 0
while i < n:
line = lines[i]
# --------------------------------------------------------------------
# Collect `.. pytestmark: xyz`
# --------------------------------------------------------------------
m = re.match(r"^\s*\.\.\s*pytestmark:\s*(\w+)\s*$", line)
if m:
pending_marks.append(m.group(1))
i += 1
continue
# --------------------------------------------------------------------
# Collect `.. pytestfixture: foo`
# --------------------------------------------------------------------
m = re.match(r"^\s*\.\.\s*pytestfixture:\s*(\w+)\s*$", line)
if m:
pending_fixtures.append(m.group(1))
i += 1
continue
# --------------------------------------------------------------------
# The `.. literalinclude` directive
# --------------------------------------------------------------------
if line.strip().startswith(".. literalinclude::"):
path = line.split(".. literalinclude::", 1)[1].strip()
name = None
# Look ahead for name
j = i + 1
while j < len(lines) and lines[j].strip():
if ":name:" in lines[j]:
name = lines[j].split(":name:", 1)[1].strip()
break
j += 1
if name and name.startswith("test_"):
full_path = resolve_literalinclude_path(base_dir, path)
if full_path:
snippet = CodeSnippet(
code=get_literalinclude_content(full_path),
line=i + 1,
name=name,
marks=pending_marks.copy(),
fixtures=pending_fixtures.copy(),
)
snippets.append(snippet)
# TODO: Is this needed?
# pending_marks.clear()
# pending_fixtures.clear()
i = j + 1
continue
# --------------------------------------------------------------------
# Collect `.. continue: foo`
# --------------------------------------------------------------------
m = re.match(r"^\s*\.\.\s*continue:\s*(\S+)\s*$", line)
if m:
pending_continue = m.group(1)
i += 1
continue
# --------------------------------------------------------------------
# Collect `.. codeblock-name: foo`
# --------------------------------------------------------------------
m = re.match(r"^\s*\.\.\s*codeblock-name:\s*(\S+)\s*$", line)
if m:
pending_name = m.group(1)
i += 1
continue
# --------------------------------------------------------------------
# The `.. code-block` directive
# --------------------------------------------------------------------
m = re.match(r"^(\s*)\.\. (?:code-block|code)::\s*(\w+)", line)
if m:
base_indent = len(m.group(1))
lang = m.group(2).lower()
if lang in ("python", "py", "python3"):
# Parse :name: option
name_val: Optional[str] = None
j = i + 1
while j < n:
ln = lines[j]
if not ln.strip():
j += 1
continue
indent = len(ln) - len(ln.lstrip())
if ln.lstrip().startswith(":") and indent > base_indent:
opt = ln.lstrip()
if opt.lower().startswith(":name:"):
name_val = opt.split(":", 2)[2].strip().split()[0]
j += 1
continue
break
# The j is first code line
if j >= n:
i = j
continue
first = lines[j]
content_indent = len(first) - len(first.lstrip())
if content_indent <= base_indent:
i = j
continue
# Collect code
buf: list[str] = []
k = j
while k < n:
ln = lines[k]
if not ln.strip():
buf.append("")
k += 1
continue
ind = len(ln) - len(ln.lstrip())
if ind >= content_indent:
buf.append(ln[content_indent:])
k += 1
else:
break
# Decide snippet name: continue overrides name_val/pending_name
if pending_continue:
sn_name = pending_continue
pending_continue = None
else:
sn_name = name_val or pending_name
sn_marks = pending_marks.copy()
sn_fixtures = pending_fixtures.copy()
pending_name = None
pending_marks.clear()
pending_fixtures.clear()
snippets.append(CodeSnippet(
name=sn_name,
code="\n".join(buf),
line=j + 1,
marks=sn_marks,
fixtures=sn_fixtures,
))
i = k
continue
else:
i += 1
continue
# --------------------------------------------------------------------
# The literal-block via "::"
# --------------------------------------------------------------------
if line.rstrip().endswith("::") and pending_name:
# Similar override logic
if pending_continue:
sn_name = pending_continue
pending_continue = None
else:
sn_name = pending_name
sn_marks = pending_marks.copy()
sn_fixtures = pending_fixtures.copy()
pending_name = None
pending_marks.clear()
pending_fixtures.clear()
j = i + 1
if j < n and not lines[j].strip():
j += 1
if j >= n:
i = j
continue
first = lines[j]
content_indent = len(first) - len(first.lstrip())
buf: list[str] = []
k = j
while k < n:
ln = lines[k]
if not ln.strip():
buf.append("")
k += 1
continue
ind = len(ln) - len(ln.lstrip())
if ind >= content_indent:
buf.append(ln[content_indent:])
k += 1
else:
break
snippets.append(CodeSnippet(
name=sn_name,
code="\n".join(buf),
line=j + 1,
marks=sn_marks,
fixtures=sn_fixtures,
))
i = k
continue
i += 1
return snippets
class RSTFile(pytest.File):
"""Collect RST code-block tests as real test functions."""
def collect(self):
text = self.path.read_text(encoding="utf-8")
raw = parse_rst(text, self.path)
# Only keep test_* snippets
tests = [
sn for sn in raw if sn.name and sn.name.startswith(TEST_PREFIX)
]
combined = group_snippets(tests)
for sn in combined:
# Bind the values we need so we don't close over `sn` itself
_sn_name = sn.name
_fpath = str(self.path)
# Build list of fixture names requested by this snippet
_fixture_names: list[str] = list(sn.fixtures)
# If snippet is marked as needing DB, also request the `db`
# fixture, unless user already added it explicitly.
if (
DJANGO_DB_MARKS.intersection(sn.marks)
and "db" not in _fixture_names
):
_fixture_names.append("db")
def make_func(
code,
sn_name=_sn_name,
fpath=_fpath,
fixture_names=_fixture_names,
):
# This inner function *actually* has a **fixtures signature,
# but we override __signature__ so pytest passes the right
# fixtures and names.
def test_block(**fixtures):
ex_code = code
if contains_top_level_await(code):
ex_code = wrap_async_code(code)
try:
compiled = compile(ex_code, fpath, "exec")
except SyntaxError as err:
raise SyntaxError(
f"Syntax error in "
f"codeblock `{sn_name}` in {fpath}:\n"
f"\n{textwrap.indent(ex_code, prefix=' ')}\n\n"
f"{traceback.format_exc()}"
) from err
try:
# Make fixtures available as top-level names
# inside the executed snippet.
exec(compiled, {"asyncio": asyncio, **dict(fixtures)})
except Exception as err:
raise Exception(
f"Error in "
f"codeblock `{sn_name}` in {fpath}:\n"
f"\n{textwrap.indent(ex_code, prefix=' ')}\n\n"
f"{traceback.format_exc()}"
) from err
# Tell pytest which fixture arguments this test has:
test_block.__signature__ = inspect.Signature(
[
inspect.Parameter(
name,
inspect.Parameter.POSITIONAL_OR_KEYWORD,
)
for name in fixture_names
]
)
return test_block
callobj = make_func(sn.code)
fn = pytest.Function.from_parent(
parent=self,
name=sn.name,
callobj=callobj
)
# Re-apply any pytest.mark.<foo> markers
for m in sn.marks:
fn.add_marker(getattr(pytest.mark, m))
yield fn
src/pytest_codeblock/tests/__init__.py
src/pytest_codeblock/tests/__init__.py
src/pytest_codeblock/tests/test_pytest_codeblock.py
src/pytest_codeblock/tests/test_pytest_codeblock.py
from ..collector import CodeSnippet, group_snippets
from ..helpers import contains_top_level_await, wrap_async_code
from ..md import parse_markdown
from ..rst import (
get_literalinclude_content,
parse_rst,
resolve_literalinclude_path,
)
__author__ = "Artur Barseghyan <artur.barseghyan@gmail.com>"
__copyright__ = "2025-2026 Artur Barseghyan"
__license__ = "MIT"
__all__ = (
"test_group_snippets_different_names",
"test_group_snippets_merges_named",
"test_parse_markdown_simple",
"test_parse_markdown_with_pytestmark",
"test_parse_rst_literalinclude",
"test_parse_rst_simple",
"test_resolve_literalinclude_and_content",
)
def test_group_snippets_merges_named():
# Two snippets with the same name should be combined
sn1 = CodeSnippet(name="foo", code="a=1", line=1, marks=["codeblock"])
sn2 = CodeSnippet(name="foo", code="b=2", line=2, marks=["codeblock", "m"])
combined = group_snippets([sn1, sn2])
assert len(combined) == 1
cs = combined[0]
assert cs.name == "foo"
# Both code parts should appear
assert "a=1" in cs.code
assert "b=2" in cs.code
# Marks should accumulate
assert "m" in cs.marks
def test_group_snippets_different_names():
# Snippets with different names are not grouped
sn1 = CodeSnippet(name="foo", code="x=1", line=1)
sn2 = CodeSnippet(name="bar", code="y=2", line=2)
combined = group_snippets([sn1, sn2])
assert len(combined) == 2
assert combined[0].name.startswith("foo")
assert combined[1].name.startswith("bar")
def test_parse_markdown_simple():
text = """
```python name=test_example
x=1
assert x==1
```"""
snippets = parse_markdown(text)
assert len(snippets) == 1
sn = snippets[0]
assert sn.name == "test_example"
assert "x=1" in sn.code
def test_parse_markdown_with_pytestmark():
text = """
<!-- pytestmark: django_db -->
```python name=test_db
from django.db import models
```"""
snippets = parse_markdown(text)
assert len(snippets) == 1
sn = snippets[0]
# Should include both default and django_db marks
assert "django_db" in sn.marks
assert "codeblock" in sn.marks
def test_resolve_literalinclude_and_content(tmp_path):
base = tmp_path / "dir"
base.mkdir()
file = base / "a.py"
file.write_text("print('hello')")
# Absolute path resolution
abs_path = resolve_literalinclude_path(base, str(file))
assert abs_path == str(file.resolve())
# Relative path resolution
rel_path = resolve_literalinclude_path(base, "a.py")
assert rel_path == str(file.resolve())
# Content read
content = get_literalinclude_content(str(file))
assert content == "print('hello')"
def test_parse_rst_simple(tmp_path):
# Basic code-block directive
rst = """
.. code-block:: python
:name: test_simple
a=2
assert a==2
"""
snippets = parse_rst(rst, tmp_path)
assert len(snippets) == 1
sn = snippets[0]
assert sn.name == "test_simple"
assert "a=2" in sn.code
def test_parse_rst_literalinclude(tmp_path):
# Create an external file to include
include_dir = tmp_path / "inc"
include_dir.mkdir()
target = include_dir / "foo.py"
target.write_text("z=3\nassert z==3")
rst = f"""
.. literalinclude:: {target.name}
:name: test_li
"""
snippets = parse_rst(rst, include_dir)
assert len(snippets) == 1
sn = snippets[0]
assert sn.name == "test_li"
assert "z=3" in sn.code
def test_contains_top_level_await_positive():
"""Verify detection of various async constructs."""
# Direct await
assert contains_top_level_await("await asyncio.sleep(0)") is True
# Async function definition
assert contains_top_level_await("async def foo(): pass") is True
# Async with
assert contains_top_level_await("async with lock: pass") is True
# Async for
assert contains_top_level_await("async for i in range(1): pass") is True
def test_contains_top_level_await_negative():
"""Verify that sync code or strings containing keywords are ignored."""
# Standard sync code
assert contains_top_level_await("import time; time.sleep(1)") is False
# Keywords inside strings
assert contains_top_level_await("print('this is an await')") is False
# Comments should be ignored
assert contains_top_level_await("# await inside comment") is False
def test_contains_top_level_await_invalid_syntax():
"""Verify that invalid syntax returns False rather than crashing."""
assert contains_top_level_await("def main(:") is False
def test_wrap_async_code_structure():
"""Verify the transformation logic and indentation."""
code = "await asyncio.sleep(1)\nreturn 42"
wrapped = wrap_async_code(code)
# Check for the boilerplate components
assert "async def __async_main__():" in wrapped
assert "asyncio.run(__async_main__())" in wrapped
# Check that the original code is indented correctly (4 spaces)
assert " await asyncio.sleep(1)" in wrapped
assert " return 42" in wrapped
def test_wrap_async_code_execution_integrity():
"""
Verify that the wrapped code is still valid Python and can be compiled.
This ensures wrap_async_code doesn't break the AST.
"""
code = "val = 1 + 1"
wrapped = wrap_async_code(code)
# If compile fails, the test fails
assert compile(wrapped, "<string>", "exec")
src/pytest_codeblock/tests/tests.rst
src/pytest_codeblock/tests/tests.rst
Tests
=====
test_group_snippets_merges_named
--------------------------------
.. code-block:: python
:name: test_group_snippets_merges_named
import pytest
from pathlib import Path
from pytest_codeblock.collector import CodeSnippet, group_snippets
from pytest_codeblock.md import parse_markdown
from pytest_codeblock.rst import (
parse_rst,
resolve_literalinclude_path,
get_literalinclude_content,
)
# Two snippets with the same name should be combined
sn1 = CodeSnippet(name="foo", code="a=1", line=1, marks=["codeblock"])
sn2 = CodeSnippet(name="foo", code="b=2", line=2, marks=["codeblock", "m"])
combined = group_snippets([sn1, sn2])
assert len(combined) == 1
cs = combined[0]
assert cs.name == "foo"
# Both code parts should appear
assert "a=1" in cs.code
assert "b=2" in cs.code
# Marks should accumulate
assert "m" in cs.marks
----
test_group_snippets_different_names
-----------------------------------
.. code-block:: python
:name: test_group_snippets_different_names
import pytest
from pathlib import Path
from pytest_codeblock.collector import CodeSnippet, group_snippets
from pytest_codeblock.md import parse_markdown
from pytest_codeblock.rst import (
parse_rst,
resolve_literalinclude_path,
get_literalinclude_content,
)
# Snippets with different names are not grouped
sn1 = CodeSnippet(name="foo", code="x=1", line=1)
sn2 = CodeSnippet(name="bar", code="y=2", line=2)
combined = group_snippets([sn1, sn2])
assert len(combined) == 2
assert combined[0].name.startswith("foo")
assert combined[1].name.startswith("bar")
----
test_parse_markdown_simple
--------------------------
.. code-block:: python
:name: test_parse_markdown_simple
import pytest
from pathlib import Path
from pytest_codeblock.collector import CodeSnippet, group_snippets
from pytest_codeblock.md import parse_markdown
from pytest_codeblock.rst import (
parse_rst,
resolve_literalinclude_path,
get_literalinclude_content,
)
text = """
```python name=test_example
x=1
assert x==1
```"""
snippets = parse_markdown(text)
assert len(snippets) == 1
sn = snippets[0]
assert sn.name == "test_example"
assert "x=1" in sn.code
----
test_parse_markdown_with_pytestmark
-----------------------------------
.. code-block:: python
:name: test_parse_markdown_with_pytestmark
import pytest
from pathlib import Path
from pytest_codeblock.collector import CodeSnippet, group_snippets
from pytest_codeblock.md import parse_markdown
from pytest_codeblock.rst import (
parse_rst,
resolve_literalinclude_path,
get_literalinclude_content,
)
text = """
<!-- pytestmark: django_db -->
```python name=test_db
from django.db import models
```"""
snippets = parse_markdown(text)
assert len(snippets) == 1
sn = snippets[0]
# Should include both default and django_db marks
assert "django_db" in sn.marks
assert "codeblock" in sn.marks
----
test_pytest_fixtures
--------------------
.. pytestfixture: tmp_path
.. pytestfixture: http_request
.. code-block:: python
:name: test_pytest_fixtures_1
d = tmp_path / "sub"
d.mkdir() # Create the directory
assert d.is_dir() # Verify it was created and is a directory
assert isinstance(http_request.GET, dict)
----
.. pytestfixture: tmp_path
.. pytestfixture: http_request
.. code-block:: python
:name: test_pytest_fixtures_2
d = tmp_path / "sub"
d.mkdir() # Create the directory
assert d.is_dir() # Verify it was created and is a directory
assert isinstance(http_request.GET, dict)
----
.. pytestfixture: tmp_path
.. pytestfixture: http_request
.. code-block:: python
:name: test_pytest_fixtures_3
d = tmp_path / "sub"
d.mkdir() # Create the directory
assert d.is_dir() # Verify it was created and is a directory
assert isinstance(http_request.GET, dict)
----
test_async_example
------------------
.. code-block:: python
:name: test_async_example
import asyncio
result = await asyncio.sleep(0.1, result=42)
assert result == 42
----
test_group_snippets
-------------------
.. code-block:: python
:name: test_group_snippets
text_2 = "Jude"
Something in between
.. continue: test_group_snippets
.. code-block:: python
:name: test_group_snippets_part_2
assert text_2
print(text_2)