pytest-codeblock **************** Test your documentation code blocks. [image: PyPI Version][image][image: Supported Python versions][image][image: Build Status][image][image: Documentation Status][image][image: llms.txt - documentation for LLMs][image][image: MIT][image][image: Coverage][image] pytest-codeblock is a Pytest plugin that discovers Python code examples in your reStructuredText and Markdown documentation files and runs them as part of your test suite. This ensures your docs stay correct and up-to-date. Features ======== * **reStructuredText and Markdown support**: Automatically find and test code blocks in reStructuredText (".rst") and Markdown (".md") files. Async code snippets are supported as well. * **Grouping**: Split a single example across multiple code blocks; the plugin concatenates them into one test. * **Pytest markers support**: Add existing or custom pytest markers to the code blocks and hook into the tests life-cycle using "conftest.py". * **Pytest fixtures support**: Request existing or custom pytest fixtures for the code blocks. Prerequisites ============= * Python 3.10+ * pytest is the only required dependency (on Python 3.11+; for Python 3.10 tomli is also required). Documentation ============= * Documentation is available on Read the Docs. * For *reStructuredText*, see a dedicated reStructuredText docs. * For *Markdown*, see a dedicated Markdown docs. * Both reStructuredText docs and Markdown docs have extensive documentation on pytest markers and corresponding "conftest.py" hooks. * For guidelines on contributing check the Contributor guidelines. Installation ============ Install with pip: pip install pytest-codeblock Or install with uv: uv pip install pytest-codeblock Configuration ============= For most use cases, no configuration needed. By default, all code blocks with a name starting with "test_" will be collected and executed as tests. This allows you to have both test and non-test code blocks in your documentation, giving you flexibility in how you structure your examples. However, if you want to test all code blocks, you can set "test_nameless_codeblocks" to "true" in your *pyproject.toml*: *Filename: pyproject.toml* [tool.pytest-codeblock] test_nameless_codeblocks = true If you still want to skip some code blocks, you can use built-in or custom pytest markers. See the dedicated reStructuredText docs and Markdown docs to learn more about *pytestmark* directive. Note, that nameless code blocks have limitations when it comes to grouping. ====================================================================== By default, all code *.rst* and *.md* files shall be picked automatically. However, if you need to add another file extension or use or another language identifier for python in codeblock, you could configure that. See the following example of *pyproject.toml* configuration: *Filename: pyproject.toml* [tool.pytest-codeblock] rst_user_codeblocks = ["c_py"] rst_user_extensions = [".rst.txt"] md_user_codeblocks = ["c_py"] md_user_extensions = [".md.txt"] See customisation docs for more. Usage ===== Note: It's highly recommended to use doc8 for catching possible markup errors, that otherwise would be difficult to spot. reStructruredText usage ----------------------- Any code directive, such as ".. code-block:: python", ".. code:: python", or literal blocks with a preceding ".. codeblock-name: ", will be collected and executed automatically by pytest. "code-block" directive example ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Note: Note that ":name:" value has a "test_" prefix. *Filename: README.rst* .. code-block:: python :name: test_basic_example import math result = math.pow(3, 2) assert result == 9 "literalinclude" directive example ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Note: Note that ":name:" value has a "test_" prefix. *Filename: README.rst* .. literalinclude:: examples/python/basic_example.py :name: test_li_basic_example See a dedicated reStructuredText docs for more. Markdown usage -------------- Any fenced code block with a recognized Python language tag (e.g., "python", "py") will be collected and executed automatically by pytest. Note: Note that "name" value has a "test_" prefix. *Filename: README.md* ```python name=test_basic_example import math result = math.pow(3, 2) assert result == 9 ``` See a dedicated Markdown docs for more. Tests ===== Run the tests with pytest: pytest Troubleshooting =============== If something doesn't work, try to add this to your pyproject.toml: *Filename: pyproject.toml* [tool.pytest.ini_options] testpaths = [ "**/*.rst", "**/*.md", ] Writing documentation ===================== Keep the following hierarchy. ===== title ===== header ====== sub-header ---------- sub-sub-header ~~~~~~~~~~~~~~ sub-sub-sub-header ^^^^^^^^^^^^^^^^^^ sub-sub-sub-sub-header ++++++++++++++++++++++ sub-sub-sub-sub-sub-header ************************** License ======= MIT Support ======= For security issues contact me at the e-mail given in the Author section. For overall issues, go to GitHub. Author ====== Artur Barseghyan ====================================================================== Quick-start reference ===================== pytest-codeblock collects Python code blocks from ".rst" and ".md" files and runs them as pytest tests. No configuration is required for basic use — files are discovered automatically. ====================================================================== Naming rules ------------ Only blocks whose name starts with "test_" are collected by default. To test all blocks regardless of name, set "test_nameless_codeblocks = true" in "pyproject.toml". ====================================================================== RST syntax ---------- .. code-block:: python :name: test_my_example result = 1 + 1 assert result == 2 Add a pytest marker (RST) ~~~~~~~~~~~~~~~~~~~~~~~~~ .. pytestmark: skip .. code-block:: python :name: test_skipped_block pass Request a fixture (RST) ~~~~~~~~~~~~~~~~~~~~~~~ .. pytestfixture: tmp_path .. code-block:: python :name: test_uses_tmp_path d = tmp_path / "sub" d.mkdir() assert d.is_dir() Group blocks (RST) — shared context ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code-block:: python :name: test_part_one x = 1 Some prose in between. .. continue: test_part_one .. code-block:: python :name: test_part_two y = x + 1 assert y == 2 All blocks sharing the same group key are concatenated into one test under the first name. Incremental grouping — each step is its own test ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ When every continuation block has a **distinct** name, each step becomes a cumulative test: .. code-block:: python :name: test_step_1 a = 1 .. continue: test_step_1 .. code-block:: python :name: test_step_2 b = a + 1 assert b == 2 This produces two tests: "test_step_1" (code: "a=1") and "test_step_2" (code: "a=1\nb=a+1\nassert b==2"). Literal block (RST) ~~~~~~~~~~~~~~~~~~~ .. codeblock-name: test_literal_block Example code:: result = "hello" assert result == "hello" Include external file (RST) ~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. literalinclude:: examples/snippet.py :name: test_external_snippet Run as standalone pytest suite (RST) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. pytestmark: pytestrun .. code-block:: python :name: test_pytest_style import pytest class TestMath: def test_add(self): assert 1 + 1 == 2 ====================================================================== Markdown syntax --------------- ```python name=test_my_example result = 1 + 1 assert result == 2 ``` Add a pytest marker (Markdown) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ```python name=test_skipped pass ``` Request a fixture (Markdown) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ```python name=test_uses_tmp_path d = tmp_path / "sub" d.mkdir() assert d.is_dir() ``` Group blocks (Markdown) ~~~~~~~~~~~~~~~~~~~~~~~ ```python name=test_setup x = 1 ``` ```python name=test_continuation y = x + 1 assert y == 2 ``` Run as standalone pytest suite (Markdown) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ```python name=test_class_example import pytest class TestMath: @pytest.fixture def value(self): return 42 def test_value(self, value): assert value == 42 ``` ====================================================================== Async support ------------- Top-level "await" is automatically wrapped — no extra config needed: .. code-block:: python :name: test_async_block import asyncio result = await asyncio.sleep(0.1, result=99) assert result == 99 ====================================================================== pyproject.toml configuration ---------------------------- [tool.pytest-codeblock] # Test all blocks regardless of test_ prefix (default: false) test_nameless_codeblocks = false # Add custom language identifiers (in addition to python, py, python3) rst_user_codeblocks = [] md_user_codeblocks = [] # Add custom file extensions rst_user_extensions = [] md_user_extensions = [] testpaths troubleshooting ------------------------- If docs are not discovered, add explicitly: [tool.pytest.ini_options] testpaths = ["src/app/tests", "docs"] ====================================================================== conftest.py hook integration ---------------------------- Use "CODEBLOCK_MARK" from "pytest_codeblock.constants" to identify doc-block tests: from pytest_codeblock.constants import CODEBLOCK_MARK def pytest_collection_modifyitems(config, items): for item in items: if item.get_closest_marker(CODEBLOCK_MARK): item.add_marker(pytest.mark.documentation) Custom fixtures used in doc blocks are defined in "conftest.py" exactly like regular fixtures. Multiple "pytestfixture" directives on consecutive lines are all applied to the next block. Fixture requests in the first block of a group automatically apply to all continuation blocks. ====================================================================== reStructuredText ================ The following directives are supported: * ".. code-block:: python" * ".. code:: python" * ".. codeblock-name: " * ".. literalinclude::" Any code directive, such as ".. code-block:: python", ".. code:: python", ".. literalinclude::" or literal blocks with a preceding ".. codeblock-name: ", will be collected and executed automatically, if your pytest configuration allows that. Usage examples -------------- Standalone code blocks ~~~~~~~~~~~~~~~~~~~~~~ "code-block" directive """""""""""""""""""""" Note: Note that ":name:" value has a "test_" prefix. *Filename: README.rst* .. code-block:: python :name: test_basic_example import math result = math.pow(3, 2) assert result == 9 ====================================================================== "literalinclude" directive """""""""""""""""""""""""" *Filename: README.rst* .. literalinclude:: examples/python/basic_example.py :name: test_li_basic_example ====================================================================== "codeblock-name" directive """""""""""""""""""""""""" You can also use a literal block with a preceding name comment: *Filename: README.rst* .. codeblock-name: test_grouping_example_literal_block This is a literal block:: y = 5 print(y * 2) ====================================================================== Grouping multiple "code-block" directives ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It's possible to split one logical test into multiple blocks. They will be tested under the first ":name:" specified. Note the ".. continue::" directive. Note: Note that "continue" directive of the "test_grouping_example_part_2" and "test_grouping_example_part_3" refers to the "test_grouping_example". *Filename: README.rst* .. code-block:: python :name: test_grouping_example x = 1 Some intervening text. .. continue: test_grouping_example .. code-block:: python :name: test_grouping_example_part_2 y = x + 1 # Uses x from the first snippet assert y == 2 Some intervening text. .. continue: test_grouping_example .. code-block:: python :name: test_grouping_example_part_3 print(y) # Uses y from the previous snippet The above mentioned three snippets will run as a single test. Note: Note, that nameless code block can't be served as a first block in a group, as there is no way to refer to it. Nameless code blocks can only be used as continuing blocks in a group. ====================================================================== Async ~~~~~ You can use *top-level await* in your code blocks. The code will be automatically wrapped in an async function. *Filename: README.rst* .. code-block:: python :name: test_async_example import asyncio result = await asyncio.sleep(0.1, result=42) assert result == 42 ====================================================================== Adding pytest markers to "code-block" and "literalinclude" directives ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It's possible to add custom pytest markers to your "code-block" or "literalinclude" directives. That allows adding custom logic and mocking in your "conftest.py". In the example below, "django_db" marker is added to the "code-block" directive. Note: Note the "pytestmark" directive "django_db" marker. *Filename: README.rst* .. pytestmark: django_db .. code-block:: python :name: test_django from django.contrib.auth.models import User user = User.objects.first() ====================================================================== Running pytest-style tests within code blocks ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The "pytestrun" marker allows code blocks to be executed as standalone pytest suites. Unlike standard code blocks that are simply executed with "exec()", blocks with the "pytestrun" marker support full pytest functionality including test classes, fixtures, and setup/teardown within documentation snippets. Note: Note the "pytestmark" directive "pytestrun" marker. *Filename: README.rst* .. pytestmark: pytestrun .. code-block:: python :name: test_pytestrun_example import pytest class TestSystemInfo: @pytest.fixture def system_name(self): return "Linux" @pytest.fixture def version_number(self): return 5 def test_combined_info(self, system_name, version_number): info = f"{system_name} v{version_number}" assert info == "Linux v5" def test_name_only(self, system_name): assert system_name.isalpha() ====================================================================== In the example below, "django_db" marker is added to the "literalinclude" directive. *Filename: README.rst* .. pytestmark: django_db .. literalinclude:: examples/python/django_example.py :name: test_li_django_example ====================================================================== Requesting pytest fixtures for "code-block" and "literalinclude" directives ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It's possible to request existing or custom pytest fixtures in "code- block" or "literalinclude" directives. That allows adding custom logic and mocking in "conftest.py". In the example below, "tmp_path" fixture is requested for the "code- block" directive. Note: Note the "pytestfixture" directive "tmp_path" fixture. *Filename: README.rst* .. pytestfixture: tmp_path .. code-block:: python :name: test_path d = tmp_path / "sub" d.mkdir() # Create the directory assert d.is_dir() # Verify it was created and is a directory ====================================================================== In the example below, "tmp_path" fixture is requested for the "literalinclude" directive. *Filename: README.rst* .. pytestfixture: tmp_path .. literalinclude:: examples/python/tmp_path_example.py :name: test_li_tmp_path_example ====================================================================== Let's consider a sample openai code to ask LLM to tell a joke. In the example below, "openai_mock" fixture is requested for the "code-block" directive. Note: Note the "pytestfixture" directive "openai_mock". *Filename: README.rst* .. pytestfixture: openai_mock .. code-block:: python :name: test_tell_me_a_joke from openai import OpenAI client = OpenAI() completion = client.chat.completions.create( model="gpt-4o", messages=[ {"role": "developer", "content": "You are a famous comedian."}, {"role": "user", "content": "Tell me a joke."}, ], ) assert isinstance(completion.choices[0].message.content, str) Same could be applied to the "literalinclude" directive. *Filename: README.rst* .. pytestfixture: openai_mock .. literalinclude:: examples/python/tell_me_a_joke_example.py :name: test_li_tell_me_a_joke ====================================================================== Multiple "pytestfixture" directives are supported. Add one on each line. Note: When combining "pytestfixture" and "continue" directives together, request pytest-fixtures only in the first "code-block", as they will automatically become available in all continuing blocks. Custom pytest-fixtures are supported as well. Just define them in your "conftest.py" file. Customisation/hooks ------------------- Tests can be extended and fine-tuned using pytest's standard hook system. Below is an example workflow: 1. **Add custom pytest markers** to the "code-block" or "literalinclude" ("fakepy", "aws", "openai"). 2. **Implement pytest hooks** in "conftest.py" to react to those markers. Add custom pytest markers ~~~~~~~~~~~~~~~~~~~~~~~~~ Add "fakepy" marker """"""""""""""""""" The example code below will generate a PDF file with random text using fake.py library. Note, that a "fakepy" marker is added to the "code- block". In the >>`Implement pytest hooks`_<< section, you will see what can be done with the markers. Note: Note the "pytestmark" directive "fakepy" marker. *Filename: README.rst* .. pytestmark: fakepy .. code-block:: python :name: test_create_pdf_file from fake import FAKER FAKER.pdf_file() ====================================================================== In the example code below, a "fakepy" marker is added to the "literalinclude" block. *Filename: README.rst* .. pytestmark: fakepy .. literalinclude:: examples/python/create_pdf_file_example.py :name: test_li_create_pdf_file ====================================================================== Add "aws" marker """""""""""""""" Sample boto3 code to create a bucket on AWS S3. Note: Note the "pytestmark" directive "aws" marker. *Filename: README.rst* .. pytestmark: aws .. code-block:: python :name: test_create_bucket import boto3 s3 = boto3.client("s3", region_name="us-east-1") s3.create_bucket(Bucket="my-bucket") assert "my-bucket" in [b["Name"] for b in s3.list_buckets()["Buckets"]] ====================================================================== Implement pytest hooks ~~~~~~~~~~~~~~~~~~~~~~ In the example below: * moto is used to mock AWS S3 service for all tests marked as "aws". * "openai_mock" is used to mock OpenAI API for tests requiring that. * "FILE_REGISTRY.clean_up()" is executed at the end of each test marked as "fakepy". *Filename: conftest.py* import contextlib import json import os from pathlib import Path from types import SimpleNamespace import pytest import respx from fake import FILE_REGISTRY from moto import mock_aws from pytest_codeblock.constants import CODEBLOCK_MARK __author__ = "Artur Barseghyan " __copyright__ = "2025-2026 Artur Barseghyan" __license__ = "MIT" __all__ = ( "http_request", "http_request_factory", "markdown_simple", "markdown_with_pytest_mark", "openai_mock", "pytest_collection_modifyitems", "pytest_runtest_setup", "pytest_runtest_teardown", ) pytest_plugins = ["pytester"] # Modify test item during collection def pytest_collection_modifyitems( config: pytest.Config, items: list[pytest.Item], ) -> None: """Modify collected test items after collection is done. :param config: The pytest configuration object. :param items: A list of collected test items. """ for item in items: if item.get_closest_marker(CODEBLOCK_MARK): # Add `documentation` marker to `pytest-codeblock` tests item.add_marker(pytest.mark.documentation) if item.get_closest_marker("aws"): # Apply `mock_aws` to all tests marked as `aws` item.obj = mock_aws(item.obj) # Setup before test runs def pytest_runtest_setup(item: pytest.Item) -> None: """Set up test environment before each test runs. :param item: The test item that is about to run. """ # Teardown after the test ends def pytest_runtest_teardown(item: pytest.Item, nextitem: pytest.Item) -> None: """Tear down test environment after each test ends. :param item: The test item that just finished running. :param nextitem: The next test item that will run (or None if this is """ if item.get_closest_marker("fakepy"): FILE_REGISTRY.clean_up() @pytest.fixture def http_request_factory(): """ Returns a function that creates a simple namespace object with a 'GET' attribute set to the provided dictionary. """ def _factory(get_data: dict): # Creates an object like: object(GET={'key': 'value'}) return SimpleNamespace(GET=get_data) return _factory @pytest.fixture def http_request(http_request_factory): test_data = {"param1": "value1", "signature": "mock-sig"} return http_request_factory(test_data) @pytest.fixture def openai_mock(): # Setup os.environ.setdefault("OPENAI_API_KEY", "test-key") cassette_path = ( Path(__file__).parent / "examples" / "cassettes" / "openai_chat_completion.json" ) with open(cassette_path) as f: response_data = json.load(f) mock = respx.mock() mock.start() mock.post("https://api.openai.com/v1/chat/completions").respond( json=response_data, ) yield mock # Teardown with contextlib.suppress(Exception): mock.stop() @pytest.fixture def markdown_simple(): return """ ```python name=test_example x=1 assert x==1 ```""" @pytest.fixture def markdown_with_pytest_mark(): return """ ```python name=test_db from django.db import models ```""" @pytest.fixture def pytester_subprocess(pytester): """ Wrapper that forces subprocess mode to avoid deprecation warning conflicts when the plugin uses the old `path` argument signature. """ pytester.runpytest = pytester.runpytest_subprocess return pytester ====================================================================== Markdown ======== Usage examples -------------- Any fenced code block with a recognized Python language tag (e.g., "python", "py") will be collected and executed automatically, if your pytest configuration allows that. Standalone code blocks ~~~~~~~~~~~~~~~~~~~~~~ Note: Note that "name" value has a "test_" prefix. *Filename: README.md* ```python name=test_basic_example import math result = math.pow(3, 2) assert result == 9 ``` ====================================================================== Grouping multiple code blocks ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It's possible to split one logical test into multiple blocks. They will be tested under the first "name" specified. Note the "" directive. Note: Note that "continue" directive of the "test_grouping_example_part_2" and "test_grouping_example_part_3" refers to the "test_grouping_example". *Filename: README.md* ```python name=test_grouping_example x = 1 ``` Some intervening text. ```python name=test_grouping_example_part_2 y = x + 1 # Uses x from the first snippet assert y == 2 ``` Some intervening text. ```python name=test_grouping_example_part_3 print(y) # Uses y from the previous snippet ``` The above mentioned three snippets will run as a single test. Note: Note, that nameless code block can't be served as a first block in a group, as there is no way to refer to it. Nameless code blocks can only be used as continuing blocks in a group. ====================================================================== Async ~~~~~ You can use *top-level await* in your code blocks. The code will be automatically wrapped in an async function. *Filename: README.md* ```python name=test_async_example import asyncio result = await asyncio.sleep(0.1, result=42) assert result == 42 ``` ====================================================================== Adding pytest markers to code blocks ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It's possible to add custom pytest markers to your code blocks. That allows adding custom logic and mocking in your "conftest.py". In the example below, "django_db" marker is added to the code block. Note: Note the "pytestmark" directive "django_db" marker. *Filename: README.md* ```python name=test_django from django.contrib.auth.models import User user = User.objects.first() ``` Running pytest-style tests within code blocks ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The "pytestrun" marker allows code blocks to be executed as standalone pytest suites. Unlike standard code blocks that are simply executed with "exec()", blocks with the "pytestrun" marker support full pytest functionality including test classes, fixtures, and setup/teardown within documentation snippets. Note: Note the "pytestmark" directive "pytestrun" marker. *Filename: README.md* ```python name=test_pytestrun_example import pytest class TestSystemInfo: @pytest.fixture def system_name(self): return "Linux" @pytest.fixture def version_number(self): return 5 def test_combined_info(self, system_name, version_number): info = f"{system_name} v{version_number}" assert info == "Linux v5" def test_name_only(self, system_name): assert system_name.isalpha() ``` Requesting pytest fixtures for code blocks ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It's possible to request existing or custom pytest fixtures for code blocks. That allows adding custom logic and mocking in "conftest.py". In the example below, "tmp_path" fixture is requested for the code block. Note: Note the "pytestfixture" directive "tmp_path" fixture. *Filename: README.md* ```python name=test_path d = tmp_path / "sub" d.mkdir() # Create the directory assert d.is_dir() # Verify it was created and is a directory ``` ====================================================================== Let's consider a sample openai code to ask LLM to tell a joke. In the example below, "openai_mock" fixture is requested for the code block. Note: Note the "pytestfixture" directive "openai_mock" fixture. *Filename: README.md* ```python name=test_tell_me_a_joke from openai import OpenAI client = OpenAI() completion = client.chat.completions.create( model="gpt-4o", messages=[ {"role": "developer", "content": "You are a famous comedian."}, {"role": "user", "content": "Tell me a joke."}, ], ) assert isinstance(completion.choices[0].message.content, str) ``` ====================================================================== Multiple "pytestfixture" directives are supported. Add one on each line. Note: When combining "pytestfixture" and "continue" directives together, request pytest-fixtures only in the first "code-block", as they will automatically become available in all continuing blocks. Custom pytest-fixtures are supported as well. Just define them in your "conftest.py" file. Customisation/hooks ------------------- Tests can be extended and fine-tuned using pytest's standard hook system. Below is an example workflow: 1. **Add custom pytest markers** to the code blocks ("fakepy", "aws", "openai"). 2. **Implement pytest hooks** in "conftest.py" to react to those markers. Add custom pytest markers ~~~~~~~~~~~~~~~~~~~~~~~~~ Add "fakepy" marker """"""""""""""""""" The example code below will generate a PDF file with random text using fake.py library. Note, that a "fakepy" marker is added to the code block. In the >>`Implement pytest hooks`_<< section, you will see what can be done with the markers. Note: Note the "pytestmark" directive "fakepy" marker. *Filename: README.md* ```python name=test_create_pdf_file from fake import FAKER FAKER.pdf_file() ``` Add "aws" marker """""""""""""""" Sample boto3 code to create a bucket on AWS S3. Note: Note the "pytestmark" directive "aws" marker. *Filename: README.md* ```python name=test_create_bucket import boto3 s3 = boto3.client("s3", region_name="us-east-1") s3.create_bucket(Bucket="my-bucket") assert "my-bucket" in [b["Name"] for b in s3.list_buckets()["Buckets"]] ``` ====================================================================== Implement pytest hooks ~~~~~~~~~~~~~~~~~~~~~~ In the example below: * moto is used to mock AWS S3 service for all tests marked as "aws". * "openai_mock" is used to mock OpenAI API for tests requiring that. * "FILE_REGISTRY.clean_up()" is executed at the end of each test marked as "fakepy". *Filename: conftest.py* import contextlib import json import os from pathlib import Path from types import SimpleNamespace import pytest import respx from fake import FILE_REGISTRY from moto import mock_aws from pytest_codeblock.constants import CODEBLOCK_MARK __author__ = "Artur Barseghyan " __copyright__ = "2025-2026 Artur Barseghyan" __license__ = "MIT" __all__ = ( "http_request", "http_request_factory", "markdown_simple", "markdown_with_pytest_mark", "openai_mock", "pytest_collection_modifyitems", "pytest_runtest_setup", "pytest_runtest_teardown", ) pytest_plugins = ["pytester"] # Modify test item during collection def pytest_collection_modifyitems( config: pytest.Config, items: list[pytest.Item], ) -> None: """Modify collected test items after collection is done. :param config: The pytest configuration object. :param items: A list of collected test items. """ for item in items: if item.get_closest_marker(CODEBLOCK_MARK): # Add `documentation` marker to `pytest-codeblock` tests item.add_marker(pytest.mark.documentation) if item.get_closest_marker("aws"): # Apply `mock_aws` to all tests marked as `aws` item.obj = mock_aws(item.obj) # Setup before test runs def pytest_runtest_setup(item: pytest.Item) -> None: """Set up test environment before each test runs. :param item: The test item that is about to run. """ # Teardown after the test ends def pytest_runtest_teardown(item: pytest.Item, nextitem: pytest.Item) -> None: """Tear down test environment after each test ends. :param item: The test item that just finished running. :param nextitem: The next test item that will run (or None if this is """ if item.get_closest_marker("fakepy"): FILE_REGISTRY.clean_up() @pytest.fixture def http_request_factory(): """ Returns a function that creates a simple namespace object with a 'GET' attribute set to the provided dictionary. """ def _factory(get_data: dict): # Creates an object like: object(GET={'key': 'value'}) return SimpleNamespace(GET=get_data) return _factory @pytest.fixture def http_request(http_request_factory): test_data = {"param1": "value1", "signature": "mock-sig"} return http_request_factory(test_data) @pytest.fixture def openai_mock(): # Setup os.environ.setdefault("OPENAI_API_KEY", "test-key") cassette_path = ( Path(__file__).parent / "examples" / "cassettes" / "openai_chat_completion.json" ) with open(cassette_path) as f: response_data = json.load(f) mock = respx.mock() mock.start() mock.post("https://api.openai.com/v1/chat/completions").respond( json=response_data, ) yield mock # Teardown with contextlib.suppress(Exception): mock.stop() @pytest.fixture def markdown_simple(): return """ ```python name=test_example x=1 assert x==1 ```""" @pytest.fixture def markdown_with_pytest_mark(): return """ ```python name=test_db from django.db import models ```""" @pytest.fixture def pytester_subprocess(pytester): """ Wrapper that forces subprocess mode to avoid deprecation warning conflicts when the plugin uses the old `path` argument signature. """ pytester.runpytest = pytester.runpytest_subprocess return pytester ====================================================================== Customisation ============= It's possible to customise which codeblock languages and file extensions are recognised by the plugin. ====================================================================== Languages --------- By default, the plugin recognises the following codeblock languages: * reStructuredText: *python*, *py*, *python3* * Markdown: *python*, *py*, *python3* reStructruredText ~~~~~~~~~~~~~~~~~ For reStructruredText defaults are configured via *rst_codeblocks* setting in the *[tool.pytest-codeblock]* section of your *pyproject.toml*. [tool.pytest-codeblock] rst_codeblocks = ["python", "py", "python3"] Note: Don't touch the defaults, unless you want to remove certain options. If you only want to add custom codeblock languages, use *rst_user_codeblocks*. The following example adds *c_py* as a custom codeblock language: [tool.pytest-codeblock] rst_user_codeblocks = ["c_py"] Now the following codeblock will be recognised and executed: .. code-block:: c_py :name: test_c_py_example print("This is a custom Python codeblock") Markdown ~~~~~~~~ For Markdown defaults configured via *md_codeblocks* setting in the *[tool.pytest-codeblock]* section of your *pyproject.toml*. [tool.pytest-codeblock] md_codeblocks = ["python", "py", "python3"] Note: Don't touch the defaults, unless you want to remove certain options. If you only want to add custom codeblock languages, use *md_user_codeblocks*. The following example adds *c_py* as a custom codeblock language: [tool.pytest-codeblock] md_user_codeblocks = ["c_py"] Now the following codeblock will be recognised and executed: ```c_py name=test_c_py_example print("This is a custom Python codeblock") ``` ====================================================================== Extensions ---------- Note: If you customise both reStructuredText and Markdown configurations, make sure to avoid overlapping file extensions. reStructruredText ~~~~~~~~~~~~~~~~~ By default, the plugin recognises the following file extensions for reStructuredText files: *.rst* These defaults are configured via *rst_extensions* setting in the *[tool.pytest-codeblock]* section of your *pyproject.toml*. [tool.pytest-codeblock] rst_extensions = [".rst"] Note: Don't touch the defaults, unless you want to remove certain options. If you only want to add custom file extensions, use *rst_user_extensions*. The following example adds *.rst.txt* as a custom reStructuredText file extension: [tool.pytest-codeblock] rst_user_extensions = [".rst.txt"] Now the following file will be recognised and processed: *Filename: example.rst.txt* .. code-block:: python :name: test_custom_rst_extension_example print("Custom .rst.txt extension example executed successfully!") Markdown ~~~~~~~~ By default, the plugin recognises the following file extensions for Markdown files: *.md*, *.markdown* These defaults are configured via *md_extensions* setting in the *[tool.pytest-codeblock]* section of your *pyproject.toml*. [tool.pytest-codeblock] md_extensions = [".md", ".markdown"] Note: Don't touch the defaults, unless you want to remove certain options. If you only want to add custom file extensions, use *md_user_extensions*. The following example adds *.md.txt* as a custom Markdown file extension: [tool.pytest-codeblock] md_user_extensions = [".md.txt"] Now the following file will be recognised and processed: *Filename: example.md.txt* ```python name=test_custom_md_extension_example print("Custom .md.txt extension example executed successfully!") ``` ====================================================================== Project source-tree =================== Below is the layout of the project (to 10 levels), followed by the contents of each key file. Project directory layout pytest-codeblock/ ├── docs │ ├── cheatsheet_markdown.rst │ ├── cheatsheet_restructured_text.rst │ ├── conf.py │ ├── customisation.rst │ ├── markdown.rst │ ├── quick_start_ref.rst │ ├── requirements.txt │ └── restructured_text.rst ├── src │ └── pytest_codeblock │ ├── tests │ │ ├── __init__.py │ │ ├── test_customisation.py │ │ ├── test_integration.py │ │ ├── test_nameless_codeblocks.py │ │ ├── test_pytest_codeblock.py │ │ ├── test_pytestrun_marker.py │ │ ├── tests.md │ │ └── tests.rst │ ├── __init__.py │ ├── collector.py │ ├── config.py │ ├── constants.py │ ├── helpers.py │ ├── md.py │ ├── pytestrun.py │ └── rst.py ├── conftest.py ├── CONTRIBUTING.rst ├── Makefile ├── pyproject.toml └── README.rst README.rst ---------- README.rst ================ pytest-codeblock ================ .. External references .. _reStructuredText: https://docutils.sourceforge.io/rst.html .. _Markdown: https://daringfireball.net/projects/markdown/ .. _pytest: https://docs.pytest.org .. _Django: https://www.djangoproject.com .. _pip: https://pypi.org/project/pip/ .. _uv: https://pypi.org/project/uv/ .. _fake.py: https://github.com/barseghyanartur/fake.py .. _boto3: https://github.com/boto/boto3 .. _moto: https://github.com/getmoto/moto .. _openai: https://github.com/openai/openai-python .. _Ollama: https://github.com/ollama/ollama .. _tomli: https://pypi.org/project/tomli/ .. _doc8: https://doc8.readthedocs.io/ .. Internal references .. _pytest-codeblock: https://github.com/barseghyanartur/pytest-codeblock/ .. _Read the Docs: http://pytest-codeblock.readthedocs.io/ .. _Examples: https://github.com/barseghyanartur/pytest-codeblock/tree/main/examples .. _Customisation docs: https://pytest-codeblock.readthedocs.io/en/latest/customisation.html .. _Contributor guidelines: https://pytest-codeblock.readthedocs.io/en/latest/contributor_guidelines.html .. _reStructuredText docs: https://pytest-codeblock.readthedocs.io/en/latest/restructured_text.html .. _Markdown docs: https://pytest-codeblock.readthedocs.io/en/latest/markdown.html .. _llms.txt: https://barseghyanartur.github.io/pytest-codeblock/llms.txt Test your documentation code blocks. .. image:: https://img.shields.io/pypi/v/pytest-codeblock.svg :target: https://pypi.python.org/pypi/pytest-codeblock :alt: PyPI Version .. image:: https://img.shields.io/pypi/pyversions/pytest-codeblock.svg :target: https://pypi.python.org/pypi/pytest-codeblock/ :alt: Supported Python versions .. image:: https://github.com/barseghyanartur/pytest-codeblock/actions/workflows/test.yml/badge.svg?branch=main :target: https://github.com/barseghyanartur/pytest-codeblock/actions :alt: Build Status .. image:: https://readthedocs.org/projects/pytest-codeblock/badge/?version=latest :target: http://pytest-codeblock.readthedocs.io :alt: Documentation Status .. image:: https://img.shields.io/badge/docs-llms.txt-blue :target: http://pytest-codeblock.readthedocs.io/en/latest/llms.txt :alt: llms.txt - documentation for LLMs .. image:: https://img.shields.io/badge/license-MIT-blue.svg :target: https://github.com/barseghyanartur/pytest-codeblock/#License :alt: MIT .. image:: https://coveralls.io/repos/github/barseghyanartur/pytest-codeblock/badge.svg?branch=main&service=github :target: https://coveralls.io/github/barseghyanartur/pytest-codeblock?branch=main :alt: Coverage `pytest-codeblock`_ is a `Pytest`_ plugin that discovers Python code examples in your `reStructuredText`_ and `Markdown`_ documentation files and runs them as part of your test suite. This ensures your docs stay correct and up-to-date. Features ======== - **reStructuredText and Markdown support**: Automatically find and test code blocks in `reStructuredText`_ (``.rst``) and `Markdown`_ (``.md``) files. Async code snippets are supported as well. - **Grouping**: Split a single example across multiple code blocks; the plugin concatenates them into one test. - **Pytest markers support**: Add existing or custom `pytest`_ markers to the code blocks and hook into the tests life-cycle using ``conftest.py``. - **Pytest fixtures support**: Request existing or custom `pytest`_ fixtures for the code blocks. Prerequisites ============= - Python 3.10+ - `pytest`_ is the only required dependency (on Python 3.11+; for Python 3.10 `tomli`_ is also required). Documentation ============= - Documentation is available on `Read the Docs`_. - For `reStructuredText`, see a dedicated `reStructuredText docs`_. - For `Markdown`, see a dedicated `Markdown docs`_. - Both `reStructuredText docs`_ and `Markdown docs`_ have extensive documentation on `pytest`_ markers and corresponding ``conftest.py`` hooks. - For guidelines on contributing check the `Contributor guidelines`_. Installation ============ Install with `pip`_: .. code-block:: sh pip install pytest-codeblock Or install with `uv`_: .. code-block:: sh uv pip install pytest-codeblock .. _configuration: Configuration ============= For most use cases, no configuration needed. By default, all code blocks with a name starting with ``test_`` will be collected and executed as tests. This allows you to have both test and non-test code blocks in your documentation, giving you flexibility in how you structure your examples. However, if you want to test all code blocks, you can set ``test_nameless_codeblocks`` to ``true`` in your `pyproject.toml`: *Filename: pyproject.toml* .. code-block:: toml [tool.pytest-codeblock] test_nameless_codeblocks = true If you still want to skip some code blocks, you can use built-in or custom pytest markers. See the dedicated `reStructuredText docs`_ and `Markdown docs`_ to learn more about `pytestmark` directive. Note, that nameless code blocks have limitations when it comes to grouping. ---- By default, all code `.rst` and `.md` files shall be picked automatically. However, if you need to add another file extension or use or another language identifier for python in codeblock, you could configure that. See the following example of `pyproject.toml` configuration: *Filename: pyproject.toml* .. code-block:: toml [tool.pytest-codeblock] rst_user_codeblocks = ["c_py"] rst_user_extensions = [".rst.txt"] md_user_codeblocks = ["c_py"] md_user_extensions = [".md.txt"] See `customisation docs`_ for more. Usage ===== .. note:: It's highly recommended to use `doc8`_ for catching possible markup errors, that otherwise would be difficult to spot. reStructruredText usage ----------------------- Any code directive, such as ``.. code-block:: python``, ``.. code:: python``, or literal blocks with a preceding ``.. codeblock-name: ``, will be collected and executed automatically by `pytest`_. ``code-block`` directive example ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. note:: Note that ``:name:`` value has a ``test_`` prefix. *Filename: README.rst* .. code-block:: rst .. code-block:: python :name: test_basic_example import math result = math.pow(3, 2) assert result == 9 ``literalinclude`` directive example ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. note:: Note that ``:name:`` value has a ``test_`` prefix. *Filename: README.rst* .. code-block:: rst .. literalinclude:: examples/python/basic_example.py :name: test_li_basic_example See a dedicated `reStructuredText docs`_ for more. Markdown usage -------------- Any fenced code block with a recognized Python language tag (e.g., ``python``, ``py``) will be collected and executed automatically by `pytest`_. .. note:: Note that ``name`` value has a ``test_`` prefix. *Filename: README.md* .. code-block:: markdown ```python name=test_basic_example import math result = math.pow(3, 2) assert result == 9 ``` See a dedicated `Markdown docs`_ for more. Tests ===== Run the tests with `pytest`_: .. code-block:: sh pytest Troubleshooting =============== If something doesn't work, try to add this to your pyproject.toml: *Filename: pyproject.toml* .. code-block:: text [tool.pytest.ini_options] testpaths = [ "**/*.rst", "**/*.md", ] Writing documentation ===================== Keep the following hierarchy. .. code-block:: text ===== title ===== header ====== sub-header ---------- sub-sub-header ~~~~~~~~~~~~~~ sub-sub-sub-header ^^^^^^^^^^^^^^^^^^ sub-sub-sub-sub-header ++++++++++++++++++++++ sub-sub-sub-sub-sub-header ************************** License ======= MIT Support ======= For security issues contact me at the e-mail given in the `Author`_ section. For overall issues, go to `GitHub `_. Author ====== Artur Barseghyan CONTRIBUTING.rst ---------------- CONTRIBUTING.rst Contributor guidelines ====================== .. _pytest-codeblock: https://pytest-codeblock.readthedocs.io .. _documentation: https://pytest-codeblock.readthedocs.io/#writing-documentation .. _testing: https://pytest-codeblock.readthedocs.io/#testing .. _pre-commit: https://pre-commit.com/#installation .. _black: https://black.readthedocs.io/ .. _isort: https://pycqa.github.io/isort/ .. _doc8: https://doc8.readthedocs.io/ .. _ruff: https://beta.ruff.rs/docs/ .. _pip-tools: https://pip-tools.readthedocs.io/ .. _uv: https://docs.astral.sh/uv/ .. _tox: https://tox.wiki .. _issues: https://github.com/barseghyanartur/pytest-codeblock/issues .. _discussions: https://github.com/barseghyanartur/pytest-codeblock/discussions .. _pull request: https://github.com/barseghyanartur/pytest-codeblock/pulls .. _support: https://pytest-codeblock.readthedocs.io/#support .. _installation: https://pytest-codeblock.readthedocs.io/#installation .. _features: https://pytest-codeblock.readthedocs.io/#features .. _prerequisites: https://fakepy.readthedocs.io/#prerequisites .. _versions manifest: https://github.com/actions/python-versions/blob/main/versions-manifest.json Developer prerequisites ----------------------- pre-commit ~~~~~~~~~~ Refer to `pre-commit`_ for installation instructions. TL;DR: .. code-block:: sh curl -LsSf https://astral.sh/uv/install.sh | sh # Install uv uv tool install pre-commit # Install pre-commit pre-commit install # Install pre-commit hooks Installing `pre-commit`_ will ensure you adhere to the project code quality standards. Code standards -------------- `ruff`_ and `doc8`_ will be automatically triggered by `pre-commit`_. `ruff`_ is configured to do the job of `black`_ and `isort`_ as well. Still, if you want to run checks manually: .. code-block:: sh make doc8 make ruff Virtual environment ------------------- You are advised to work in virtual environment. TL;DR: .. code-block:: sh uv sync --all-extras Documentation ------------- Check the `documentation`_. For building documentation locally: .. code-block:: sh make build-docs For running documentation locally on port 5001: .. code-block:: sh make serve-docs Requirements are compiled using `uv`_ (to be used by ReadTheDocs). .. code-block:: sh make compile-requirements Testing ------- Check `testing`_. If you introduce changes or fixes, make sure to test them locally using all supported environments. For that use `tox`_. .. code-block:: sh tox In any case, GitHub Actions will catch potential errors, but using tox speeds things up. For a quick test of the package and all examples, use the following `Makefile` command: .. code-block:: sh make test Pull requests ------------- You can contribute to the project by making a `pull request`_. .. note:: Create a pull requests to the `dev` branch only! Never to `main` directly. For example: - To fix documentation typos. - To improve documentation (for instance, to add new recipe or fix an existing recipe that doesn't seem to work). - To introduce a new feature (for instance, add support for a non-supported file type). **Good to know:** - This library is almost dependency free. Do not submit pull requests with external dependencies unless it's really necessary. **General list to go through:** - Does your change require documentation update? - Does your change require update to tests? - Does your change rely on third-party package or a cloud based service? If so, please consider turning it into a dedicated standalone package, since this library is dependency free (and will always stay so). **When fixing bugs (in addition to the general list):** - Make sure to add regression tests. **When adding a new feature (in addition to the general list):** - Make sure to update the documentation (check whether the `installation`_ and `features`_ require changes). GitHub Actions -------------- Only non-EOL versions of Python and software `pytest-codeblock`_ aims to integrate with are supported. On GitHub Actions includes tests for more than 40 different variations of Python versions and integration packages. Future, non-stable versions of Python are being tested too, so that new features/incompatibilities could be seen and adopted early. For the list of Python versions supported by GitHub, see GitHub Actions `versions manifest`_. Questions --------- Questions can be asked on GitHub `discussions`_. Issues ------ For reporting a bug or filing a feature request, use GitHub `issues`_. **Do not report security issues on GitHub**. Check the `support`_ section. docs/quick_start_ref.rst ------------------------ docs/quick_start_ref.rst Quick-start reference ===================== pytest-codeblock collects Python code blocks from ``.rst`` and ``.md`` files and runs them as pytest tests. No configuration is required for basic use — files are discovered automatically. -------------- Naming rules ------------ Only blocks whose name starts with ``test_`` are collected by default. To test all blocks regardless of name, set ``test_nameless_codeblocks = true`` in ``pyproject.toml``. -------------- RST syntax ---------- .. code:: rst .. code-block:: python :name: test_my_example result = 1 + 1 assert result == 2 Add a pytest marker (RST) ~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: rst .. pytestmark: skip .. code-block:: python :name: test_skipped_block pass Request a fixture (RST) ~~~~~~~~~~~~~~~~~~~~~~~ .. code:: rst .. pytestfixture: tmp_path .. code-block:: python :name: test_uses_tmp_path d = tmp_path / "sub" d.mkdir() assert d.is_dir() Group blocks (RST) — shared context ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: rst .. code-block:: python :name: test_part_one x = 1 Some prose in between. .. continue: test_part_one .. code-block:: python :name: test_part_two y = x + 1 assert y == 2 All blocks sharing the same group key are concatenated into one test under the first name. Incremental grouping — each step is its own test ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ When every continuation block has a **distinct** name, each step becomes a cumulative test: .. code:: rst .. code-block:: python :name: test_step_1 a = 1 .. continue: test_step_1 .. code-block:: python :name: test_step_2 b = a + 1 assert b == 2 This produces two tests: ``test_step_1`` (code: ``a=1``) and ``test_step_2`` (code: ``a=1\nb=a+1\nassert b==2``). Literal block (RST) ~~~~~~~~~~~~~~~~~~~ .. code:: rst .. codeblock-name: test_literal_block Example code:: result = "hello" assert result == "hello" Include external file (RST) ~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: rst .. literalinclude:: examples/snippet.py :name: test_external_snippet Run as standalone pytest suite (RST) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: rst .. pytestmark: pytestrun .. code-block:: python :name: test_pytest_style import pytest class TestMath: def test_add(self): assert 1 + 1 == 2 -------------- Markdown syntax --------------- .. code:: markdown ```python name=test_my_example result = 1 + 1 assert result == 2 ``` Add a pytest marker (Markdown) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: markdown ```python name=test_skipped pass ``` Request a fixture (Markdown) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: markdown ```python name=test_uses_tmp_path d = tmp_path / "sub" d.mkdir() assert d.is_dir() ``` Group blocks (Markdown) ~~~~~~~~~~~~~~~~~~~~~~~ .. code:: markdown ```python name=test_setup x = 1 ``` ```python name=test_continuation y = x + 1 assert y == 2 ``` Run as standalone pytest suite (Markdown) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code:: markdown ```python name=test_class_example import pytest class TestMath: @pytest.fixture def value(self): return 42 def test_value(self, value): assert value == 42 ``` -------------- Async support ------------- Top-level ``await`` is automatically wrapped — no extra config needed: .. code:: rst .. code-block:: python :name: test_async_block import asyncio result = await asyncio.sleep(0.1, result=99) assert result == 99 -------------- pyproject.toml configuration ---------------------------- .. code:: toml [tool.pytest-codeblock] # Test all blocks regardless of test_ prefix (default: false) test_nameless_codeblocks = false # Add custom language identifiers (in addition to python, py, python3) rst_user_codeblocks = [] md_user_codeblocks = [] # Add custom file extensions rst_user_extensions = [] md_user_extensions = [] testpaths troubleshooting ------------------------- If docs are not discovered, add explicitly: .. code:: toml [tool.pytest.ini_options] testpaths = ["src/app/tests", "docs"] -------------- conftest.py hook integration ---------------------------- Use ``CODEBLOCK_MARK`` from ``pytest_codeblock.constants`` to identify doc-block tests: .. code:: python from pytest_codeblock.constants import CODEBLOCK_MARK def pytest_collection_modifyitems(config, items): for item in items: if item.get_closest_marker(CODEBLOCK_MARK): item.add_marker(pytest.mark.documentation) Custom fixtures used in doc blocks are defined in ``conftest.py`` exactly like regular fixtures. Multiple ``pytestfixture`` directives on consecutive lines are all applied to the next block. Fixture requests in the first block of a group automatically apply to all continuation blocks. docs/restructured_text.rst -------------------------- docs/restructured_text.rst reStructuredText ================ .. External references .. _reStructuredText: https://docutils.sourceforge.io/rst.html .. _pytest: https://docs.pytest.org .. _Django: https://www.djangoproject.com .. _pip: https://pypi.org/project/pip/ .. _uv: https://pypi.org/project/uv/ .. _fake.py: https://github.com/barseghyanartur/fake.py .. _boto3: https://github.com/boto/boto3 .. _moto: https://github.com/getmoto/moto .. _openai: https://github.com/openai/openai-python .. _Ollama: https://github.com/ollama/ollama The following directives are supported: - ``.. code-block:: python`` - ``.. code:: python`` - ``.. codeblock-name: `` - ``.. literalinclude::`` Any code directive, such as ``.. code-block:: python``, ``.. code:: python``, ``.. literalinclude::`` or literal blocks with a preceding ``.. codeblock-name: ``, will be collected and executed automatically, if your `pytest`_ :ref:`configuration ` allows that. Usage examples -------------- Standalone code blocks ~~~~~~~~~~~~~~~~~~~~~~ ``code-block`` directive ^^^^^^^^^^^^^^^^^^^^^^^^ .. note:: Note that ``:name:`` value has a ``test_`` prefix. *Filename: README.rst* .. code-block:: rst .. code-block:: python :name: test_basic_example import math result = math.pow(3, 2) assert result == 9 ---- ``literalinclude`` directive ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ *Filename: README.rst* .. code-block:: rst .. literalinclude:: examples/python/basic_example.py :name: test_li_basic_example ---- ``codeblock-name`` directive ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ You can also use a literal block with a preceding name comment: *Filename: README.rst* .. code-block:: rst .. codeblock-name: test_grouping_example_literal_block This is a literal block:: y = 5 print(y * 2) ---- Grouping multiple ``code-block`` directives ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It's possible to split one logical test into multiple blocks. They will be tested under the first ``:name:`` specified. Note the ``.. continue::`` directive. .. note:: Note that ``continue`` directive of the ``test_grouping_example_part_2`` and ``test_grouping_example_part_3`` refers to the ``test_grouping_example``. *Filename: README.rst* .. code-block:: rst .. code-block:: python :name: test_grouping_example x = 1 Some intervening text. .. continue: test_grouping_example .. code-block:: python :name: test_grouping_example_part_2 y = x + 1 # Uses x from the first snippet assert y == 2 Some intervening text. .. continue: test_grouping_example .. code-block:: python :name: test_grouping_example_part_3 print(y) # Uses y from the previous snippet The above mentioned three snippets will run as a single test. .. note:: Note, that nameless code block can't be served as a first block in a group, as there is no way to refer to it. Nameless code blocks can only be used as continuing blocks in a group. ---- Async ~~~~~ You can use `top-level await` in your code blocks. The code will be automatically wrapped in an async function. *Filename: README.rst* .. code-block:: rst .. code-block:: python :name: test_async_example import asyncio result = await asyncio.sleep(0.1, result=42) assert result == 42 ---- Adding pytest markers to ``code-block`` and ``literalinclude`` directives ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It's possible to add custom pytest markers to your ``code-block`` or ``literalinclude`` directives. That allows adding custom logic and mocking in your ``conftest.py``. In the example below, ``django_db`` marker is added to the ``code-block`` directive. .. note:: Note the ``pytestmark`` directive ``django_db`` marker. *Filename: README.rst* .. code-block:: rst .. pytestmark: django_db .. code-block:: python :name: test_django from django.contrib.auth.models import User user = User.objects.first() ---- Running pytest-style tests within code blocks ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ``pytestrun`` marker allows code blocks to be executed as standalone pytest suites. Unlike standard code blocks that are simply executed with ``exec()``, blocks with the ``pytestrun`` marker support full pytest functionality including test classes, fixtures, and setup/teardown within documentation snippets. .. note:: Note the ``pytestmark`` directive ``pytestrun`` marker. *Filename: README.rst* .. code-block:: rst .. pytestmark: pytestrun .. code-block:: python :name: test_pytestrun_example import pytest class TestSystemInfo: @pytest.fixture def system_name(self): return "Linux" @pytest.fixture def version_number(self): return 5 def test_combined_info(self, system_name, version_number): info = f"{system_name} v{version_number}" assert info == "Linux v5" def test_name_only(self, system_name): assert system_name.isalpha() ---- In the example below, ``django_db`` marker is added to the ``literalinclude`` directive. *Filename: README.rst* .. code-block:: rst .. pytestmark: django_db .. literalinclude:: examples/python/django_example.py :name: test_li_django_example ---- Requesting pytest fixtures for ``code-block`` and ``literalinclude`` directives ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It's possible to request existing or custom pytest fixtures in ``code-block`` or ``literalinclude`` directives. That allows adding custom logic and mocking in ``conftest.py``. In the example below, ``tmp_path`` fixture is requested for the ``code-block`` directive. .. note:: Note the ``pytestfixture`` directive ``tmp_path`` fixture. *Filename: README.rst* .. code-block:: rst .. pytestfixture: tmp_path .. code-block:: python :name: test_path d = tmp_path / "sub" d.mkdir() # Create the directory assert d.is_dir() # Verify it was created and is a directory ---- In the example below, ``tmp_path`` fixture is requested for the ``literalinclude`` directive. *Filename: README.rst* .. code-block:: rst .. pytestfixture: tmp_path .. literalinclude:: examples/python/tmp_path_example.py :name: test_li_tmp_path_example ---- Let's consider a sample `openai`_ code to ask LLM to tell a joke. In the example below, ``openai_mock`` fixture is requested for the ``code-block`` directive. .. note:: Note the ``pytestfixture`` directive ``openai_mock``. *Filename: README.rst* .. code-block:: rst .. pytestfixture: openai_mock .. code-block:: python :name: test_tell_me_a_joke from openai import OpenAI client = OpenAI() completion = client.chat.completions.create( model="gpt-4o", messages=[ {"role": "developer", "content": "You are a famous comedian."}, {"role": "user", "content": "Tell me a joke."}, ], ) assert isinstance(completion.choices[0].message.content, str) Same could be applied to the ``literalinclude`` directive. *Filename: README.rst* .. code-block:: rst .. pytestfixture: openai_mock .. literalinclude:: examples/python/tell_me_a_joke_example.py :name: test_li_tell_me_a_joke ---- Multiple ``pytestfixture`` directives are supported. Add one on each line. .. note:: When combining ``pytestfixture`` and ``continue`` directives together, request pytest-fixtures only in the first ``code-block``, as they will automatically become available in all continuing blocks. Custom pytest-fixtures are supported as well. Just define them in your ``conftest.py`` file. Customisation/hooks ------------------- Tests can be extended and fine-tuned using `pytest`_'s standard hook system. Below is an example workflow: 1. **Add custom pytest markers** to the ``code-block`` or ``literalinclude`` (``fakepy``, ``aws``, ``openai``). 2. **Implement pytest hooks** in ``conftest.py`` to react to those markers. Add custom pytest markers ~~~~~~~~~~~~~~~~~~~~~~~~~ Add ``fakepy`` marker ^^^^^^^^^^^^^^^^^^^^^ The example code below will generate a PDF file with random text using `fake.py`_ library. Note, that a ``fakepy`` marker is added to the ``code-block``. In the `Implement pytest hooks`_ section, you will see what can be done with the markers. .. note:: Note the ``pytestmark`` directive ``fakepy`` marker. *Filename: README.rst* .. code-block:: rst .. pytestmark: fakepy .. code-block:: python :name: test_create_pdf_file from fake import FAKER FAKER.pdf_file() ---- In the example code below, a ``fakepy`` marker is added to the ``literalinclude`` block. *Filename: README.rst* .. code-block:: rst .. pytestmark: fakepy .. literalinclude:: examples/python/create_pdf_file_example.py :name: test_li_create_pdf_file ---- Add ``aws`` marker ^^^^^^^^^^^^^^^^^^ Sample `boto3`_ code to create a bucket on AWS S3. .. note:: Note the ``pytestmark`` directive ``aws`` marker. *Filename: README.rst* .. code-block:: rst .. pytestmark: aws .. code-block:: python :name: test_create_bucket import boto3 s3 = boto3.client("s3", region_name="us-east-1") s3.create_bucket(Bucket="my-bucket") assert "my-bucket" in [b["Name"] for b in s3.list_buckets()["Buckets"]] ---- Implement pytest hooks ~~~~~~~~~~~~~~~~~~~~~~ .. include:: _implement_pytest_hooks.rst docs/markdown.rst ----------------- docs/markdown.rst Markdown ======== .. External references .. _Markdown: https://daringfireball.net/projects/markdown/ .. _pytest: https://docs.pytest.org .. _Django: https://www.djangoproject.com .. _pip: https://pypi.org/project/pip/ .. _uv: https://pypi.org/project/uv/ .. _fake.py: https://github.com/barseghyanartur/fake.py .. _boto3: https://github.com/boto/boto3 .. _moto: https://github.com/getmoto/moto .. _openai: https://github.com/openai/openai-python .. _Ollama: https://github.com/ollama/ollama Usage examples -------------- Any fenced code block with a recognized Python language tag (e.g., ``python``, ``py``) will be collected and executed automatically, if your `pytest`_ :ref:`configuration ` allows that. Standalone code blocks ~~~~~~~~~~~~~~~~~~~~~~ .. note:: Note that ``name`` value has a ``test_`` prefix. *Filename: README.md* .. code-block:: markdown ```python name=test_basic_example import math result = math.pow(3, 2) assert result == 9 ``` ---- Grouping multiple code blocks ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It's possible to split one logical test into multiple blocks. They will be tested under the first ``name`` specified. Note the ```` directive. .. note:: Note that ``continue`` directive of the ``test_grouping_example_part_2`` and ``test_grouping_example_part_3`` refers to the ``test_grouping_example``. *Filename: README.md* .. code-block:: markdown ```python name=test_grouping_example x = 1 ``` Some intervening text. ```python name=test_grouping_example_part_2 y = x + 1 # Uses x from the first snippet assert y == 2 ``` Some intervening text. ```python name=test_grouping_example_part_3 print(y) # Uses y from the previous snippet ``` The above mentioned three snippets will run as a single test. .. note:: Note, that nameless code block can't be served as a first block in a group, as there is no way to refer to it. Nameless code blocks can only be used as continuing blocks in a group. ---- Async ~~~~~ You can use `top-level await` in your code blocks. The code will be automatically wrapped in an async function. *Filename: README.md* .. code-block:: markdown ```python name=test_async_example import asyncio result = await asyncio.sleep(0.1, result=42) assert result == 42 ``` ---- Adding pytest markers to code blocks ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It's possible to add custom pytest markers to your code blocks. That allows adding custom logic and mocking in your ``conftest.py``. In the example below, ``django_db`` marker is added to the code block. .. note:: Note the ``pytestmark`` directive ``django_db`` marker. *Filename: README.md* .. code-block:: markdown ```python name=test_django from django.contrib.auth.models import User user = User.objects.first() ``` Running pytest-style tests within code blocks ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ``pytestrun`` marker allows code blocks to be executed as standalone pytest suites. Unlike standard code blocks that are simply executed with ``exec()``, blocks with the ``pytestrun`` marker support full pytest functionality including test classes, fixtures, and setup/teardown within documentation snippets. .. note:: Note the ``pytestmark`` directive ``pytestrun`` marker. *Filename: README.md* .. code-block:: markdown ```python name=test_pytestrun_example import pytest class TestSystemInfo: @pytest.fixture def system_name(self): return "Linux" @pytest.fixture def version_number(self): return 5 def test_combined_info(self, system_name, version_number): info = f"{system_name} v{version_number}" assert info == "Linux v5" def test_name_only(self, system_name): assert system_name.isalpha() ``` Requesting pytest fixtures for code blocks ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It's possible to request existing or custom pytest fixtures for code blocks. That allows adding custom logic and mocking in ``conftest.py``. In the example below, ``tmp_path`` fixture is requested for the code block. .. note:: Note the ``pytestfixture`` directive ``tmp_path`` fixture. *Filename: README.md* .. code-block:: markdown ```python name=test_path d = tmp_path / "sub" d.mkdir() # Create the directory assert d.is_dir() # Verify it was created and is a directory ``` ---- Let's consider a sample `openai`_ code to ask LLM to tell a joke. In the example below, ``openai_mock`` fixture is requested for the code block. .. note:: Note the ``pytestfixture`` directive ``openai_mock`` fixture. *Filename: README.md* .. code-block:: markdown ```python name=test_tell_me_a_joke from openai import OpenAI client = OpenAI() completion = client.chat.completions.create( model="gpt-4o", messages=[ {"role": "developer", "content": "You are a famous comedian."}, {"role": "user", "content": "Tell me a joke."}, ], ) assert isinstance(completion.choices[0].message.content, str) ``` ---- Multiple ``pytestfixture`` directives are supported. Add one on each line. .. note:: When combining ``pytestfixture`` and ``continue`` directives together, request pytest-fixtures only in the first ``code-block``, as they will automatically become available in all continuing blocks. Custom pytest-fixtures are supported as well. Just define them in your ``conftest.py`` file. Customisation/hooks ------------------- Tests can be extended and fine-tuned using `pytest`_'s standard hook system. Below is an example workflow: 1. **Add custom pytest markers** to the code blocks (``fakepy``, ``aws``, ``openai``). 2. **Implement pytest hooks** in ``conftest.py`` to react to those markers. Add custom pytest markers ~~~~~~~~~~~~~~~~~~~~~~~~~ Add ``fakepy`` marker ^^^^^^^^^^^^^^^^^^^^^ The example code below will generate a PDF file with random text using `fake.py`_ library. Note, that a ``fakepy`` marker is added to the code block. In the `Implement pytest hooks`_ section, you will see what can be done with the markers. .. note:: Note the ``pytestmark`` directive ``fakepy`` marker. *Filename: README.md* .. code-block:: markdown ```python name=test_create_pdf_file from fake import FAKER FAKER.pdf_file() ``` Add ``aws`` marker ^^^^^^^^^^^^^^^^^^ Sample `boto3`_ code to create a bucket on AWS S3. .. note:: Note the ``pytestmark`` directive ``aws`` marker. *Filename: README.md* .. code-block:: markdown ```python name=test_create_bucket import boto3 s3 = boto3.client("s3", region_name="us-east-1") s3.create_bucket(Bucket="my-bucket") assert "my-bucket" in [b["Name"] for b in s3.list_buckets()["Buckets"]] ``` ---- Implement pytest hooks ~~~~~~~~~~~~~~~~~~~~~~ .. include:: _implement_pytest_hooks.rst docs/cheatsheet_restructured_text.rst ------------------------------------- docs/cheatsheet_restructured_text.rst reStructuredText cheatsheet =========================== This cheatsheet provides a quick reference to some of the most commonly used features and commands. Marking code-block as xfailed ----------------------------- To mark a code-block as expected to fail (xfailed), use the following syntax: .. code-block:: rst .. pytestmark: xfail .. code-block:: python :name: test_example_xfail # Normally this test would fail, but it will xfail instead assert False Requesting specific pytest fixtures for a code-block ---------------------------------------------------- To request specific pytest fixtures for a code-block, use the following syntax: .. code-block:: rst .. pytestfixture: tmp_path .. code-block:: python :name: test_example_with_fixtures # Use the tmp_path fixture in your test file_path = tmp_path / "example.txt" file_path.write_text("Hello, World!") assert file_path.read_text() == "Hello, World!" docs/cheatsheet_markdown.rst ---------------------------- docs/cheatsheet_markdown.rst Markdown cheatsheet =================== This cheatsheet provides a quick reference to some of the most commonly used features and commands. Marking code-block as xfailed ----------------------------- To mark a code-block as expected to fail (xfailed), use the following syntax: .. code-block:: markdown ```python name=test_example_xfail # Normally this test would fail, but it will xfail instead assert False ``` Requesting specific pytest fixtures for a code-block ---------------------------------------------------- To request specific pytest fixtures for a code-block, use the following syntax: .. code-block:: markdown ```python name=test_example_with_fixtures # Use the tmp_path fixture in your test file_path = tmp_path / "example.txt" file_path.write_text("Hello, World!") assert file_path.read_text() == "Hello, World!" ``` docs/customisation.rst ---------------------- docs/customisation.rst Customisation ============= It's possible to customise which codeblock languages and file extensions are recognised by the plugin. ---- Languages --------- By default, the plugin recognises the following codeblock languages: - reStructuredText: `python`, `py`, `python3` - Markdown: `python`, `py`, `python3` reStructruredText ~~~~~~~~~~~~~~~~~ For reStructruredText defaults are configured via `rst_codeblocks` setting in the `[tool.pytest-codeblock]` section of your `pyproject.toml`. .. code-block:: toml [tool.pytest-codeblock] rst_codeblocks = ["python", "py", "python3"] .. note:: Don't touch the defaults, unless you want to remove certain options. If you only want to add custom codeblock languages, use `rst_user_codeblocks`. The following example adds `c_py` as a custom codeblock language: .. code-block:: toml [tool.pytest-codeblock] rst_user_codeblocks = ["c_py"] Now the following codeblock will be recognised and executed: .. code-block:: rst .. code-block:: c_py :name: test_c_py_example print("This is a custom Python codeblock") Markdown ~~~~~~~~ For Markdown defaults configured via `md_codeblocks` setting in the `[tool.pytest-codeblock]` section of your `pyproject.toml`. .. code-block:: toml [tool.pytest-codeblock] md_codeblocks = ["python", "py", "python3"] .. note:: Don't touch the defaults, unless you want to remove certain options. If you only want to add custom codeblock languages, use `md_user_codeblocks`. The following example adds `c_py` as a custom codeblock language: .. code-block:: toml [tool.pytest-codeblock] md_user_codeblocks = ["c_py"] Now the following codeblock will be recognised and executed: .. code-block:: markdown ```c_py name=test_c_py_example print("This is a custom Python codeblock") ``` ---- Extensions ---------- .. note:: If you customise both reStructuredText and Markdown configurations, make sure to avoid overlapping file extensions. reStructruredText ~~~~~~~~~~~~~~~~~ By default, the plugin recognises the following file extensions for reStructuredText files: `.rst` These defaults are configured via `rst_extensions` setting in the `[tool.pytest-codeblock]` section of your `pyproject.toml`. .. code-block:: toml [tool.pytest-codeblock] rst_extensions = [".rst"] .. note:: Don't touch the defaults, unless you want to remove certain options. If you only want to add custom file extensions, use `rst_user_extensions`. The following example adds `.rst.txt` as a custom reStructuredText file extension: .. code-block:: toml [tool.pytest-codeblock] rst_user_extensions = [".rst.txt"] Now the following file will be recognised and processed: .. code-block:: rst *Filename: example.rst.txt* .. code-block:: python :name: test_custom_rst_extension_example print("Custom .rst.txt extension example executed successfully!") Markdown ~~~~~~~~ By default, the plugin recognises the following file extensions for Markdown files: `.md`, `.markdown` These defaults are configured via `md_extensions` setting in the `[tool.pytest-codeblock]` section of your `pyproject.toml`. .. code-block:: toml [tool.pytest-codeblock] md_extensions = [".md", ".markdown"] .. note:: Don't touch the defaults, unless you want to remove certain options. If you only want to add custom file extensions, use `md_user_extensions`. The following example adds `.md.txt` as a custom Markdown file extension: .. code-block:: toml [tool.pytest-codeblock] md_user_extensions = [".md.txt"] Now the following file will be recognised and processed: .. code-block:: markdown *Filename: example.md.txt* ```python name=test_custom_md_extension_example print("Custom .md.txt extension example executed successfully!") ``` conftest.py ----------- conftest.py import contextlib import json import os from pathlib import Path from types import SimpleNamespace import pytest import respx from fake import FILE_REGISTRY from moto import mock_aws from pytest_codeblock.constants import CODEBLOCK_MARK __author__ = "Artur Barseghyan " __copyright__ = "2025-2026 Artur Barseghyan" __license__ = "MIT" __all__ = ( "http_request", "http_request_factory", "markdown_simple", "markdown_with_pytest_mark", "openai_mock", "pytest_collection_modifyitems", "pytest_runtest_setup", "pytest_runtest_teardown", ) pytest_plugins = ["pytester"] # Modify test item during collection def pytest_collection_modifyitems( config: pytest.Config, items: list[pytest.Item], ) -> None: """Modify collected test items after collection is done. :param config: The pytest configuration object. :param items: A list of collected test items. """ for item in items: if item.get_closest_marker(CODEBLOCK_MARK): # Add `documentation` marker to `pytest-codeblock` tests item.add_marker(pytest.mark.documentation) if item.get_closest_marker("aws"): # Apply `mock_aws` to all tests marked as `aws` item.obj = mock_aws(item.obj) # Setup before test runs def pytest_runtest_setup(item: pytest.Item) -> None: """Set up test environment before each test runs. :param item: The test item that is about to run. """ # Teardown after the test ends def pytest_runtest_teardown(item: pytest.Item, nextitem: pytest.Item) -> None: """Tear down test environment after each test ends. :param item: The test item that just finished running. :param nextitem: The next test item that will run (or None if this is """ if item.get_closest_marker("fakepy"): FILE_REGISTRY.clean_up() @pytest.fixture def http_request_factory(): """ Returns a function that creates a simple namespace object with a 'GET' attribute set to the provided dictionary. """ def _factory(get_data: dict): # Creates an object like: object(GET={'key': 'value'}) return SimpleNamespace(GET=get_data) return _factory @pytest.fixture def http_request(http_request_factory): test_data = {"param1": "value1", "signature": "mock-sig"} return http_request_factory(test_data) @pytest.fixture def openai_mock(): # Setup os.environ.setdefault("OPENAI_API_KEY", "test-key") cassette_path = ( Path(__file__).parent / "examples" / "cassettes" / "openai_chat_completion.json" ) with open(cassette_path) as f: response_data = json.load(f) mock = respx.mock() mock.start() mock.post("https://api.openai.com/v1/chat/completions").respond( json=response_data, ) yield mock # Teardown with contextlib.suppress(Exception): mock.stop() @pytest.fixture def markdown_simple(): return """ ```python name=test_example x=1 assert x==1 ```""" @pytest.fixture def markdown_with_pytest_mark(): return """ ```python name=test_db from django.db import models ```""" @pytest.fixture def pytester_subprocess(pytester): """ Wrapper that forces subprocess mode to avoid deprecation warning conflicts when the plugin uses the old `path` argument signature. """ pytester.runpytest = pytester.runpytest_subprocess return pytester docs/conf.py ------------ docs/conf.py # Configuration file for the Sphinx documentation builder. # # For the full list of built-in configuration values, see the documentation: # https://www.sphinx-doc.org/en/master/usage/configuration.html import os import sys sys.path.insert(0, os.path.abspath(os.path.join("..", "src"))) # -- Project information ----------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information try: import pytest_codeblock version = pytest_codeblock.__version__ project = pytest_codeblock.__title__ copyright = pytest_codeblock.__copyright__ author = pytest_codeblock.__author__ except ImportError: version = "0.1" project = "pytest-codeblock" copyright = "2025, Artur Barseghyan " author = "Artur Barseghyan " # -- General configuration --------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration extensions = [ "sphinx.ext.autodoc", "sphinx.ext.viewcode", "sphinx.ext.todo", "sphinx_no_pragma", "sphinx_llms_txt_link", ] templates_path = ["_templates"] exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"] language = "en" release = version # The suffix of source filenames. source_suffix = { ".rst": "restructuredtext", } pygments_style = "sphinx" # -- Options for HTML output ------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output html_theme = "sphinx_rtd_theme" html_static_path = ["_static"] # html_extra_path = ["examples"] prismjs_base = "//cdnjs.cloudflare.com/ajax/libs/prism/1.29.0" html_css_files = [ f"{prismjs_base}/themes/prism.min.css", f"{prismjs_base}/plugins/toolbar/prism-toolbar.min.css", # "https://cdn.jsdelivr.net/gh/barseghyanartur/jsphinx@1.3.4/src/css/sphinx_rtd_theme.css", "https://cdn.jsdelivr.net/gh/barseghyanartur/jsphinx/src/css/sphinx_rtd_theme.css", ] html_js_files = [ f"{prismjs_base}/prism.min.js", f"{prismjs_base}/plugins/autoloader/prism-autoloader.min.js", f"{prismjs_base}/plugins/toolbar/prism-toolbar.min.js", f"{prismjs_base}/plugins/copy-to-clipboard/prism-copy-to-clipboard.min.js", # "https://cdn.jsdelivr.net/gh/barseghyanartur/jsphinx@1.3.4/src/js/download_adapter.js", "https://cdn.jsdelivr.net/gh/barseghyanartur/jsphinx/src/js/download_adapter.js", ] # -- Options for todo extension ---------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/extensions/todo.html#configuration todo_include_todos = True # -- Options for Epub output ---------------------------------------------- epub_title = project epub_author = author epub_publisher = "GitHub" epub_copyright = copyright epub_identifier = "https://github.com/barseghyanartur/pytest-codeblock" # URL or ISBN epub_scheme = "URL" # or "ISBN" epub_uid = "https://github.com/barseghyanartur/pytest-codeblock" pyproject.toml -------------- pyproject.toml [project] name = "pytest-codeblock" description = "Pytest plugin to collect and test code blocks in reStructuredText and Markdown files." readme = "README.rst" version = "0.5.9" requires-python = ">=3.10" dependencies = [ "pytest", "tomli; python_version < '3.11'", ] authors = [ { name = "Artur Barseghyan", email = "artur.barseghyan@gmail.com" }, ] maintainers = [ { name = "Artur Barseghyan", email = "artur.barseghyan@gmail.com" }, ] license = "MIT" classifiers = [ "Framework :: Pytest", "Development Status :: 4 - Beta", "Intended Audience :: Developers", "Operating System :: OS Independent", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.14", "Programming Language :: Python", "Topic :: Software Development :: Testing", "Topic :: Software Development", ] keywords = [ "pytest", "plugin", "documentation", "code blocks", "markdown", "rst", ] [project.urls] Homepage = "https://github.com/barseghyanartur/pytest-codeblock/" Repository = "https://github.com/barseghyanartur/pytest-codeblock/" Issues = "https://github.com/barseghyanartur/pytest-codeblock/issues" Documentation = "https://pytest-codeblock.readthedocs.io/" Changelog = "https://pytest-codeblock.readthedocs.io/en/latest/changelog.html" [project.optional-dependencies] all = ["pytest-codeblock[dev,test,docs,build]"] dev = [ "detect-secrets", "doc8", "ipython", "mypy", "pydoclint", "ruff", "twine", "uv", ] test = [ "django", "moto[s3]", "openai", "pytest", "pytest-cov", "pytest-django", "respx", "langchain-tests", # Module-scoped fixtures for testing scope resolution ] docs = [ "sphinx", "sphinx-autobuild", "sphinx-rtd-theme>=1.3.0", "sphinx-no-pragma", "sphinx-llms-txt-link", "sphinx-source-tree", # "standard-imghdr", ] build = [ "build", "twine", "wheel", ] [project.entry-points."pytest11"] pytest_codeblock = "pytest_codeblock" [tool.setuptools] package-dir = {"" = "src"} [tool.setuptools.packages.find] where = ["src"] include = ["pytest_codeblock", "pytest_codeblock.*"] [build-system] requires = ["setuptools>=41.0", "wheel"] build-backend = "setuptools.build_meta" [tool.ruff] line-length = 80 lint.select = [ "B", # Bugbear "C4", # Complexity "E", # Pycodestyle errors "F", # Pyflakes errors "G", # Logging format "I", # Import sorting "ISC", # Naming "INP", # Implicit namespace "N", # Naming "PERF", # Performance "Q", # Q for Q "SIM", # Simplify "TD", # TODO formatting ] lint.ignore = [ "G004", # Allow f-strings in logging "ISC003", "TD002", "TD003", ] # Enable auto-fix for formatting and import sorting fix = true src = ["src/pytest_codeblock"] exclude = [ ".bzr", ".direnv", ".eggs", ".git", ".hg", ".mypy_cache", ".nox", ".pants.d", ".ruff_cache", ".svn", ".tox", ".venv", "__pypackages__", "_build", "buck-out", "build", "dist", "node_modules", "venv", "docs", ] lint.dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$" target-version = "py39" [tool.ruff.lint.per-file-ignores] "conftest.py" = ["PERF203"] "src/fake.py" = ["INP001"] [tool.ruff.lint.isort] known-first-party = ["pytest_codeblock"] known-third-party = [] [tool.doc8] ignore-path = [ "docs/requirements.txt", "src/pytest-codeblock.egg-info/SOURCES.txt", "examples/customisation_example/*.rst", "examples/customisation_example/*.rst.txt", "examples/customisation_example/*.md", "examples/customisation_example/*.md.txt", ] [tool.pytest.ini_options] addopts = [ "-ra", "-vvv", "-q", "--cov=pytest_codeblock", "--ignore=.tox", "--ignore=requirements", "--ignore=release", "--ignore=tmp", "--cov-report=html", "--cov-report=term", "--cov-report=annotate", "--cov-append", "--capture=no", ] testpaths = [ "**/test*.py", "**/*.rst", "**/*.md", ] pythonpath = [ "src", "examples/md_example", "examples/rst_example", ] norecursedirs = [".git"] DJANGO_SETTINGS_MODULE = "django_settings" markers = [ "slow: mark a test that takes a long time to run.", # "codeblock: pytest-codeblock markers", "aws: mark test as a AWS test", "documentation: mark test as a documentation test", "fakepy: mark test as a fake.py test", "openai: mark test as a openai test", ] [tool.coverage.run] relative_files = true omit = [".tox/*"] [tool.coverage.report] show_missing = true exclude_lines = [ "pragma: no cover", "@overload", ] [tool.mypy] check_untyped_defs = true warn_unused_ignores = true warn_redundant_casts = true warn_unused_configs = true ignore_missing_imports = true [tool.pydoclint] style = "sphinx" exclude = "\\.git|\\.tox|tests/data|\\.venv|fake" require-return-section-when-returning-nothing = false allow-init-docstring = true arg-type-hints-in-docstring = false [tool.sphinx-source-tree] ignore = [ "__pycache__", "*.pyc", "*.pyo", "*.py,cover", ".git", ".hg", ".svn", ".tox", ".nox", ".venv", "venv", "env", "*.egg-info", "dist", "build", "node_modules", ".mypy_cache", ".pytest_cache", ".coverage", "htmlcov", ".idea", ".vscode", ".DS_Store", ".claude", "Thumbs.db", ".ruff_cache", ".coverage.*", ".secrets.baseline", ".pre-commit-config.yaml", ".pre-commit-hooks.yaml", ".readthedocs.yaml", "CHANGELOG.rst", "CODE_OF_CONDUCT.rst", "LICENSE", "SECURITY.rst", "docs/_implement_pytest_hooks.rst", "docs/changelog.rst", "docs/code_of_conduct.rst", "docs/contributor_guidelines.rst", "docs/documentation.rst", "docs/full-llms.rst", "docs/index.rst", "docs/llms.rst", "docs/package.rst", "docs/security.rst", "docs/source_tree.rst", "docs/source_tree_full.rst", "docs/make.bat", "docs/Makefile", "src/pytest_codeblock/tests/pytest_codeblock_*.py", ] order = [ "README.rst", "CONTRIBUTING.rst", "docs/quick_start_ref.rst", "docs/restructured_text.rst", "docs/markdown.rst", "docs/cheatsheet_restructured_text.rst", "docs/cheatsheet_markdown.rst", "docs/customisation.rst", ] [[tool.sphinx-source-tree.files]] output = "docs/source_tree_full.rst" title = "Full project source-tree" [[tool.sphinx-source-tree.files]] output = "docs/source_tree.rst" title = "Project source-tree" ignore = [ "__pycache__", "*.pyc", "*.pyo", "*.py,cover", ".git", ".hg", ".svn", ".tox", ".nox", ".venv", "venv", "env", "*.egg-info", "dist", "build", "node_modules", ".mypy_cache", ".pytest_cache", ".coverage", "htmlcov", ".idea", ".vscode", ".DS_Store", ".claude", "Thumbs.db", ".ruff_cache", ".coverage.*", ".secrets.baseline", ".pre-commit-config.yaml", ".pre-commit-hooks.yaml", ".readthedocs.yaml", "CHANGELOG.rst", "CODE_OF_CONDUCT.rst", "LICENSE", "SECURITY.rst", "docs/_implement_pytest_hooks.rst", "docs/changelog.rst", "docs/code_of_conduct.rst", "docs/contributor_guidelines.rst", "docs/documentation.rst", "docs/full-llms.rst", "docs/index.rst", "docs/llms.rst", "docs/package.rst", "docs/security.rst", "docs/source_tree.rst", "docs/source_tree_full.rst", "docs/make.bat", "docs/Makefile", "examples", "src/pytest_codeblock/tests/pytest_codeblock_*.py", ] src/pytest_codeblock/__init__.py -------------------------------- src/pytest_codeblock/__init__.py from pathlib import Path from .config import get_config from .constants import CODEBLOCK_MARK, PYTESTRUN_MARK from .md import MarkdownFile from .rst import RSTFile __title__ = "pytest-codeblock" __version__ = "0.5.9" __author__ = "Artur Barseghyan " __copyright__ = "2025-2026 Artur Barseghyan" __license__ = "MIT" __all__ = ( "pytest_collect_file", "pytest_configure", ) def pytest_collect_file(parent, path): """Collect .md and .rst files for codeblock tests.""" config = get_config() # Determine file extension (works for py.path or pathlib.Path) file_name = str(path).lower() if any(file_name.endswith(ext) for ext in config.all_md_extensions): # Use the MarkdownFile collector for Markdown files return MarkdownFile.from_parent(parent=parent, path=Path(path)) if any(file_name.endswith(ext) for ext in config.all_rst_extensions): # Use the RSTFile collector for reStructuredText files return RSTFile.from_parent(parent=parent, path=Path(path)) return None def pytest_configure(config): """Register the codeblock marker if not already registered.""" # Get existing markers existing_markers = config.getini("markers") marker_names = [m.split(":")[0].strip() for m in existing_markers] # Only register if not already present if CODEBLOCK_MARK not in marker_names: config.addinivalue_line( "markers", f"{CODEBLOCK_MARK}: pytest-codeblock markers (auto-registered)", ) # Only register if not already present if PYTESTRUN_MARK not in marker_names: config.addinivalue_line( "markers", f"{PYTESTRUN_MARK}: pytest-codeblock markers (auto-registered)", ) src/pytest_codeblock/collector.py --------------------------------- src/pytest_codeblock/collector.py from dataclasses import dataclass, field from typing import Optional __author__ = "Artur Barseghyan " __copyright__ = "2025-2026 Artur Barseghyan" __license__ = "MIT" __all__ = ( "CodeSnippet", "group_snippets", ) @dataclass class CodeSnippet: """Data container for an extracted code snippet.""" code: str # The code content line: int # Starting line number in the source name: Optional[str] = None # Identifier for grouping (None if anonymous) marks: list[str] = field(default_factory=list) # Collected pytest marks (e.g. ['django_db']), parsed from doc comments fixtures: list[str] = field(default_factory=list) # Collected pytest fixtures (e.g. ['tmp_path']), parsed from doc comments group: Optional[str] = None # Set by ``continue:`` directives; names the group this snippet belongs to def group_snippets(snippets: list[CodeSnippet]) -> list[CodeSnippet]: """ Combine snippets that share a group key, using one of two modes: - Merge mode (default): snippets sharing the same name (no ``group`` set, or nameless/same-name continuations) are concatenated into a single test, accumulating marks and fixtures. This is the default behaviour. - Incremental mode: when every continuation snippet (``group`` set) in a group also carries its own distinct name, emit one test per snippet. Each test's code is the cumulative concatenation of all preceding snippets plus itself, so each step is exercised in isolation. Unnamed snippets receive unique auto-keys so they are never merged. """ # Pass 1: bucket each snippet by its group key, preserving insertion order buckets: dict[str, list[CodeSnippet]] = {} order: list[str] = [] anon_count = 0 for sn in snippets: if sn.group: key = sn.group elif sn.name: key = sn.name else: anon_count += 1 key = f"codeblock{anon_count}" if key not in buckets: buckets[key] = [] order.append(key) buckets[key].append(sn) # Pass 2: emit merged or incremental snippets per bucket combined: list[CodeSnippet] = [] for key in order: members = buckets[key] continuations = [sn for sn in members if sn.group] # Incremental only when every continuation has a distinct own name incremental = continuations and all( sn.name and sn.name != key for sn in continuations ) if incremental: acc_code = "" acc_marks: list[str] = [] acc_fixtures: list[str] = [] for sn in members: acc_code = acc_code + "\n" + sn.code if acc_code else sn.code acc_marks.extend(sn.marks) acc_fixtures.extend(sn.fixtures) combined.append(CodeSnippet( name=sn.name, code=acc_code, line=sn.line, marks=list(acc_marks), fixtures=list(acc_fixtures), )) else: # Merge mode (default behaviour) first = members[0] merged_marks = list(first.marks) merged_fixtures = list(first.fixtures) merged_code = first.code for sn in members[1:]: merged_code += "\n" + sn.code merged_marks.extend(sn.marks) merged_fixtures.extend(sn.fixtures) combined.append(CodeSnippet( name=first.name, code=merged_code, line=first.line, marks=merged_marks, fixtures=merged_fixtures, )) return combined src/pytest_codeblock/config.py ------------------------------ src/pytest_codeblock/config.py """Configuration loading from pyproject.toml.""" import sys from pathlib import Path from typing import Optional if sys.version_info >= (3, 11): import tomllib else: try: import tomli as tomllib except ImportError: tomllib = None # type: ignore[assignment] __author__ = "Artur Barseghyan " __copyright__ = "2025-2026 Artur Barseghyan" __license__ = "MIT" __all__ = ( "get_config", "Config", ) # Default values DEFAULT_RST_CODEBLOCKS = ("py", "python", "python3") DEFAULT_MD_CODEBLOCKS = ("py", "python", "python3") DEFAULT_RST_EXTENSIONS = (".rst",) DEFAULT_MD_EXTENSIONS = (".md", ".markdown") DEFAULT_TEST_NAMELESS_CODEBLOCKS = False class Config: """Configuration container for pytest-codeblock.""" def __init__( self, rst_codeblocks: tuple[str, ...] = DEFAULT_RST_CODEBLOCKS, rst_user_codeblocks: tuple[str, ...] = (), md_codeblocks: tuple[str, ...] = DEFAULT_MD_CODEBLOCKS, md_user_codeblocks: tuple[str, ...] = (), rst_extensions: tuple[str, ...] = DEFAULT_RST_EXTENSIONS, rst_user_extensions: tuple[str, ...] = (), md_extensions: tuple[str, ...] = DEFAULT_MD_EXTENSIONS, md_user_extensions: tuple[str, ...] = (), test_nameless_codeblocks: bool = DEFAULT_TEST_NAMELESS_CODEBLOCKS, ): self.rst_codeblocks = rst_codeblocks self.rst_user_codeblocks = rst_user_codeblocks self.md_codeblocks = md_codeblocks self.md_user_codeblocks = md_user_codeblocks self.rst_extensions = rst_extensions self.rst_user_extensions = rst_user_extensions self.md_extensions = md_extensions self.md_user_extensions = md_user_extensions self.test_nameless_codeblocks = test_nameless_codeblocks @property def all_rst_codeblocks(self) -> tuple[str, ...]: """Combined RST codeblocks (system + user).""" return self.rst_codeblocks + self.rst_user_codeblocks @property def all_md_codeblocks(self) -> tuple[str, ...]: """Combined MD codeblocks (system + user).""" return self.md_codeblocks + self.md_user_codeblocks @property def all_rst_extensions(self) -> tuple[str, ...]: """Combined RST extensions (system + user).""" return self.rst_extensions + self.rst_user_extensions @property def all_md_extensions(self) -> tuple[str, ...]: """Combined MD extensions (system + user).""" return self.md_extensions + self.md_user_extensions _cached_config: Optional[Config] = None def _find_pyproject_toml() -> Optional[Path]: """Find pyproject.toml starting from cwd and going up.""" cwd = Path.cwd() for parent in [cwd, *cwd.parents]: candidate = parent / "pyproject.toml" if candidate.is_file(): return candidate return None def _load_config_from_pyproject(path: Path) -> dict: """Load [tool.pytest-codeblock] section from pyproject.toml.""" if tomllib is None: return {} try: with open(path, "rb") as f: data = tomllib.load(f) return data.get("tool", {}).get("pytest-codeblock", {}) except Exception: return {} def _to_tuple(val, default: tuple[str, ...]) -> tuple[str, ...]: if val is None: return default if isinstance(val, (list, tuple)): return tuple(val) return default def _to_bool(val, default: bool) -> bool: if val is None: return default if isinstance(val, bool): return val if isinstance(val, str): return val.lower() in ("true", "1", "yes") return default def get_config(*, force_reload: bool = False) -> Config: """Get the configuration, loading from pyproject.toml if available.""" global _cached_config if _cached_config is not None and not force_reload: return _cached_config pyproject_path = _find_pyproject_toml() if pyproject_path is None: _cached_config = Config() return _cached_config raw = _load_config_from_pyproject(pyproject_path) _cached_config = Config( rst_codeblocks=_to_tuple( raw.get("rst_codeblocks"), DEFAULT_RST_CODEBLOCKS ), rst_user_codeblocks=_to_tuple(raw.get("rst_user_codeblocks"), ()), md_codeblocks=_to_tuple( raw.get("md_codeblocks"), DEFAULT_MD_CODEBLOCKS ), md_user_codeblocks=_to_tuple(raw.get("md_user_codeblocks"), ()), rst_extensions=_to_tuple( raw.get("rst_extensions"), DEFAULT_RST_EXTENSIONS ), rst_user_extensions=_to_tuple(raw.get("rst_user_extensions"), ()), md_extensions=_to_tuple( raw.get("md_extensions"), DEFAULT_MD_EXTENSIONS ), md_user_extensions=_to_tuple(raw.get("md_user_extensions"), ()), test_nameless_codeblocks=_to_bool( raw.get("test_nameless_codeblocks"), DEFAULT_TEST_NAMELESS_CODEBLOCKS, ), ) return _cached_config src/pytest_codeblock/constants.py --------------------------------- src/pytest_codeblock/constants.py __author__ = "Artur Barseghyan " __copyright__ = "2025-2026 Artur Barseghyan" __license__ = "MIT" __all__ = ( "CODEBLOCK_MARK", "DJANGO_DB_MARKS", "PYTESTRUN_MARK", "TEST_PREFIX", ) DJANGO_DB_MARKS = { "django_db", "db", "transactional_db", } TEST_PREFIX = "test_" CODEBLOCK_MARK = "codeblock" # When this mark is present on a code block, the plugin will exec() the block # and then discover and run any Test* classes / test_* functions found in it, # rather than treating the whole block as a single test body. PYTESTRUN_MARK = "pytestrun" src/pytest_codeblock/helpers.py ------------------------------- src/pytest_codeblock/helpers.py import ast import textwrap __author__ = "Artur Barseghyan " __copyright__ = "2025-2026 Artur Barseghyan" __license__ = "MIT" __all__ = ( "contains_top_level_await", "wrap_async_code", ) def contains_top_level_await(code: str) -> bool: """Analyzes code to detect presence of async patterns.""" try: tree = ast.parse(code) except SyntaxError: # If the code is invalid, it technically doesn't # contain valid async patterns. return False # Define the AST nodes that represent async constructs async_nodes = ( ast.AsyncFunctionDef, # async def ... ast.Await, # await ... ast.AsyncWith, # async with ... ast.AsyncFor, # async for ... ) return any(isinstance(node, async_nodes) for node in ast.walk(tree)) def wrap_async_code(code: str) -> str: """Wrap code containing top-level await in an async function.""" ind = textwrap.indent(code, " ") return ( f"async def __async_main__():\n{ind}\n\nasyncio.run(__async_main__())" ) src/pytest_codeblock/md.py -------------------------- src/pytest_codeblock/md.py import asyncio import inspect import re import textwrap import traceback import types from collections.abc import Generator from typing import Optional import pytest from .collector import CodeSnippet, group_snippets from .config import get_config from .constants import ( CODEBLOCK_MARK, DJANGO_DB_MARKS, PYTESTRUN_MARK, TEST_PREFIX, ) from .helpers import contains_top_level_await, wrap_async_code from .pytestrun import run_pytest_style_code __author__ = "Artur Barseghyan " __copyright__ = "2025-2026 Artur Barseghyan" __license__ = "MIT" __all__ = ( "MarkdownFile", "parse_markdown", ) def parse_markdown(text: str) -> list[CodeSnippet]: """ Parse Markdown text and extract Python code snippets as CodeSnippet objects. Supports: - comments immediately before a code fence - comments for naming - comments for grouping with a named snippet - Fenced code blocks with ```python (and optional name= in the info string) Captures each snippet's name, code, starting line, and any pytest marks. """ config = get_config() snippets: list[CodeSnippet] = [] lines = text.splitlines() pending_name: Optional[str] = None pending_continue: Optional[str] = None pending_marks: list[str] = [CODEBLOCK_MARK] pending_fixtures: list[str] = [] in_block = False fence = "" block_indent = 0 code_buffer: list[str] = [] snippet_name: Optional[str] = None start_line = 0 for idx, line in enumerate(lines, start=1): stripped = line.strip() if not in_block: # Check for pytest mark comment if stripped.startswith("", stripped) if m: pending_marks.append(m.group(1)) continue # Check for pytest fixture comment if stripped.startswith("", stripped) if m: pending_fixtures.append(m.group(1)) continue # Check for continue comment if stripped.startswith("", stripped) if m: pending_continue = m.group(1) continue # Check for name comment if stripped.startswith("", stripped ) if m: pending_name = m.group(1) continue # Start of fenced code block? if line.lstrip().startswith("```"): indent = len(line) - len(line.lstrip()) m = re.match(r"^`{3,}", line.lstrip()) if not m: continue fence = m.group(0) info = line.lstrip()[len(fence):].strip() parts = info.split(None, 1) lang = parts[0].lower() if parts else "" extra = parts[1] if len(parts) > 1 else "" if lang in config.all_md_codeblocks: in_block = True block_indent = indent start_line = idx + 1 code_buffer = [] # Determine name from info string or pending comment snippet_name = None for token in extra.split(): if ( token.startswith("name=") or token.startswith("name:") ): snippet_name = ( token.split("=", 1)[-1] if "=" in token else token.split(":", 1)[-1] ) break if snippet_name is None: snippet_name = pending_name # Reset pending_name; marks stay until block closes pending_name = None else: # Inside a fenced code block if line.lstrip().startswith(fence): # End of block in_block = False code_text = "\n".join(code_buffer) snippet_group = None # Continue overrides snippet_name for grouping if pending_continue: snippet_group = pending_continue pending_continue = None snippets.append(CodeSnippet( name=snippet_name, code=code_text, line=start_line, marks=pending_marks.copy(), fixtures=pending_fixtures.copy(), group=snippet_group, )) # Reset pending marks after collecting pending_marks = [CODEBLOCK_MARK] # Reset to default snippet_name = None pending_fixtures.clear() # Clear pending fixtures else: # Collect code lines (dedent by block_indent) if line.strip() == "": code_buffer.append("") else: if len(line) >= block_indent: code_buffer.append(line[block_indent:]) else: code_buffer.append(line.lstrip()) return snippets class MarkdownFile(pytest.Module): """ Collector for Markdown files, extracting only `test_`-prefixed code snippets. """ def _getobj(self) -> types.ModuleType: m = types.ModuleType(self.path.stem) m.__file__ = str(self.path) m.__test__ = False # prevent PyCollector from auto-collecting return m def collect(self) -> Generator[pytest.Function, None, None]: # Register with fixture manager so module-scoped fixtures can find # a pytest.Module parent node (fixes scope resolution when plugins # like pytest-recording/langchain-tests define module-scoped fixtures). self.session._fixturemanager.parsefactories(self) text = self.path.read_text(encoding="utf-8") raw = parse_markdown(text) config = get_config() # Include both named and nameless blocks, if config allows nameless # blocks. Nameless blocks will be auto-named based on the module # name and a counter, ensuring they get collected as tests. if config.test_nameless_codeblocks: tests = [] counter = 1 module_name = self.path.stem for sn in raw: if sn.name and sn.name.startswith(TEST_PREFIX): tests.append(sn) elif not sn.name: auto_name = f"{TEST_PREFIX}{module_name}_{counter}" counter += 1 sn.name = auto_name tests.append(sn) # If config does not allow nameless blocks, only those with explicit # names starting with TEST_PREFIX will be collected. else: # keep only snippets named test_* tests = [ sn for sn in raw if sn.name and sn.name.startswith(TEST_PREFIX) ] combined = group_snippets(tests) for sn in combined: # Bind the values we need so we don't close over `sn` itself _sn_name = sn.name _fpath = str(self.path) _is_pytestrun = PYTESTRUN_MARK in sn.marks # Build list of fixture names requested by this snippet _fixture_names: list[str] = list(sn.fixtures) # If snippet is marked as needing DB, also request the `db` # fixture, unless user already added it explicitly. if ( DJANGO_DB_MARKS.intersection(sn.marks) and "db" not in _fixture_names ): _fixture_names.append("db") # Generate a real pytest Function so fixtures work def make_func( code, sn_name=_sn_name, fpath=_fpath, fixture_names=_fixture_names, is_pytestrun=_is_pytestrun, ): # This inner function *actually* has a **fixtures signature, # but we override __signature__ so pytest passes the right # fixtures and names. def test_block(**fixtures): if is_pytestrun: run_pytest_style_code( code=code, snippet_name=sn_name, path=fpath, ) return # Normal (non-pytestrun) execution path ex_code = code if contains_top_level_await(code): # Auto-wrap async code ex_code = wrap_async_code(code) try: compiled = compile(ex_code, fpath, "exec") except SyntaxError as err: raise SyntaxError( f"Syntax error in " f"codeblock `{sn_name}` in {fpath}:\n" f"\n{textwrap.indent(ex_code, prefix=' ')}\n\n" f"{traceback.format_exc()}" ) from err try: # Make fixtures available as top-level names # inside the executed snippet. exec(compiled, {"asyncio": asyncio, **dict(fixtures)}) except Exception as err: raise Exception( f"Error in " f"codeblock `{sn_name}` in {fpath}:\n" f"\n{textwrap.indent(ex_code, prefix=' ')}\n\n" f"{traceback.format_exc()}" ) from err # Tell pytest which fixture arguments this test has: test_block.__signature__ = inspect.Signature( [ inspect.Parameter( name, inspect.Parameter.POSITIONAL_OR_KEYWORD, ) for name in fixture_names ] ) return test_block callobj = make_func(sn.code) fn = pytest.Function.from_parent( parent=self, name=sn.name, callobj=callobj, ) # Apply any marks (e.g. django_db) for m in sn.marks: fn.add_marker(getattr(pytest.mark, m)) yield fn src/pytest_codeblock/pytestrun.py --------------------------------- src/pytest_codeblock/pytestrun.py """ Helper module for running pytest-style tests found inside executed code blocks. When a code block is marked with `pytestrun`, its code is written to a temporary file and executed by pytest as a subprocess, so that fixtures, markers, setup/teardown, and assertions all work correctly. """ import os import subprocess import sys import tempfile __author__ = "Artur Barseghyan " __copyright__ = "2025-2026 Artur Barseghyan" __license__ = "MIT" __all__ = ("run_pytest_style_code",) def run_pytest_style_code( code: str, snippet_name: str, path: str, ) -> None: """ Write the code block to a temporary file and run it with pytest. Raises AssertionError on any test failures. """ project_root = os.getcwd() # Place the temp directory alongside the source file so that pytest walks # up through its real directory hierarchy and discovers conftest.py files # (including project-root ones that define fixtures). source_dir = os.path.dirname(os.path.abspath(path)) pytest_cache_dir = os.path.join(source_dir, ".pytest_cache") os.makedirs(pytest_cache_dir, exist_ok=True) tmpdir = tempfile.mkdtemp(prefix="pytest_codeblock_", dir=pytest_cache_dir) tmpfile = os.path.join(tmpdir, f"{snippet_name}.py") try: with open(tmpfile, "w") as f: f.write(code) env = os.environ.copy() env["PYTHONPATH"] = os.pathsep.join(sys.path) result = subprocess.run( [ sys.executable, "-m", "pytest", tmpfile, f"--rootdir={project_root}", "--no-header", "-q", ], capture_output=True, text=True, cwd=project_root, env=env, ) if result.returncode != 0: output = (result.stdout + result.stderr).strip() raise AssertionError( f"pytestrun block `{snippet_name}` in {path} failed:\n\n" f"{output}" ) finally: try: os.unlink(tmpfile) os.rmdir(tmpdir) except OSError: pass src/pytest_codeblock/rst.py --------------------------- src/pytest_codeblock/rst.py import asyncio import inspect import re import textwrap import traceback import types from collections.abc import Generator from pathlib import Path from typing import Optional, Union import pytest from .collector import CodeSnippet, group_snippets from .config import get_config from .constants import ( CODEBLOCK_MARK, DJANGO_DB_MARKS, PYTESTRUN_MARK, TEST_PREFIX, ) from .helpers import contains_top_level_await, wrap_async_code from .pytestrun import run_pytest_style_code __author__ = "Artur Barseghyan " __copyright__ = "2025-2026 Artur Barseghyan" __license__ = "MIT" __all__ = ( "RSTFile", "parse_rst", "resolve_literalinclude_path", "get_literalinclude_content", ) def resolve_literalinclude_path( base_dir: Union[str, Path], include_path: str, ) -> Optional[str]: """ Resolve the full path for a literalinclude directive. Returns None if the file doesn't exist. """ _include_path = Path(include_path) # If `include_path` is already absolute or relative and exists, done if _include_path.exists(): return str(_include_path.resolve()) # If base_path is a file, switch to its parent directory _base_path = Path(base_dir) if _base_path.is_file(): _base_path = _base_path.parent try: full_path = _base_path / include_path if full_path.exists(): return str(full_path.resolve()) except Exception: pass return None def get_literalinclude_content(path): try: with open(path) as f: return f.read() except Exception as e: raise RuntimeError( f"Failed to read literalinclude file {path}: {e}" ) from e def parse_rst(text: str, base_dir: Path) -> list[CodeSnippet]: """ Parse an RST document into CodeSnippet objects, capturing: - .. pytestmark: - .. continue: - .. codeblock-name: - .. code-block:: python """ config = get_config() snippets: list[CodeSnippet] = [] lines = text.splitlines() n = len(lines) pending_name: Optional[str] = None pending_marks: list[str] = [CODEBLOCK_MARK] pending_fixtures: list[str] = [] pending_continue: Optional[str] = None i = 0 while i < n: line = lines[i] # -------------------------------------------------------------------- # Collect `.. pytestmark: xyz` # -------------------------------------------------------------------- m = re.match(r"^\s*\.\.\s*pytestmark:\s*(\w+)\s*$", line) if m: pending_marks.append(m.group(1)) i += 1 continue # -------------------------------------------------------------------- # Collect `.. pytestfixture: foo` # -------------------------------------------------------------------- m = re.match(r"^\s*\.\.\s*pytestfixture:\s*(\w+)\s*$", line) if m: pending_fixtures.append(m.group(1)) i += 1 continue # -------------------------------------------------------------------- # The `.. literalinclude` directive # -------------------------------------------------------------------- if line.strip().startswith(".. literalinclude::"): path = line.split(".. literalinclude::", 1)[1].strip() name = None # Look ahead for name j = i + 1 while j < len(lines) and lines[j].strip(): if ":name:" in lines[j]: name = lines[j].split(":name:", 1)[1].strip() break j += 1 if name and name.startswith("test_"): full_path = resolve_literalinclude_path(base_dir, path) if full_path: snippet = CodeSnippet( code=get_literalinclude_content(full_path), line=i + 1, name=name, marks=pending_marks.copy(), fixtures=pending_fixtures.copy(), ) snippets.append(snippet) pending_marks = [CODEBLOCK_MARK] pending_fixtures.clear() i = j + 1 continue # -------------------------------------------------------------------- # Collect `.. continue: foo` # -------------------------------------------------------------------- m = re.match(r"^\s*\.\.\s*continue:\s*(\S+)\s*$", line) if m: pending_continue = m.group(1) i += 1 continue # -------------------------------------------------------------------- # Collect `.. codeblock-name: foo` # -------------------------------------------------------------------- m = re.match(r"^\s*\.\.\s*codeblock-name:\s*(\S+)\s*$", line) if m: pending_name = m.group(1) i += 1 continue # -------------------------------------------------------------------- # The `.. code-block` directive # -------------------------------------------------------------------- m = re.match(r"^(\s*)\.\. (?:code-block|code)::\s*(\w+)", line) if m: base_indent = len(m.group(1)) lang = m.group(2).lower() if lang in config.all_rst_codeblocks: # Parse :name: option name_val: Optional[str] = None j = i + 1 while j < n: ln = lines[j] if not ln.strip(): j += 1 continue indent = len(ln) - len(ln.lstrip()) if ln.lstrip().startswith(":") and indent > base_indent: opt = ln.lstrip() if opt.lower().startswith(":name:"): name_val = opt.split(":", 2)[2].strip().split()[0] j += 1 continue break # The j is first code line if j >= n: i = j continue first = lines[j] content_indent = len(first) - len(first.lstrip()) if content_indent <= base_indent: i = j continue # Collect code buf: list[str] = [] k = j while k < n: ln = lines[k] if not ln.strip(): buf.append("") k += 1 continue ind = len(ln) - len(ln.lstrip()) if ind >= content_indent: buf.append(ln[content_indent:]) k += 1 else: break sn_group = None # Decide snippet name: continue overrides name_val/pending_name if pending_continue: sn_group = pending_continue pending_continue = None sn_name = name_val or pending_name sn_marks = pending_marks.copy() sn_fixtures = pending_fixtures.copy() pending_name = None pending_marks = [CODEBLOCK_MARK] # clear pending marks pending_fixtures.clear() snippets.append(CodeSnippet( name=sn_name, code="\n".join(buf), line=j + 1, marks=sn_marks, fixtures=sn_fixtures, group=sn_group, )) i = k continue else: i += 1 continue # -------------------------------------------------------------------- # The literal-block via "::" # -------------------------------------------------------------------- if line.rstrip().endswith("::") and pending_name: # Similar override logic sn_group = None if pending_continue: sn_group = pending_continue pending_continue = None sn_name = pending_name sn_marks = pending_marks.copy() sn_fixtures = pending_fixtures.copy() pending_name = None pending_marks = [CODEBLOCK_MARK] # clear pending marks pending_fixtures.clear() j = i + 1 if j < n and not lines[j].strip(): j += 1 if j >= n: i = j continue first = lines[j] content_indent = len(first) - len(first.lstrip()) buf: list[str] = [] k = j while k < n: ln = lines[k] if not ln.strip(): buf.append("") k += 1 continue ind = len(ln) - len(ln.lstrip()) if ind >= content_indent: buf.append(ln[content_indent:]) k += 1 else: break snippets.append(CodeSnippet( name=sn_name, code="\n".join(buf), line=j + 1, marks=sn_marks, fixtures=sn_fixtures, group=sn_group, )) i = k continue i += 1 return snippets class RSTFile(pytest.Module): """Collect RST code-block tests as real test functions.""" def _getobj(self) -> types.ModuleType: m = types.ModuleType(self.path.stem) m.__file__ = str(self.path) m.__test__ = False # prevent PyCollector from auto-collecting return m def collect(self) -> Generator[pytest.Function, None, None]: # Register this node with the fixture manager so that module-scoped # fixtures (e.g. vcr_cassette_dir from pytest-recording/langchain-tests) # can resolve their scope by walking up to a pytest.Module parent. self.session._fixturemanager.parsefactories(self) text = self.path.read_text(encoding="utf-8") raw = parse_rst(text, self.path) config = get_config() # Include both named and nameless blocks, if config allows nameless # blocks. Nameless blocks will be auto-named based on the module # name and a counter, ensuring they get collected as tests. if config.test_nameless_codeblocks: tests = [] counter = 1 module_name = self.path.stem for sn in raw: if sn.name and sn.name.startswith(TEST_PREFIX): tests.append(sn) elif not sn.name: auto_name = f"{TEST_PREFIX}{module_name}_{counter}" counter += 1 sn.name = auto_name tests.append(sn) # If config does not allow nameless blocks, only those with explicit # names starting with TEST_PREFIX will be collected. else: # Only keep test_* snippets tests = [ sn for sn in raw if sn.name and sn.name.startswith(TEST_PREFIX) ] combined = group_snippets(tests) for sn in combined: # Bind the values we need so we don't close over `sn` itself _sn_name = sn.name _fpath = str(self.path) _is_pytestrun = PYTESTRUN_MARK in sn.marks # Build list of fixture names requested by this snippet _fixture_names: list[str] = list(sn.fixtures) # If snippet is marked as needing DB, also request the `db` # fixture, unless user already added it explicitly. if ( DJANGO_DB_MARKS.intersection(sn.marks) and "db" not in _fixture_names ): _fixture_names.append("db") def make_func( code, sn_name=_sn_name, fpath=_fpath, fixture_names=_fixture_names, is_pytestrun=_is_pytestrun, ): # This inner function *actually* has a **fixtures signature, # but we override __signature__ so pytest passes the right # fixtures and names. def test_block(**fixtures): if is_pytestrun: run_pytest_style_code( code=code, snippet_name=sn_name, path=fpath, ) return # Normal (non-pytestrun) execution path ex_code = code if contains_top_level_await(code): ex_code = wrap_async_code(code) try: compiled = compile(ex_code, fpath, "exec") except SyntaxError as err: raise SyntaxError( f"Syntax error in " f"codeblock `{sn_name}` in {fpath}:\n" f"\n{textwrap.indent(ex_code, prefix=' ')}\n\n" f"{traceback.format_exc()}" ) from err try: # Make fixtures available as top-level names # inside the executed snippet. exec(compiled, {"asyncio": asyncio, **dict(fixtures)}) except Exception as err: raise Exception( f"Error in " f"codeblock `{sn_name}` in {fpath}:\n" f"\n{textwrap.indent(ex_code, prefix=' ')}\n\n" f"{traceback.format_exc()}" ) from err # Tell pytest which fixture arguments this test has: test_block.__signature__ = inspect.Signature( [ inspect.Parameter( name, inspect.Parameter.POSITIONAL_OR_KEYWORD, ) for name in fixture_names ] ) return test_block callobj = make_func(sn.code) fn = pytest.Function.from_parent( parent=self, name=sn.name, callobj=callobj ) # Re-apply any pytest.mark. markers for m in sn.marks: fn.add_marker(getattr(pytest.mark, m)) yield fn src/pytest_codeblock/tests/__init__.py -------------------------------------- src/pytest_codeblock/tests/__init__.py src/pytest_codeblock/tests/test_customisation.py ------------------------------------------------ src/pytest_codeblock/tests/test_customisation.py """Tests for customisation of languages and extensions.""" from unittest.mock import patch from ..config import get_config from ..md import parse_markdown class TestCustomLanguages: """Test custom language support in markdown.""" def test_custom_md_language_recognized(self): """Test that custom markdown language is recognised when configured.""" # Mock config to include custom language mock_config = { "all_md_codeblocks": ["python", "djc_py"], # djc_py is custom "all_rst_codeblocks": ["python"], "all_md_extensions": [".md"], "all_rst_extensions": [".rst"], } text = """ ```djc_py name=custom_lang x = 1 ``` """ with patch("pytest_codeblock.md.get_config") as mock_get_config: mock_config_obj = type("Config", (), mock_config)() mock_get_config.return_value = mock_config_obj snippets = parse_markdown(text) assert len(snippets) == 1 assert snippets[0].name == "custom_lang" assert "x = 1" in snippets[0].code # ------------------------------------------------------------------------ def test_unknown_language_ignored(self): """Test that unknown language fence is ignored.""" text = """ ```unknown_lang x = 1 ``` """ snippets = parse_markdown(text) assert len(snippets) == 0 class TestCustomExtensions: """Test custom file extension support.""" def test_config_includes_custom_extensions(self): """Test that config can include custom extensions.""" config = get_config() # By default should include .md and .rst assert ".md" in config.all_md_extensions assert ".rst" in config.all_rst_extensions def test_python_as_custom_md_extension(self): """Test that .py files can be configured as markdown sources.""" # This test verifies the config structure supports it mock_config = { "all_md_codeblocks": ["python"], "all_rst_codeblocks": ["python"], "all_md_extensions": [".md", ".txt"], # .txt added "all_rst_extensions": [".rst"], } mock_config_obj = type("Config", (), mock_config)() # Verify .txt extension is in the list assert ".txt" in mock_config_obj.all_md_extensions class TestDefaults: """Test that defaults are preserved.""" def test_default_python_language_works(self): """Test that default Python language is always available.""" text = """ ```python x = 1 ``` """ snippets = parse_markdown(text) assert len(snippets) == 1 # ------------------------------------------------------------------------ def test_default_md_extension(self): """Test that .md is always a supported extension.""" config = get_config() assert ".md" in config.all_md_extensions def test_default_rst_extension(self): """Test that .rst is always a supported extension.""" config = get_config() assert ".rst" in config.all_rst_extensions src/pytest_codeblock/tests/test_integration.py ---------------------------------------------- src/pytest_codeblock/tests/test_integration.py """ Integration tests that directly import and test all module components. This module exists to ensure 100% coverage when running with pytest-cov, by explicitly importing all functions and classes at test time rather than relying on plugin auto-loading (which happens before coverage starts). """ from dataclasses import fields from unittest.mock import MagicMock import pytest from .. import ( pytest_collect_file, ) from ..collector import ( CodeSnippet, group_snippets, ) from ..constants import ( CODEBLOCK_MARK, DJANGO_DB_MARKS, TEST_PREFIX, ) from ..helpers import ( contains_top_level_await, wrap_async_code, ) from ..md import ( MarkdownFile, parse_markdown, ) from ..rst import ( RSTFile, get_literalinclude_content, parse_rst, resolve_literalinclude_path, ) # ============================================================================ # Test constants.py # ============================================================================ class TestConstants: """Test constants module values.""" def test_codeblock_mark(self): assert CODEBLOCK_MARK == "codeblock" def test_django_db_marks(self): assert isinstance(DJANGO_DB_MARKS, set) assert "django_db" in DJANGO_DB_MARKS assert "db" in DJANGO_DB_MARKS assert "transactional_db" in DJANGO_DB_MARKS def test_test_prefix(self): assert TEST_PREFIX == "test_" # ============================================================================ # Test collector.py - CodeSnippet dataclass # ============================================================================ class TestCodeSnippet: """Test CodeSnippet dataclass.""" def test_code_snippet_creation(self): """Test basic CodeSnippet creation.""" sn = CodeSnippet(code="x = 1", line=10) assert sn.code == "x = 1" assert sn.line == 10 assert sn.name is None assert sn.marks == [] assert sn.fixtures == [] def test_code_snippet_with_all_fields(self): """Test CodeSnippet with all fields.""" sn = CodeSnippet( code="y = 2", line=20, name="test_example", marks=["codeblock", "django_db"], fixtures=["tmp_path", "capsys"], ) assert sn.name == "test_example" assert "codeblock" in sn.marks assert "tmp_path" in sn.fixtures def test_code_snippet_is_dataclass(self): """Verify CodeSnippet is a proper dataclass.""" field_names = [f.name for f in fields(CodeSnippet)] assert "code" in field_names assert "line" in field_names assert "name" in field_names assert "marks" in field_names assert "fixtures" in field_names # ============================================================================ # Test collector.py - group_snippets function # ============================================================================ class TestGroupSnippets: """Test group_snippets function.""" def test_group_snippets_single(self): """Test with single snippet.""" sn = CodeSnippet(name="test_one", code="a=1", line=1) result = group_snippets([sn]) assert len(result) == 1 assert result[0].name == "test_one" def test_group_snippets_merge_same_name(self): """Test merging snippets with same name.""" sn1 = CodeSnippet(name="test_foo", code="a=1", line=1, marks=["m1"]) sn2 = CodeSnippet(name="test_foo", code="b=2", line=5, marks=["m2"]) result = group_snippets([sn1, sn2]) assert len(result) == 1 assert "a=1" in result[0].code assert "b=2" in result[0].code assert "m1" in result[0].marks assert "m2" in result[0].marks def test_group_snippets_different_names(self): """Test snippets with different names stay separate.""" sn1 = CodeSnippet(name="test_a", code="a=1", line=1) sn2 = CodeSnippet(name="test_b", code="b=2", line=5) result = group_snippets([sn1, sn2]) assert len(result) == 2 def test_group_snippets_anonymous(self): """Test anonymous snippets (name=None) get auto-generated names.""" sn1 = CodeSnippet(name=None, code="a=1", line=1) sn2 = CodeSnippet(name=None, code="b=2", line=5) sn3 = CodeSnippet(name=None, code="c=3", line=10) combined = group_snippets([sn1, sn2, sn3]) assert len(combined) == 3 # Anonymous snippets get codeblock1, codeblock2, codeblock3 names = [sn.name for sn in combined] # name stays None but key used assert "codeblock1" in names or combined[0].name is None # The snippets should remain separate since they have different # auto-keys assert combined[0].code == "a=1" assert combined[1].code == "b=2" assert combined[2].code == "c=3" def test_group_snippets_fixtures_merge(self): """Test fixtures are accumulated when merging.""" sn1 = CodeSnippet( name="test_f", code="x=1", line=1, fixtures=["tmp_path"] ) sn2 = CodeSnippet( name="test_f", code="y=2", line=5, fixtures=["capsys"] ) combined = group_snippets([sn1, sn2]) assert len(combined) == 1 # Fixtures should be merged assert "tmp_path" in combined[0].fixtures assert "capsys" in combined[0].fixtures # Code should be concatenated assert "x=1" in combined[0].code assert "y=2" in combined[0].code # ============================================================================ # Test helpers.py - contains_top_level_await # ============================================================================ class TestContainsTopLevelAwait: """Test contains_top_level_await function.""" def test_await_expression(self): assert contains_top_level_await("await asyncio.sleep(0)") is True def test_async_function_def(self): assert contains_top_level_await("async def foo(): pass") is True def test_async_with(self): assert contains_top_level_await("async with lock: pass") is True def test_async_for(self): assert contains_top_level_await("async for i in gen: pass") is True def test_sync_code(self): assert contains_top_level_await("x = 1 + 2") is False def test_await_in_string(self): assert contains_top_level_await("print('await something')") is False def test_syntax_error_returns_false(self): """Test invalid syntax returns False (covers except SyntaxError).""" assert contains_top_level_await("def broken(:") is False # ============================================================================ # Test helpers.py - wrap_async_code # ============================================================================ class TestWrapAsyncCode: """Test wrap_async_code function.""" def test_wrap_basic(self): code = "await asyncio.sleep(1)" wrapped = wrap_async_code(code) assert "async def __async_main__():" in wrapped assert "asyncio.run(__async_main__())" in wrapped assert " await asyncio.sleep(1)" in wrapped def test_wrap_multiline(self): code = "x = 1\nawait asyncio.sleep(0)\ny = 2" wrapped = wrap_async_code(code) assert " x = 1" in wrapped assert " await asyncio.sleep(0)" in wrapped assert " y = 2" in wrapped def test_wrapped_code_compiles(self): """Verify wrapped code is valid Python.""" code = "result = 42" wrapped = wrap_async_code(code) # Should not raise compile(wrapped, "", "exec") # ============================================================================ # Test __init__.py - pytest_collect_file hook # ============================================================================ class TestPytestCollectFile: """Test pytest_collect_file hook function.""" def test_collect_markdown_file(self, tmp_path): """Test .md file returns MarkdownFile.""" md_file = tmp_path / "test.md" md_file.write_text("# Test") parent = MagicMock() parent.path = tmp_path parent.session = MagicMock() parent.config = MagicMock() result = pytest_collect_file(parent, md_file) assert result is not None assert isinstance(result, MarkdownFile) def test_collect_markdown_extension(self, tmp_path): """Test .markdown extension.""" md_file = tmp_path / "test.markdown" md_file.write_text("# Test") parent = MagicMock() parent.path = tmp_path parent.session = MagicMock() parent.config = MagicMock() result = pytest_collect_file(parent, md_file) assert isinstance(result, MarkdownFile) def test_collect_rst_file(self, tmp_path): """Test .rst file returns RSTFile.""" rst_file = tmp_path / "test.rst" rst_file.write_text("Test\n====") parent = MagicMock() parent.path = tmp_path parent.session = MagicMock() parent.config = MagicMock() result = pytest_collect_file(parent, rst_file) assert result is not None assert isinstance(result, RSTFile) def test_collect_other_file_returns_none(self, tmp_path): """Test other file types return None.""" txt_file = tmp_path / "test.txt" txt_file.write_text("Some text") parent = MagicMock() result = pytest_collect_file(parent, txt_file) assert result is None def test_collect_uppercase_extension(self, tmp_path): """Test case-insensitive extension matching.""" md_file = tmp_path / "test.MD" md_file.write_text("# Test") parent = MagicMock() parent.path = tmp_path parent.session = MagicMock() parent.config = MagicMock() result = pytest_collect_file(parent, md_file) assert isinstance(result, MarkdownFile) # ============================================================================ # Test md.py - parse_markdown function # ============================================================================ class TestParseMarkdown: """Test parse_markdown function.""" def test_parse_simple_codeblock(self): """Test basic code block parsing.""" text = """ ```python name=test_simple x = 1 ``` """ snippets = parse_markdown(text) assert len(snippets) == 1 assert snippets[0].name == "test_simple" assert "x = 1" in snippets[0].code # ------------------------------------------------------------------------ def test_parse_with_pytestmark(self): """Test the directive.""" text = """ ```python name=test_marked pass ``` """ snippets = parse_markdown(text) assert "django_db" in snippets[0].marks # ------------------------------------------------------------------------ def test_parse_with_pytestfixture(self): """Test the directive.""" text = """ ```python name=test_with_fixtures print("hello") ``` """ snippets = parse_markdown(text) assert len(snippets) == 1 assert "tmp_path" in snippets[0].fixtures assert "capsys" in snippets[0].fixtures # ------------------------------------------------------------------------ def test_parse_continue_directive(self): """Test the directive for grouping snippets.""" text = """ ```python name=test_setup x = 1 ``` Some text in between. ```python y = x + 1 assert y == 2 ``` """ snippets = parse_markdown(text) # Both blocks should be grouped under test_setup grouped = group_snippets(snippets) test_snippets = [s for s in grouped if s.name == "test_setup"] assert len(test_snippets) == 1 assert "x = 1" in test_snippets[0].code assert "y = x + 1" in test_snippets[0].code # ------------------------------------------------------------------------ def test_parse_incremental_continuation(self): """Named continuation blocks produce N cumulative tests.""" text = """ ```python name=test_something something = 1 ``` ```python name=test_something_2 something = "a" ``` ```python name=test_something_3 something = Exception("") ``` """ snippets = parse_markdown(text) grouped = group_snippets(snippets) assert len(grouped) == 3 assert grouped[0].name == "test_something" assert grouped[0].code.strip() == "something = 1" assert grouped[1].name == "test_something_2" assert "something = 1" in grouped[1].code assert 'something = "a"' in grouped[1].code assert grouped[2].name == "test_something_3" assert "something = 1" in grouped[2].code assert 'something = "a"' in grouped[2].code assert 'something = Exception("")' in grouped[2].code # ------------------------------------------------------------------------ def test_parse_codeblock_name_directive(self): """Test the directive.""" text = """ ```python z = 42 assert z == 42 ``` """ snippets = parse_markdown(text) assert len(snippets) == 1 assert snippets[0].name == "test_named" # ------------------------------------------------------------------------ def test_parse_py_language(self): """Test markdown with 'py' as language identifier.""" text = """ ```py name=test_py_lang x = 1 ``` """ snippets = parse_markdown(text) assert len(snippets) == 1 assert snippets[0].name == "test_py_lang" # ------------------------------------------------------------------------ def test_parse_python3_language(self): """Test markdown with 'python3' as language identifier.""" text = """ ```python3 name=test_python3 x = 1 ``` """ snippets = parse_markdown(text) assert len(snippets) == 1 assert snippets[0].name == "test_python3" # ------------------------------------------------------------------------ def test_parse_non_python_codeblock_ignored(self): """Test that non-Python code blocks are skipped.""" text = """ ```javascript name=test_js console.log("hi"); ``` ```python name=test_py x = 1 ``` """ snippets = parse_markdown(text) # Only Python blocks should be collected assert len(snippets) == 1 assert snippets[0].name == "test_py" # ------------------------------------------------------------------------ def test_parse_name_colon_syntax(self): """Test name= vs name: syntax in fence info string.""" text = """ ```python name:test_colon x = 1 ``` """ snippets = parse_markdown(text) assert snippets[0].name == "test_colon" # ------------------------------------------------------------------------ def test_parse_empty_codeblock(self): """Test parse empty code block.""" text = """ ```python name=test_empty ``` """ snippets = parse_markdown(text) assert len(snippets) == 1 assert snippets[0].code == "" # ------------------------------------------------------------------------ def test_parse_indented_fence(self): """Test fence with indentation.""" text = """ ```python name=test_indented x = 1 ``` """ snippets = parse_markdown(text) assert len(snippets) == 1 # ------------------------------------------------------------------------ # TODO: Remove? def test_parse_fence_regex_edge_case(self): """Test that malformed fence is handled.""" # This edge case is hard to trigger since ``` always matches text = """ ```python name=test_normal x = 1 ``` """ snippets = parse_markdown(text) assert len(snippets) == 1 # ------------------------------------------------------------------------ def test_parse_markdown_mixed_indentation(self): """Test parsing codeblock with mixed indentation levels.""" text = """ ```python name=test_indented x = 1 y = 2 z = 3 ``` """ snippets = parse_markdown(text) assert len(snippets) == 1 # Code should be dedented based on fence indentation assert "x = 1" in snippets[0].code # ------------------------------------------------------------------------ def test_parse_short_line_in_block(self): """Test code block with line shorter than indent.""" # Code block where some lines are shorter than the fence indentation text = """ ```python name=test_short_line x = 1 y z = 3 ``` """ snippets = parse_markdown(text) assert len(snippets) == 1 # The short line 'y' should still be captured assert "y" in snippets[0].code or "x = 1" in snippets[0].code # ============================================================================ # Test rst.py - resolve_literalinclude_path # ============================================================================ class TestResolveLiteralincludePath: """Test resolve_literalinclude_path function.""" def test_absolute_path_exists(self, tmp_path): """Test with an absolute path that exists.""" file = tmp_path / "test.py" file.write_text("print('hello')") result = resolve_literalinclude_path(tmp_path, str(file)) assert result == str(file.resolve()) def test_relative_path_exists(self, tmp_path): """Test with a relative path that exists.""" file = tmp_path / "subdir" / "test.py" file.parent.mkdir(parents=True) file.write_text("print('hello')") result = resolve_literalinclude_path(tmp_path, "subdir/test.py") assert result == str(file.resolve()) def test_base_is_file(self, tmp_path): """Test when base_dir is a file (uses parent).""" base_file = tmp_path / "doc.rst" base_file.write_text("some rst") target = tmp_path / "code.py" target.write_text("x = 1") # Pass the file as base_dir - function should use its parent result = resolve_literalinclude_path(base_file, "code.py") assert result == str(target.resolve()) def test_nonexistent_returns_none(self, tmp_path): """Test with a path that doesn't exist.""" result = resolve_literalinclude_path(tmp_path, "nonexistent.py") assert result is None def test_exception_handling(self, tmp_path): """Test exception branch.""" # Use a path that might cause issues result = resolve_literalinclude_path(tmp_path, "\x00invalid") assert result is None # ============================================================================ # Test rst.py - get_literalinclude_content # ============================================================================ class TestGetLiteralincludeContent: """Test get_literalinclude_content function.""" def test_read_success(self, tmp_path): """Test reads file correctly.""" file = tmp_path / "test.py" file.write_text("x = 42\ny = 43") content = get_literalinclude_content(str(file)) assert content == "x = 42\ny = 43" def test_read_failure(self, tmp_path): """Test get_literalinclude_content raises on missing file.""" with pytest.raises( RuntimeError, match="Failed to read literalinclude file" ): get_literalinclude_content(str(tmp_path / "missing.py")) # ============================================================================ # Test rst.py - parse_rst function # ============================================================================ class TestParseRst: """Test parse_rst function.""" def test_parse_code_block(self, tmp_path): """Test .. code-block:: python directive.""" rst = """ .. code-block:: python :name: test_rst x = 1 """ snippets = parse_rst(rst, tmp_path) assert len(snippets) == 1 assert snippets[0].name == "test_rst" # ------------------------------------------------------------------------ def test_parse_code_directive(self, tmp_path): """Test .. code:: python (alternative to code-block).""" rst = """ .. code:: python :name: test_code y = 2 """ snippets = parse_rst(rst, tmp_path) assert len(snippets) == 1 assert snippets[0].name == "test_code" # ------------------------------------------------------------------------ def test_parse_pytestmark(self, tmp_path): rst = """ .. pytestmark: django_db .. code-block:: python :name: test_marked pass """ snippets = parse_rst(rst, tmp_path) assert "django_db" in snippets[0].marks # ------------------------------------------------------------------------ def test_parse_pytestfixture(self, tmp_path): """Test the .. pytestfixture: directive.""" rst = """ .. pytestfixture: tmp_path .. code-block:: python :name: test_fixture_rst import os """ snippets = parse_rst(rst, tmp_path) assert len(snippets) == 1 assert "tmp_path" in snippets[0].fixtures # ------------------------------------------------------------------------ def test_parse_continue_directive(self, tmp_path): """Test the .. continue: directive for grouping RST snippets.""" rst = """ .. code-block:: python :name: test_rst_setup a = 10 Some text. .. continue: test_rst_setup .. code-block:: python b = a + 5 assert b == 15 """ snippets = parse_rst(rst, tmp_path) grouped = group_snippets(snippets) test_snippets = [s for s in grouped if s.name == "test_rst_setup"] assert len(test_snippets) == 1 assert "a = 10" in test_snippets[0].code assert "b = a + 5" in test_snippets[0].code # ------------------------------------------------------------------------ def test_parse_codeblock_name(self, tmp_path): rst = """ .. codeblock-name: test_named .. code-block:: python z = 99 """ snippets = parse_rst(rst, tmp_path) assert snippets[0].name == "test_named" # ------------------------------------------------------------------------ def test_parse_literal_block(self, tmp_path): """Test parsing of literal blocks via :: syntax.""" rst = """ .. codeblock-name: test_literal Example code:: result = 1 + 2 assert result == 3 """ snippets = parse_rst(rst, tmp_path) assert len(snippets) == 1 assert snippets[0].name == "test_literal" assert "result = 1 + 2" in snippets[0].code # ------------------------------------------------------------------------ def test_parse_rst_continue_in_literal_block(self, tmp_path): """Test continue directive with literal block syntax.""" rst = """ .. codeblock-name: test_lit_continue Part 1:: a = 1 .. continue: test_lit_continue .. codeblock-name: test_lit_continue Part 2:: b = 2 """ snippets = parse_rst(rst, tmp_path) grouped = group_snippets(snippets) # Should have grouped the snippets matching = [s for s in grouped if s.name == "test_lit_continue"] assert len(matching) >= 1 # ------------------------------------------------------------------------ def test_parse_literalinclude(self, tmp_path): """Test literalinclude directive with test_ name.""" # Create the file to include code_file = tmp_path / "example.py" code_file.write_text("def hello(): pass") rst = """ .. literalinclude:: example.py :name: test_include """ snippets = parse_rst(rst, tmp_path) assert len(snippets) == 1 assert "def hello():" in snippets[0].code # ------------------------------------------------------------------------ def test_parse_literalinclude_no_test_prefix(self, tmp_path): """Test literalinclude without test_ prefix is skipped.""" code_file = tmp_path / "example.py" code_file.write_text("x = 1") rst = """ .. literalinclude:: example.py :name: example_not_test """ snippets = parse_rst(rst, tmp_path) # Should be empty because name doesn't start with test_ assert len(snippets) == 0 # ------------------------------------------------------------------------ def test_parse_non_python_code_block(self, tmp_path): """Non-python code blocks are skipped.""" rst = """ .. code-block:: javascript console.log("hi"); """ snippets = parse_rst(rst, tmp_path) assert len(snippets) == 0 # ------------------------------------------------------------------------ def test_parse_wrong_indent(self, tmp_path): """Code at wrong indent level.""" rst = """ .. code-block:: python :name: test_wrong x = 1 """ # Content 'x = 1' is at column 0, not indented under the directive snippets = parse_rst(rst, tmp_path) # Should not collect this as a valid snippet assert len(snippets) == 0 # ------------------------------------------------------------------------ def test_parse_literal_codeblock_eof(self, tmp_path): """Test literal block at end of file.""" rst = """ .. codeblock-name: test_eof Block::""" # No content after the :: - end of file snippets = parse_rst(rst, tmp_path) # Should handle gracefully assert len(snippets) == 0 # ------------------------------------------------------------------------ def test_parse_empty_codeblock(self, tmp_path): """Test parsing an empty code block.""" rst = """ .. code-block:: python :name: test_empty """ snippets = parse_rst(rst, tmp_path) # Empty blocks are collected but have no snippets assert len(snippets) == 0 # ------------------------------------------------------------------------ def test_parse_literal_block_empty_line_after(self, tmp_path): """Test literal block with just empty line after (edge case).""" rst = """ .. codeblock-name: test_empty_after Block:: """ snippets = parse_rst(rst, tmp_path) # Empty block at end assert len(snippets) == 0 # ============================================================================ # Integration tests using pytester - exercises collectors and hook # ============================================================================ # ---------------------------------------------------------------------------- # Test RSTFile.collect() method # ---------------------------------------------------------------------------- class TestMarkdownCollector: """Integration tests for MarkdownFile collector.""" def test_collect_simple_markdown(self, pytester_subprocess): """Test that MarkdownFile collects and runs test snippets.""" pytester_subprocess.makefile( ".md", test_simple=""" # Test File ```python name=test_basic x = 1 assert x == 1 ``` """, ) result = pytester_subprocess.runpytest("-v", "-p", "no:django") result.assert_outcomes(passed=1) assert "test_basic" in result.stdout.str() # ------------------------------------------------------------------------ def test_collect_with_fixture(self, pytester_subprocess): """Test that fixtures are properly injected.""" pytester_subprocess.makefile( ".md", test_fixture=""" ```python name=test_uses_tmp_path assert tmp_path.exists() ``` """, ) result = pytester_subprocess.runpytest("-v", "-p", "no:django") result.assert_outcomes(passed=1) # ------------------------------------------------------------------------ def test_collect_async_code(self, pytester_subprocess): """Test that async code is automatically wrapped.""" pytester_subprocess.makefile( ".md", test_async=""" ```python name=test_async_snippet import asyncio await asyncio.sleep(0) ``` """, ) result = pytester_subprocess.runpytest("-v", "-p", "no:django") result.assert_outcomes(passed=1) # ------------------------------------------------------------------------ def test_syntax_error_reporting(self, pytester_subprocess): """Test that syntax errors in snippets are properly reported.""" pytester_subprocess.makefile( ".md", test_syntax=""" ```python name=test_bad_syntax def broken(: pass ``` """, ) result = pytester_subprocess.runpytest("-v", "-p", "no:django") result.assert_outcomes(failed=1) assert ( "SyntaxError" in result.stdout.str() or "syntax" in result.stdout.str().lower() ) # ------------------------------------------------------------------------ def test_runtime_error_reporting(self, pytester_subprocess): """Test that runtime errors in snippets are properly reported.""" pytester_subprocess.makefile( ".md", test_runtime=""" ```python name=test_runtime_error raise ValueError("intentional error") ``` """, ) result = pytester_subprocess.runpytest("-v", "-p", "no:django") result.assert_outcomes(failed=1) assert "ValueError" in result.stdout.str() # ------------------------------------------------------------------------ def test_incremental_continue_collects_separate_tests( self, pytester_subprocess ): """Each named continuation block becomes its own cumulative test.""" pytester_subprocess.makefile( ".md", test_incremental=""" ```python name=test_step_one something = 1 assert something == 1 ``` ```python name=test_step_two something = "a" assert something == "a" ``` ```python name=test_step_three something = Exception("") assert isinstance(something, Exception) ``` """, ) result = pytester_subprocess.runpytest("-v", "-p", "no:django") result.assert_outcomes(passed=3) stdout = result.stdout.str() assert "test_step_one" in stdout assert "test_step_two" in stdout assert "test_step_three" in stdout # ---------------------------------------------------------------------------- # Test RSTFile.collect() method # ---------------------------------------------------------------------------- class TestRSTCollector: """Integration tests for RSTFile collector.""" def test_collect_simple_rst(self, pytester_subprocess): """Test that RSTFile collects and runs test snippets.""" pytester_subprocess.makefile( ".rst", test_simple=""" Test File ========= .. code-block:: python :name: test_rst_basic y = 2 assert y == 2 """, ) result = pytester_subprocess.runpytest("-v", "-p", "no:django") result.assert_outcomes(passed=1) assert "test_rst_basic" in result.stdout.str() # ------------------------------------------------------------------------ def test_collect_with_fixture(self, pytester_subprocess): """Test that RST fixtures are properly injected.""" pytester_subprocess.makefile( ".rst", test_fixture=""" .. pytestfixture: tmp_path .. code-block:: python :name: test_rst_fixture assert tmp_path.is_dir() """, ) result = pytester_subprocess.runpytest("-v", "-p", "no:django") result.assert_outcomes(passed=1) # ------------------------------------------------------------------------ def test_collect_async_code(self, pytester_subprocess): """Test that RST async code is automatically wrapped.""" pytester_subprocess.makefile( ".rst", test_async=""" .. code-block:: python :name: test_rst_async import asyncio await asyncio.sleep(0) """, ) result = pytester_subprocess.runpytest("-v", "-p", "no:django") result.assert_outcomes(passed=1) # ------------------------------------------------------------------------ def test_syntax_error_reporting(self, pytester_subprocess): """Test that syntax errors in RST snippets are reported.""" pytester_subprocess.makefile( ".rst", test_syntax=""" .. code-block:: python :name: test_rst_bad_syntax class Broken(: pass """, ) result = pytester_subprocess.runpytest("-v", "-p", "no:django") result.assert_outcomes(failed=1) # ---------------------------------------------------------------------------- # Tests for pytest_collect_file hook dispatch # ---------------------------------------------------------------------------- class TestPytestCollectFileHook: """Tests for pytest_collect_file hook dispatch.""" def test_hook_dispatches_markdown(self, pytester_subprocess): """Test that .md files are dispatched to MarkdownFile.""" pytester_subprocess.makefile( ".md", readme=""" ```python name=test_md_hook assert True ``` """, ) result = pytester_subprocess.runpytest( "-v", "--collect-only", "-p", "no:django" ) assert "test_md_hook" in result.stdout.str() # ------------------------------------------------------------------------ def test_hook_dispatches_rst(self, pytester_subprocess): """Test that .rst files are dispatched to RSTFile.""" pytester_subprocess.makefile( ".rst", readme=""" .. code-block:: python :name: test_rst_hook assert True """, ) result = pytester_subprocess.runpytest( "-v", "--collect-only", "-p", "no:django" ) assert "test_rst_hook" in result.stdout.str() # ------------------------------------------------------------------------ def test_hook_ignores_other_files(self, pytester_subprocess): """Test that non-.md/.rst files are ignored.""" pytester_subprocess.makefile(".txt", notes="Some notes") result = pytester_subprocess.runpytest( "-v", "--collect-only", "-p", "no:django" ) # Should not fail, just collect nothing from .txt assert result.ret == 5 # Exit code 5 = no tests collected # --------------------------------------------------------------------------- # Tests for Django DB mark handling # --------------------------------------------------------------------------- class TestDjangoDbMarks: """Tests for Django DB mark handling.""" def test_django_db_mark_applied(self, pytester_subprocess): """Test that django_db mark is applied when specified.""" pytester_subprocess.makefile( ".md", test_marks=""" ```python name=test_with_db_mark # This would use the db fixture in a real Django project x = 1 ``` """, ) result = pytester_subprocess.runpytest( "-v", "--collect-only", "-p", "no:django" ) assert "test_with_db_mark" in result.stdout.str() # The mark should be present (we can't fully test Django integration # without Django) src/pytest_codeblock/tests/test_nameless_codeblocks.py ------------------------------------------------------ src/pytest_codeblock/tests/test_nameless_codeblocks.py """ Unit tests for the test_nameless_codeblocks configuration feature. Tests cover: - Configuration loading - Auto-naming behavior - Markdown collector - RST collector - Integration scenarios - Edge cases """ from pathlib import Path from unittest.mock import MagicMock, patch import pytest from ..collector import group_snippets from ..config import Config from ..constants import CODEBLOCK_MARK from ..md import parse_markdown from ..rst import parse_rst __author__ = "Artur Barseghyan " __copyright__ = "2025-2026 Artur Barseghyan" __license__ = "MIT" __all__ = ( "TestConfigLoading", "TestAutoNaming", "TestMarkdownNameless", "TestRSTNameless", "TestIntegration", "TestEdgeCases", ) # ============================================================================ # Test Configuration Loading # ============================================================================ class TestConfigLoading: """Test that test_nameless_codeblocks config loads correctly.""" def test_default_value_is_false(self): """Test that default value is False.""" config = Config() assert config.test_nameless_codeblocks is False def test_explicit_true_value(self): """Test setting value to True.""" config = Config(test_nameless_codeblocks=True) assert config.test_nameless_codeblocks is True def test_explicit_false_value(self): """Test setting value to False explicitly.""" config = Config(test_nameless_codeblocks=False) assert config.test_nameless_codeblocks is False def test_config_from_dict(self): """Test loading from configuration dict.""" # Simulate loading from pyproject.toml raw = {"test_nameless_codeblocks": True} config = Config( test_nameless_codeblocks=raw.get("test_nameless_codeblocks", False) ) assert config.test_nameless_codeblocks is True def test_config_from_dict_missing_key(self): """Test loading from dict without the key uses default.""" raw = {} config = Config( test_nameless_codeblocks=raw.get("test_nameless_codeblocks", False) ) assert config.test_nameless_codeblocks is False # ============================================================================ # Test Auto-naming Logic # ============================================================================ class TestAutoNaming: """Test the auto-naming scheme for nameless code blocks.""" def test_auto_name_format(self): """ Test that auto-generated names follow test_{module}_{counter} format. """ module_name = "README" counter = 1 auto_name = f"test_{module_name}_{counter}" assert auto_name == "test_README_1" def test_auto_name_increment(self): """Test that counter increments for multiple nameless blocks.""" module_name = "guide" names = [f"test_{module_name}_{i}" for i in range(1, 4)] assert names == ["test_guide_1", "test_guide_2", "test_guide_3"] def test_module_name_extraction(self): """Test extracting module name from Path.stem.""" path = Path("/path/to/README.md") module_name = path.stem assert module_name == "README" def test_module_name_with_dots(self): """Test module name extraction with dots in filename.""" path = Path("/path/to/my.doc.md") module_name = path.stem # Path.stem returns everything before the last dot assert module_name == "my.doc" # ============================================================================ # Test Markdown Collector with Nameless Blocks # ============================================================================ class TestMarkdownNameless: """Test MarkdownFile collector with test_nameless_codeblocks.""" def test_parse_markdown_nameless_snippets(self): """Test that parse_markdown extracts nameless snippets.""" text = """ ```python x = 1 ``` ```python y = 2 ``` """ snippets = parse_markdown(text) # parse_markdown should extract them (name=None) assert len(snippets) == 2 assert snippets[0].name is None assert snippets[1].name is None assert "x = 1" in snippets[0].code assert "y = 2" in snippets[1].code # ------------------------------------------------------------------------ def test_parse_markdown_mixed_named_and_nameless(self): """Test parsing mix of named and nameless blocks.""" text = """ ```python name=test_one a = 1 ``` ```python b = 2 ``` ```python name=test_two c = 3 ``` ```python d = 4 ``` """ snippets = parse_markdown(text) assert len(snippets) == 4 assert snippets[0].name == "test_one" assert snippets[1].name is None assert snippets[2].name == "test_two" assert snippets[3].name is None # ------------------------------------------------------------------------ def test_collect_nameless_disabled_default(self): """Test that nameless blocks are ignored by default.""" text = """ ```python name=test_explicit x = 1 ``` ```python y = 2 ``` """ # Mock config with default (False) mock_config = Config(test_nameless_codeblocks=False) with patch("pytest_codeblock.md.get_config", return_value=mock_config): # Create a mock MarkdownFile parent = MagicMock() parent.config = MagicMock() path = Path("/tmp/test.md") # We can't easily test collect() without full pytest setup, # but we can test the filtering logic raw = parse_markdown(text) # Apply the filtering logic from collect() if mock_config.test_nameless_codeblocks: tests = [] counter = 1 module_name = path.stem for sn in raw: if sn.name and sn.name.startswith("test_"): tests.append(sn) elif not sn.name: auto_name = f"test_{module_name}_{counter}" counter += 1 sn.name = auto_name tests.append(sn) else: tests = [ sn for sn in raw if sn.name and sn.name.startswith("test_") ] # Should only have the named test assert len(tests) == 1 assert tests[0].name == "test_explicit" # ------------------------------------------------------------------------ def test_collect_nameless_enabled(self): """Test that nameless blocks are collected when enabled.""" text = """ ```python name=test_explicit x = 1 ``` ```python y = 2 ``` ```python z = 3 ``` """ # Mock config with feature enabled mock_config = Config(test_nameless_codeblocks=True) with patch("pytest_codeblock.md.get_config", return_value=mock_config): path = Path("/tmp/test.md") raw = parse_markdown(text) # Apply the filtering logic from collect() tests = [] counter = 1 module_name = path.stem for sn in raw: if sn.name and sn.name.startswith("test_"): tests.append(sn) elif not sn.name: auto_name = f"test_{module_name}_{counter}" counter += 1 sn.name = auto_name tests.append(sn) # Should have all three blocks assert len(tests) == 3 assert tests[0].name == "test_explicit" assert tests[1].name == "test_test_1" assert tests[2].name == "test_test_2" # ------------------------------------------------------------------------ def test_auto_naming_preserves_code(self): """Test that auto-naming doesn't modify the code content.""" text = """ ```python original_code = "unchanged" assert original_code == "unchanged" ``` """ mock_config = Config(test_nameless_codeblocks=True) with patch("pytest_codeblock.md.get_config", return_value=mock_config): path = Path("/tmp/myfile.md") raw = parse_markdown(text) tests = [] counter = 1 module_name = path.stem for sn in raw: if not sn.name: sn.name = f"test_{module_name}_{counter}" counter += 1 tests.append(sn) assert len(tests) == 1 assert tests[0].name == "test_myfile_1" assert "original_code" in tests[0].code assert "unchanged" in tests[0].code # ------------------------------------------------------------------------ def test_auto_naming_preserves_marks(self): """Test that auto-naming preserves pytest marks.""" text = """ ```python from django.contrib.auth.models import User user = User.objects.first() ``` """ mock_config = Config(test_nameless_codeblocks=True) with patch("pytest_codeblock.md.get_config", return_value=mock_config): path = Path("/tmp/test.md") raw = parse_markdown(text) tests = [] counter = 1 module_name = path.stem for sn in raw: if not sn.name: sn.name = f"test_{module_name}_{counter}" counter += 1 tests.append(sn) assert len(tests) == 1 assert "django_db" in tests[0].marks # ------------------------------------------------------------------------ def test_codeblock_marks_on_all_blocks(self): """Test that all blocks have default codeblock marks.""" text = """ ```python assert True ``` ```python assert True ``` """ mock_config = Config(test_nameless_codeblocks=True) with patch("pytest_codeblock.md.get_config", return_value=mock_config): raw = parse_markdown(text) assert len(raw) == 2 for sn in raw: assert CODEBLOCK_MARK in sn.marks # ------------------------------------------------------------------------ def test_auto_naming_preserves_fixtures(self): """Test that auto-naming preserves pytest fixtures.""" text = """ ```python d = tmp_path / "test" d.mkdir() ``` """ mock_config = Config(test_nameless_codeblocks=True) with patch("pytest_codeblock.md.get_config", return_value=mock_config): path = Path("/tmp/test.md") raw = parse_markdown(text) tests = [] counter = 1 module_name = path.stem for sn in raw: if not sn.name: sn.name = f"test_{module_name}_{counter}" counter += 1 tests.append(sn) assert len(tests) == 1 assert "tmp_path" in tests[0].fixtures assert "capsys" in tests[0].fixtures # ============================================================================ # Test RST Collector with Nameless Blocks # ============================================================================ class TestRSTNameless: """Test RSTFile collector with test_nameless_codeblocks.""" def test_parse_rst_nameless_snippets(self, tmp_path): """Test that parse_rst extracts nameless snippets.""" text = """ .. code-block:: python x = 1 .. code-block:: python y = 2 """ snippets = parse_rst(text, tmp_path) # parse_rst should extract them (name=None) assert len(snippets) == 2 assert snippets[0].name is None assert snippets[1].name is None assert "x = 1" in snippets[0].code assert "y = 2" in snippets[1].code # ------------------------------------------------------------------------ def test_parse_rst_mixed_named_and_nameless(self, tmp_path): """Test parsing mix of named and nameless blocks.""" text = """ .. code-block:: python :name: test_one a = 1 .. code-block:: python b = 2 .. code-block:: python :name: test_two c = 3 .. code-block:: python d = 4 """ snippets = parse_rst(text, tmp_path) assert len(snippets) == 4 assert snippets[0].name == "test_one" assert snippets[1].name is None assert snippets[2].name == "test_two" assert snippets[3].name is None # ------------------------------------------------------------------------ def test_collect_nameless_disabled_default(self, tmp_path): """Test that nameless blocks are ignored by default in RST.""" text = """ .. code-block:: python :name: test_explicit x = 1 .. code-block:: python y = 2 """ mock_config = Config(test_nameless_codeblocks=False) with patch( "pytest_codeblock.rst.get_config", return_value=mock_config ): path = Path("/tmp/test.rst") raw = parse_rst(text, tmp_path) # Apply the filtering logic from collect() if mock_config.test_nameless_codeblocks: tests = [] counter = 1 module_name = path.stem for sn in raw: if sn.name and sn.name.startswith("test_"): tests.append(sn) elif not sn.name: auto_name = f"test_{module_name}_{counter}" counter += 1 sn.name = auto_name tests.append(sn) else: tests = [ sn for sn in raw if sn.name and sn.name.startswith("test_") ] # Should only have the named test assert len(tests) == 1 assert tests[0].name == "test_explicit" # ------------------------------------------------------------------------ def test_collect_nameless_enabled(self, tmp_path): """Test that nameless blocks are collected when enabled in RST.""" text = """ .. code-block:: python :name: test_explicit x = 1 .. code-block:: python y = 2 .. code-block:: python z = 3 """ mock_config = Config(test_nameless_codeblocks=True) with patch( "pytest_codeblock.rst.get_config", return_value=mock_config ): path = Path("/tmp/test.rst") raw = parse_rst(text, tmp_path) # Apply the filtering logic from collect() tests = [] counter = 1 module_name = path.stem for sn in raw: if sn.name and sn.name.startswith("test_"): tests.append(sn) elif not sn.name: auto_name = f"test_{module_name}_{counter}" counter += 1 sn.name = auto_name tests.append(sn) # Should have all three blocks assert len(tests) == 3 assert tests[0].name == "test_explicit" assert tests[1].name == "test_test_1" assert tests[2].name == "test_test_2" # ------------------------------------------------------------------------ def test_auto_naming_preserves_code_rst(self, tmp_path): """Test that auto-naming doesn't modify the code content in RST.""" text = """ .. code-block:: python original_code = "unchanged" assert original_code == "unchanged" """ mock_config = Config(test_nameless_codeblocks=True) with patch( "pytest_codeblock.rst.get_config", return_value=mock_config ): path = Path("/tmp/myfile.rst") raw = parse_rst(text, tmp_path) tests = [] counter = 1 module_name = path.stem for sn in raw: if not sn.name: sn.name = f"test_{module_name}_{counter}" counter += 1 tests.append(sn) assert len(tests) == 1 assert tests[0].name == "test_myfile_1" assert "original_code" in tests[0].code assert "unchanged" in tests[0].code # ------------------------------------------------------------------------ def test_auto_naming_preserves_marks_rst(self, tmp_path): """Test that auto-naming preserves pytest marks in RST.""" text = """ .. pytestmark: django_db .. code-block:: python from django.contrib.auth.models import User user = User.objects.first() """ mock_config = Config(test_nameless_codeblocks=True) with patch( "pytest_codeblock.rst.get_config", return_value=mock_config ): path = Path("/tmp/test.rst") raw = parse_rst(text, tmp_path) tests = [] counter = 1 module_name = path.stem for sn in raw: if not sn.name: sn.name = f"test_{module_name}_{counter}" counter += 1 tests.append(sn) assert len(tests) == 1 assert "django_db" in tests[0].marks # ------------------------------------------------------------------------ def test_codeblock_marks_on_all_blocks_rst(self, tmp_path): """Test that all blocks in RST file have default codeblock mark.""" text = """ .. code-block:: python assert True .. code-block::python assert True """ mock_config = Config(test_nameless_codeblocks=True) with patch( "pytest_codeblock.rst.get_config", return_value=mock_config ): raw = parse_rst(text, tmp_path) assert len(raw) == 2 for sn in raw: assert CODEBLOCK_MARK in sn.marks # ------------------------------------------------------------------------ def test_auto_naming_preserves_fixtures_rst(self, tmp_path): """Test that auto-naming preserves pytest fixtures in RST.""" text = """ .. pytestfixture: tmp_path .. pytestfixture: capsys .. code-block:: python d = tmp_path / "test" d.mkdir() """ mock_config = Config(test_nameless_codeblocks=True) with patch( "pytest_codeblock.rst.get_config", return_value=mock_config ): path = Path("/tmp/test.rst") raw = parse_rst(text, tmp_path) tests = [] counter = 1 module_name = path.stem for sn in raw: if not sn.name: sn.name = f"test_{module_name}_{counter}" counter += 1 tests.append(sn) assert len(tests) == 1 assert "tmp_path" in tests[0].fixtures assert "capsys" in tests[0].fixtures # ============================================================================ # Test Integration Scenarios # ============================================================================ @pytest.mark.skip( reason="Skip due to pytest 9 py.path.local deprecation issue in hooks" ) class TestIntegration: """Integration tests using pytester.""" def test_markdown_nameless_integration(self, pytester): """Test nameless blocks work end-to-end in Markdown.""" pytester.makepyprojecttoml(""" [tool.pytest-codeblock] test_nameless_codeblocks = true """) pytester.makefile(".md", test_integration=""" # Test File ```python name=test_explicit x = 1 assert x == 1 ``` ```python y = 2 assert y == 2 ``` ```python z = 3 assert z == 3 ``` """) result = pytester.runpytest("-v", "-p", "no:django") result.assert_outcomes(passed=3) assert "test_explicit" in result.stdout.str() assert "test_integration_1" in result.stdout.str() assert "test_integration_2" in result.stdout.str() # ------------------------------------------------------------------------ def test_rst_nameless_integration(self, pytester): """Test nameless blocks work end-to-end in RST.""" pytester.makepyprojecttoml(""" [tool.pytest-codeblock] test_nameless_codeblocks = true """) pytester.makefile(".rst", test_integration=""" Test File ========= .. code-block:: python :name: test_explicit x = 1 assert x == 1 .. code-block:: python y = 2 assert y == 2 .. code-block:: python z = 3 assert z == 3 """) result = pytester.runpytest("-v", "-p", "no:django") result.assert_outcomes(passed=3) assert "test_explicit" in result.stdout.str() assert "test_integration_1" in result.stdout.str() assert "test_integration_2" in result.stdout.str() # ------------------------------------------------------------------------ def test_nameless_disabled_integration(self, pytester): """Test that nameless blocks are ignored when disabled.""" # Don't set test_nameless_codeblocks (default False) pytester.makefile(".md", test_default=""" # Test File ```python name=test_explicit x = 1 assert x == 1 ``` ```python y = 2 assert y == 2 ``` """) result = pytester.runpytest("-v", "-p", "no:django") result.assert_outcomes(passed=1) assert "test_explicit" in result.stdout.str() assert "test_default_1" not in result.stdout.str() # ------------------------------------------------------------------------ def test_multiple_files_separate_counters(self, pytester): """Test that each file has its own counter.""" pytester.makepyprojecttoml(""" [tool.pytest-codeblock] test_nameless_codeblocks = true """) pytester.makefile(".md", file1=""" ```python x = 1 ``` ```python y = 2 ``` """) pytester.makefile(".md", file2=""" ```python a = 1 ``` ```python b = 2 ``` """) result = pytester.runpytest("-v", "-p", "no:django") result.assert_outcomes(passed=4) # Each file should have _1 and _2 assert "test_file1_1" in result.stdout.str() assert "test_file1_2" in result.stdout.str() assert "test_file2_1" in result.stdout.str() assert "test_file2_2" in result.stdout.str() # ============================================================================ # Test Edge Cases # ============================================================================ class TestEdgeCases: """Test edge cases and corner scenarios.""" @pytest.mark.skip( reason="Skip due to pytest 9 py.path.local deprecation issue in hooks" ) def test_only_nameless_blocks(self, pytester): """Test file with only nameless blocks.""" pytester.makepyprojecttoml(""" [tool.pytest-codeblock] test_nameless_codeblocks = true """) pytester.makefile(".md", only_nameless=""" ```python x = 1 ``` ```python y = 2 ``` ```python z = 3 ``` """) result = pytester.runpytest("-v", "-p", "no:django") result.assert_outcomes(passed=3) # ------------------------------------------------------------------------ @pytest.mark.skip( reason="Skip due to pytest 9 py.path.local deprecation issue in hooks" ) def test_only_named_blocks(self, pytester): """Test file with only named blocks.""" pytester.makepyprojecttoml(""" [tool.pytest-codeblock] test_nameless_codeblocks = true """) pytester.makefile(".md", only_named=""" ```python name=test_one x = 1 ``` ```python name=test_two y = 2 ``` """) result = pytester.runpytest("-v", "-p", "no:django") result.assert_outcomes(passed=2) assert "test_one" in result.stdout.str() assert "test_two" in result.stdout.str() # No auto-generated names assert "test_only_named_1" not in result.stdout.str() # ------------------------------------------------------------------------ @pytest.mark.skip( reason="Skip due to pytest 9 py.path.local deprecation issue in hooks" ) def test_empty_code_blocks(self, pytester): """Test that empty nameless blocks are handled.""" pytester.makepyprojecttoml(""" [tool.pytest-codeblock] test_nameless_codeblocks = true """) pytester.makefile(".md", empty=""" ```python ``` ```python x = 1 assert x == 1 ``` """) # Empty blocks might not be collected by parser result = pytester.runpytest("-v", "-p", "no:django") # Should have at least the non-empty one assert result.ret == 0 # ------------------------------------------------------------------------ @pytest.mark.skip( reason="Skip due to pytest 9 py.path.local deprecation issue in hooks" ) def test_non_python_blocks_ignored(self, pytester): """Test that non-Python blocks are still ignored.""" pytester.makepyprojecttoml(""" [tool.pytest-codeblock] test_nameless_codeblocks = true """) pytester.makefile(".md", mixed_lang=""" ```python x = 1 ``` ```javascript console.log("ignored"); ``` ```python y = 2 ``` """) result = pytester.runpytest("-v", "-p", "no:django") result.assert_outcomes(passed=2) # Only Python blocks should be collected # ------------------------------------------------------------------------ def test_nameless_with_continue_directive_md(self): """Test nameless blocks with continue directive in Markdown.""" text = """ ```python name=test_base x = 1 ``` ```python y = x + 1 assert y == 2 ``` """ mock_config = Config(test_nameless_codeblocks=True) with patch("pytest_codeblock.md.get_config", return_value=mock_config): raw = parse_markdown(text) # Group snippets as in collect() combined = group_snippets(raw) # Should have merged into one test_base assert len(combined) == 1 assert combined[0].name == "test_base" assert "x = 1" in combined[0].code assert "y = x + 1" in combined[0].code # ------------------------------------------------------------------------ def test_nameless_with_continue_directive_rst(self, tmp_path): """Test nameless blocks with continue directive in RST.""" text = """ .. code-block:: python :name: test_base x = 1 .. continue: test_base .. code-block:: python y = x + 1 assert y == 2 """ mock_config = Config(test_nameless_codeblocks=True) with patch( "pytest_codeblock.rst.get_config", return_value=mock_config ): raw = parse_rst(text, tmp_path) # Group snippets as in collect() combined = group_snippets(raw) # Should have merged into one test_base assert len(combined) == 1 assert combined[0].name == "test_base" assert "x = 1" in combined[0].code assert "y = x + 1" in combined[0].code # ------------------------------------------------------------------------ def test_counter_only_increments_for_nameless(self): """Test that counter only increments for nameless blocks.""" text = """ ```python name=test_explicit_1 a = 1 ``` ```python b = 2 ``` ```python name=test_explicit_2 c = 3 ``` ```python d = 4 ``` ```python name=test_explicit_3 e = 5 ``` ```python f = 6 ``` """ mock_config = Config(test_nameless_codeblocks=True) with patch( "pytest_codeblock.md.get_config", return_value=mock_config ): path = Path("/tmp/test.md") raw = parse_markdown(text) tests = [] counter = 1 module_name = path.stem for sn in raw: if sn.name and sn.name.startswith("test_"): tests.append(sn) elif not sn.name: sn.name = f"test_{module_name}_{counter}" counter += 1 tests.append(sn) # Should have 6 tests total assert len(tests) == 6 # Named blocks keep their names assert tests[0].name == "test_explicit_1" assert tests[2].name == "test_explicit_2" assert tests[4].name == "test_explicit_3" # Nameless blocks get sequential numbers assert tests[1].name == "test_test_1" assert tests[3].name == "test_test_2" assert tests[5].name == "test_test_3" # ------------------------------------------------------------------------ def test_filename_with_special_characters(self): """Test auto-naming with special characters in filename.""" # Path.stem handles most special chars path = Path("/tmp/my-test_file.md") module_name = path.stem auto_name = f"test_{module_name}_1" assert auto_name == "test_my-test_file_1" # ------------------------------------------------------------------------ @pytest.mark.skip( reason="Skip due to pytest 9 py.path.local deprecation issue in hooks" ) def test_both_formats_disabled(self, pytester): """Test both MD and RST with feature disabled.""" pytester.makefile(".md", test_md=""" ```python x = 1 ``` """) pytester.makefile(".rst", test_rst=""" .. code-block:: python y = 2 """) result = pytester.runpytest("-v", "-p", "no:django") # No tests should be collected assert result.ret == 5 # Exit code 5 = no tests collected # ------------------------------------------------------------------------ @pytest.mark.skip( reason="Skip due to pytest 9 py.path.local deprecation issue in hooks" ) def test_both_formats_enabled(self, pytester): """Test both MD and RST with feature enabled.""" pytester.makepyprojecttoml(""" [tool.pytest-codeblock] test_nameless_codeblocks = true """) pytester.makefile(".md", test_md=""" ```python x = 1 assert x == 1 ``` """) pytester.makefile(".rst", test_rst=""" .. code-block:: python y = 2 assert y == 2 """) result = pytester.runpytest("-v", "-p", "no:django") result.assert_outcomes(passed=2) assert "test_md_1" in result.stdout.str() assert "test_rst_1" in result.stdout.str() src/pytest_codeblock/tests/test_pytest_codeblock.py --------------------------------------------------- src/pytest_codeblock/tests/test_pytest_codeblock.py from ..collector import CodeSnippet, group_snippets from ..helpers import contains_top_level_await, wrap_async_code from ..md import parse_markdown from ..rst import ( get_literalinclude_content, parse_rst, resolve_literalinclude_path, ) __author__ = "Artur Barseghyan " __copyright__ = "2025-2026 Artur Barseghyan" __license__ = "MIT" __all__ = ( "test_group_snippets_different_names", "test_group_snippets_merges_named", "test_parse_markdown_simple", "test_parse_markdown_with_pytestmark", "test_parse_rst_literalinclude", "test_parse_rst_simple", "test_resolve_literalinclude_and_content", ) def test_group_snippets_merges_named(): # Two snippets with the same name should be combined sn1 = CodeSnippet(name="foo", code="a=1", line=1, marks=["codeblock"]) sn2 = CodeSnippet(name="foo", code="b=2", line=2, marks=["codeblock", "m"]) combined = group_snippets([sn1, sn2]) assert len(combined) == 1 cs = combined[0] assert cs.name == "foo" # Both code parts should appear assert "a=1" in cs.code assert "b=2" in cs.code # Marks should accumulate assert "m" in cs.marks def test_group_snippets_incremental(): # Continuation snippets with distinct own names produce N cumulative tests. sn1 = CodeSnippet(name="test_root", code="a=1", line=1) sn2 = CodeSnippet(name="test_step2", code="b=2", line=5, group="test_root") sn3 = CodeSnippet(name="test_step3", code="c=3", line=9, group="test_root") combined = group_snippets([sn1, sn2, sn3]) assert len(combined) == 3 assert combined[0].name == "test_root" assert combined[1].name == "test_step2" assert combined[2].name == "test_step3" assert combined[0].code == "a=1" assert combined[1].code == "a=1\nb=2" assert combined[2].code == "a=1\nb=2\nc=3" def test_group_snippets_different_names(): # Snippets with different names are not grouped sn1 = CodeSnippet(name="foo", code="x=1", line=1) sn2 = CodeSnippet(name="bar", code="y=2", line=2) combined = group_snippets([sn1, sn2]) assert len(combined) == 2 assert combined[0].name.startswith("foo") assert combined[1].name.startswith("bar") def test_parse_markdown_simple(): text = """ ```python name=test_example x=1 assert x==1 ```""" snippets = parse_markdown(text) assert len(snippets) == 1 sn = snippets[0] assert sn.name == "test_example" assert "x=1" in sn.code def test_parse_markdown_with_pytestmark(): text = """ ```python name=test_db from django.db import models ```""" snippets = parse_markdown(text) assert len(snippets) == 1 sn = snippets[0] # Should include both default and django_db marks assert "django_db" in sn.marks assert "codeblock" in sn.marks def test_resolve_literalinclude_and_content(tmp_path): base = tmp_path / "dir" base.mkdir() file = base / "a.py" file.write_text("print('hello')") # Absolute path resolution abs_path = resolve_literalinclude_path(base, str(file)) assert abs_path == str(file.resolve()) # Relative path resolution rel_path = resolve_literalinclude_path(base, "a.py") assert rel_path == str(file.resolve()) # Content read content = get_literalinclude_content(str(file)) assert content == "print('hello')" def test_parse_rst_simple(tmp_path): # Basic code-block directive rst = """ .. code-block:: python :name: test_simple a=2 assert a==2 """ snippets = parse_rst(rst, tmp_path) assert len(snippets) == 1 sn = snippets[0] assert sn.name == "test_simple" assert "a=2" in sn.code def test_parse_rst_literalinclude(tmp_path): # Create an external file to include include_dir = tmp_path / "inc" include_dir.mkdir() target = include_dir / "foo.py" target.write_text("z=3\nassert z==3") rst = f""" .. literalinclude:: {target.name} :name: test_li """ snippets = parse_rst(rst, include_dir) assert len(snippets) == 1 sn = snippets[0] assert sn.name == "test_li" assert "z=3" in sn.code def test_contains_top_level_await_positive(): """Verify detection of various async constructs.""" # Direct await assert contains_top_level_await("await asyncio.sleep(0)") is True # Async function definition assert contains_top_level_await("async def foo(): pass") is True # Async with assert contains_top_level_await("async with lock: pass") is True # Async for assert contains_top_level_await("async for i in range(1): pass") is True def test_contains_top_level_await_negative(): """Verify that sync code or strings containing keywords are ignored.""" # Standard sync code assert contains_top_level_await("import time; time.sleep(1)") is False # Keywords inside strings assert contains_top_level_await("print('this is an await')") is False # Comments should be ignored assert contains_top_level_await("# await inside comment") is False def test_contains_top_level_await_invalid_syntax(): """Verify that invalid syntax returns False rather than crashing.""" assert contains_top_level_await("def main(:") is False def test_wrap_async_code_structure(): """Verify the transformation logic and indentation.""" code = "await asyncio.sleep(1)\nreturn 42" wrapped = wrap_async_code(code) # Check for the boilerplate components assert "async def __async_main__():" in wrapped assert "asyncio.run(__async_main__())" in wrapped # Check that the original code is indented correctly (4 spaces) assert " await asyncio.sleep(1)" in wrapped assert " return 42" in wrapped def test_wrap_async_code_execution_integrity(): """ Verify that the wrapped code is still valid Python and can be compiled. This ensures wrap_async_code doesn't break the AST. """ code = "val = 1 + 1" wrapped = wrap_async_code(code) # If compile fails, the test fails assert compile(wrapped, "", "exec") src/pytest_codeblock/tests/test_pytestrun_marker.py --------------------------------------------------- src/pytest_codeblock/tests/test_pytestrun_marker.py """ Tests for the `pytestrun` marker functionality. When a code block is marked with ``pytestrun``, the plugin writes the block to a temporary file and executes it via pytest as a subprocess, so that ``Test*`` classes, ``test_*`` functions, fixtures, markers, and setup/teardown all behave exactly as they would in a normal pytest run. """ import textwrap import pytest from ..constants import CODEBLOCK_MARK, PYTESTRUN_MARK from ..md import parse_markdown from ..pytestrun import run_pytest_style_code from ..rst import parse_rst __author__ = "Artur Barseghyan " __copyright__ = "2025-2026 Artur Barseghyan" __license__ = "MIT" __all__ = ( "TestPytestrunMarkParsing", "TestRunPytestStyleCode", ) # ============================================================================ # Test that pytestrun mark is parsed correctly from MD and RST sources # ============================================================================ class TestPytestrunMarkParsing: """Test that the pytestrun mark is captured during parsing.""" def test_md_pytestmark_pytestrun_captured(self): """The pytestrun mark is present after parsing a marked MD block.""" text = textwrap.dedent("""\ ```python name=test_pytestrun_parse def test_ok(): assert True ``` """) snippets = parse_markdown(text) assert len(snippets) == 1 assert PYTESTRUN_MARK in snippets[0].marks def test_md_both_codeblock_and_pytestrun_marks(self): """Both codeblock and pytestrun marks are present simultaneously.""" text = textwrap.dedent("""\ ```python name=test_marks_coexist def test_x(): pass ``` """) snippets = parse_markdown(text) assert CODEBLOCK_MARK in snippets[0].marks assert PYTESTRUN_MARK in snippets[0].marks def test_md_no_pytestrun_mark_by_default(self): """A plain code block does NOT carry the pytestrun mark.""" text = textwrap.dedent("""\ ```python name=test_plain x = 1 ``` """) snippets = parse_markdown(text) assert PYTESTRUN_MARK not in snippets[0].marks def test_rst_pytestmark_pytestrun_captured(self, tmp_path): """The pytestrun mark is present after parsing a marked RST block.""" rst = textwrap.dedent("""\ .. pytestmark: pytestrun .. code-block:: python :name: test_pytestrun_rst def test_ok(): assert True """) snippets = parse_rst(rst, tmp_path) assert len(snippets) == 1 assert PYTESTRUN_MARK in snippets[0].marks def test_rst_no_pytestrun_mark_by_default(self, tmp_path): """A plain RST code block does NOT carry the pytestrun mark.""" rst = textwrap.dedent("""\ .. code-block:: python :name: test_plain_rst x = 1 """) snippets = parse_rst(rst, tmp_path) assert PYTESTRUN_MARK not in snippets[0].marks # ============================================================================ # Test run_pytest_style_code directly # ============================================================================ class TestRunPytestStyleCode: """Unit tests for run_pytest_style_code helper.""" def test_passing_code_does_not_raise(self, tmp_path): """A block with a passing test must not raise.""" code = textwrap.dedent("""\ def test_simple(): assert 1 + 1 == 2 """) run_pytest_style_code( code=code, snippet_name="test_simple", path=str(tmp_path / "dummy.md"), ) def test_failing_code_raises_assertion_error(self, tmp_path): """A block with a failing test must raise AssertionError.""" code = textwrap.dedent("""\ def test_fail(): assert False, "intentional failure" """) with pytest.raises(AssertionError, match="test_fail"): run_pytest_style_code( code=code, snippet_name="test_fail", path=str(tmp_path / "dummy.md"), ) def test_error_message_contains_snippet_name(self, tmp_path): """The AssertionError message should reference the snippet name.""" code = textwrap.dedent("""\ def test_broken(): raise ValueError("boom") """) with pytest.raises(AssertionError) as exc_info: run_pytest_style_code( code=code, snippet_name="test_broken_snippet", path=str(tmp_path / "dummy.md"), ) assert "test_broken_snippet" in str(exc_info.value) def test_class_based_tests_pass(self, tmp_path): """A block containing a Test* class should execute correctly.""" code = textwrap.dedent("""\ class TestMath: def test_addition(self): assert 2 + 2 == 4 def test_subtraction(self): assert 5 - 3 == 2 """) run_pytest_style_code( code=code, snippet_name="test_math_class", path=str(tmp_path / "dummy.md"), ) def test_class_based_tests_fail(self, tmp_path): """A block with a failing Test* class must raise AssertionError.""" code = textwrap.dedent("""\ class TestBroken: def test_bad(self): assert 1 == 2 """) with pytest.raises(AssertionError): run_pytest_style_code( code=code, snippet_name="test_broken_class", path=str(tmp_path / "dummy.md"), ) def test_class_with_fixture_passes(self, tmp_path): """A block using a class-level fixture should pass.""" code = textwrap.dedent("""\ import pytest class TestFixture: @pytest.fixture def greeting(self): return "hello" def test_greeting(self, greeting): assert greeting == "hello" """) run_pytest_style_code( code=code, snippet_name="test_class_fixture", path=str(tmp_path / "dummy.md"), ) def test_parametrize_passes(self, tmp_path): """A block using @pytest.mark.parametrize should pass.""" code = textwrap.dedent("""\ import pytest @pytest.mark.parametrize("n,expected", [(1, 2), (2, 4), (3, 6)]) def test_double(n, expected): assert n * 2 == expected """) run_pytest_style_code( code=code, snippet_name="test_parametrize", path=str(tmp_path / "dummy.md"), ) def test_parametrize_failure_raises(self, tmp_path): """A parametrized block with a bad case must raise AssertionError.""" code = textwrap.dedent("""\ import pytest @pytest.mark.parametrize("n,expected", [(1, 99)]) def test_wrong(n, expected): assert n * 2 == expected """) with pytest.raises(AssertionError): run_pytest_style_code( code=code, snippet_name="test_bad_parametrize", path=str(tmp_path / "dummy.md"), ) def test_setup_teardown_runs(self, tmp_path): """setup_method / teardown_method hooks should execute correctly.""" code = textwrap.dedent("""\ class TestSetup: def setup_method(self): self.value = 42 def test_value(self): assert self.value == 42 def teardown_method(self): self.value = None """) run_pytest_style_code( code=code, snippet_name="test_setup_teardown", path=str(tmp_path / "dummy.md"), ) def test_nested_fixtures_pass(self, tmp_path): """Nested fixture dependencies should be resolved correctly.""" code = textwrap.dedent("""\ import pytest class TestNested: @pytest.fixture def base(self): return 10 @pytest.fixture def derived(self, base): return base * 3 def test_derived(self, derived): assert derived == 30 """) run_pytest_style_code( code=code, snippet_name="test_nested_fixtures", path=str(tmp_path / "dummy.md"), ) def test_multiple_test_functions_all_pass(self, tmp_path): """Multiple top-level test functions in one block should all run.""" code = textwrap.dedent("""\ def test_one(): assert "a" == "a" def test_two(): assert [1, 2, 3][0] == 1 def test_three(): assert {"k": "v"}["k"] == "v" """) run_pytest_style_code( code=code, snippet_name="test_multiple_fns", path=str(tmp_path / "dummy.md"), ) def test_multiple_test_functions_one_fails(self, tmp_path): """If any test function fails, AssertionError must be raised.""" code = textwrap.dedent("""\ def test_good(): assert True def test_bad(): assert False """) with pytest.raises(AssertionError): run_pytest_style_code( code=code, snippet_name="test_multi_one_fails", path=str(tmp_path / "dummy.md"), ) def test_empty_code_raises_no_tests_collected(self, tmp_path): """Empty block (no test functions) should fail with non-zero exit.""" code = "# no tests here\nx = 1\n" with pytest.raises(AssertionError): run_pytest_style_code( code=code, snippet_name="test_empty_block", path=str(tmp_path / "dummy.md"), ) def test_syntax_error_in_code_raises(self, tmp_path): """A block with a syntax error should cause a non-zero pytest exit.""" code = "def broken(:\n pass\n" with pytest.raises(AssertionError): run_pytest_style_code( code=code, snippet_name="test_syntax_err", path=str(tmp_path / "dummy.md"), ) src/pytest_codeblock/tests/tests.md ----------------------------------- src/pytest_codeblock/tests/tests.md # Tests ## test_group_snippets_merges_named ```python name=test_group_snippets_merges_named import pytest from pathlib import Path from pytest_codeblock.collector import CodeSnippet, group_snippets from pytest_codeblock.md import parse_markdown from pytest_codeblock.rst import ( parse_rst, resolve_literalinclude_path, get_literalinclude_content, ) # Two snippets with the same name should be combined sn1 = CodeSnippet(name="foo", code="a=1", line=1, marks=["codeblock"]) sn2 = CodeSnippet(name="foo", code="b=2", line=2, marks=["codeblock", "m"]) combined = group_snippets([sn1, sn2]) assert len(combined) == 1 cs = combined[0] assert cs.name == "foo" # Both code parts should appear assert "a=1" in cs.code assert "b=2" in cs.code # Marks should accumulate assert "m" in cs.marks ``` ---- ## test_group_snippets_different_names ```python name=test_group_snippets_different_names import pytest from pathlib import Path from pytest_codeblock.collector import CodeSnippet, group_snippets from pytest_codeblock.md import parse_markdown from pytest_codeblock.rst import ( parse_rst, resolve_literalinclude_path, get_literalinclude_content, ) # Snippets with different names are not grouped sn1 = CodeSnippet(name="foo", code="x=1", line=1) sn2 = CodeSnippet(name="bar", code="y=2", line=2) combined = group_snippets([sn1, sn2]) assert len(combined) == 2 assert combined[0].name.startswith("foo") assert combined[1].name.startswith("bar") ``` ---- ## test_parse_markdown_simple ```python name=test_parse_markdown_simple import pytest from pathlib import Path from pytest_codeblock.collector import CodeSnippet, group_snippets from pytest_codeblock.md import parse_markdown from pytest_codeblock.rst import ( parse_rst, resolve_literalinclude_path, get_literalinclude_content, ) text = markdown_simple snippets = parse_markdown(text) assert len(snippets) == 1 sn = snippets[0] assert sn.name == "test_example" assert "x=1" in sn.code ``` ---- ## markdown_with_pytest_mark ```python name=test_parse_markdown_with_pytestmark import pytest from pathlib import Path from pytest_codeblock.collector import CodeSnippet, group_snippets from pytest_codeblock.md import parse_markdown from pytest_codeblock.rst import ( parse_rst, resolve_literalinclude_path, get_literalinclude_content, ) text = markdown_with_pytest_mark snippets = parse_markdown(text) assert len(snippets) == 1 sn = snippets[0] # Should include both default and django_db marks assert "django_db" in sn.marks assert "codeblock" in sn.marks ``` ---- ## test_pytest_fixtures ```python name=test_pytest_fixtures_1 d = tmp_path / "sub" d.mkdir() # Create the directory assert d.is_dir() # Verify it was created and is a directory assert isinstance(http_request.GET, dict) ``` ---- ```python name=test_pytest_fixtures_2 d = tmp_path / "sub" d.mkdir() # Create the directory assert d.is_dir() # Verify it was created and is a directory assert isinstance(http_request.GET, dict) ``` ---- ```python name=test_pytest_fixtures_3 d = tmp_path / "sub" d.mkdir() # Create the directory assert d.is_dir() # Verify it was created and is a directory assert isinstance(http_request.GET, dict) ``` ---- ## test_async_example ```python name=test_async_example import asyncio result = await asyncio.sleep(0.1, result=42) assert result == 42 ``` ---- ## test_group_old_syntax ```python name=test_group_old_syntax text_1 = "Hey" ``` Something in between ```python name=test_group_old_syntax assert text_1 print(text_1) ``` ---- ## test_group_new_syntax ```python name=test_group_new_syntax text_2 = "Jude" ``` Something in between ```python name=test_group_new_syntax_part_2 assert text_2 print(text_2) ``` ---- ## test_pytestrun_marker ```python name=test_pytestrun_marker import pytest class TestSystemInfo: @pytest.fixture def system_name(self): return "Linux" @pytest.fixture def version_number(self): return 5 def test_combined_info(self, system_name, version_number): info = f"{system_name} v{version_number}" assert info == "Linux v5" def test_name_only(self, system_name): assert system_name.isalpha() ``` ---- ## test_pytestrun_marker_and_conftest_fixtures ```python name=test_pytestrun_marker_and_conftest_fixtures import pytest class TestSystemInfo: def test_request(self, http_request): assert isinstance(http_request.GET, dict) ``` ---- ## test_pytestrun_with_setup_teardown ```python name=test_pytestrun_with_setup_teardown import pytest class TestWithSetup: def setup_method(self): self.value = 10 def test_value_set(self): assert self.value == 10 def test_value_modified(self): self.value += 5 assert self.value == 15 ``` ---- ## test_pytestrun_with_parametrize ```python name=test_pytestrun_with_parametrize import pytest class TestParametrized: @pytest.mark.parametrize("input,expected", [(1, 2), (3, 4)]) def test_increment(self, input, expected): assert input + 1 == expected ``` ---- ## test_pytestrun_nested_fixtures ```python name=test_pytestrun_nested_fixtures import pytest class TestNestedFixtures: @pytest.fixture def base_value(self): return 100 @pytest.fixture def derived_value(self, base_value): return base_value * 2 def test_derived(self, derived_value): assert derived_value == 200 ``` ---- ## test_pytestrun_with_conftest_and_class_fixtures ```python name=test_pytestrun_with_conftest_and_class_fixtures import pytest class TestMixedFixtures: @pytest.fixture def local_data(self): return {"key": "value"} def test_both_fixtures(self, http_request, local_data): assert isinstance(http_request.GET, dict) assert local_data["key"] == "value" ``` ---- ## test_pytestrun_multiple_test_methods ```python name=test_pytestrun_multiple_test_methods import pytest class TestMultipleMethods: @pytest.fixture def shared_list(self): return [1, 2, 3] def test_length(self, shared_list): assert len(shared_list) == 3 def test_first_element(self, shared_list): assert shared_list[0] == 1 def test_sum(self, shared_list): assert sum(shared_list) == 6 ``` ---- ## test_pytestrun_multiple_test_methods_multiple_markers ```python name=test_pytestrun_multiple_test_methods_multiple_markers import pytest class TestMultipleMethodsMultipleMarkers: @pytest.fixture def letter_a(self): return "a" def test_class_level_fixture(self, letter_a): assert letter_a == "a" def test_pytest_built_in_fixture(self, tmp_path): d = tmp_path / "sub" d.mkdir() # Create the directory assert d.is_dir() # Verify it was created and is a directory def test_pytest_user_defined_fixture(self, http_request): assert isinstance(http_request.GET, dict) ``` ---- ## test_updated_grouping ```python name=test_updated_grouping names = ["Jude"] assert len(names) == 1 print(names) ``` Something in between ```python name=test_updated_grouping_part_2 assert names print(names) names.append("Lora") assert len(names) == 2 ``` Something in between ```python name=test_updated_grouping_part_3 assert names print(names) names.append("Alice") assert len(names) == 3 ``` ---- ## test_updated_grouping_pytestrun_marker ```python name=test_updated_grouping_pytestrun_marker import pytest class TestSample: @pytest.fixture def system_name(self): return "Linux" @pytest.fixture def version_number(self): return 5 def test_combined_info(self, system_name, version_number): info = f"{system_name} v{version_number}" assert info == "Linux v5" print(info) ``` Some text in between ```python name=test_updated_grouping_pytestrun_marker_part_2 class TestSample: @pytest.fixture def system_name(self): return "macOS" @pytest.fixture def version_number(self): return 17 def test_combined_info(self, system_name, version_number): info = f"{system_name} v{version_number}" assert info == "macOS v17" print(info) ``` src/pytest_codeblock/tests/tests.rst ------------------------------------ src/pytest_codeblock/tests/tests.rst Tests ===== test_group_snippets_merges_named -------------------------------- .. code-block:: python :name: test_group_snippets_merges_named import pytest from pathlib import Path from pytest_codeblock.collector import CodeSnippet, group_snippets from pytest_codeblock.md import parse_markdown from pytest_codeblock.rst import ( parse_rst, resolve_literalinclude_path, get_literalinclude_content, ) # Two snippets with the same name should be combined sn1 = CodeSnippet(name="foo", code="a=1", line=1, marks=["codeblock"]) sn2 = CodeSnippet(name="foo", code="b=2", line=2, marks=["codeblock", "m"]) combined = group_snippets([sn1, sn2]) assert len(combined) == 1 cs = combined[0] assert cs.name == "foo" # Both code parts should appear assert "a=1" in cs.code assert "b=2" in cs.code # Marks should accumulate assert "m" in cs.marks ---- test_group_snippets_different_names ----------------------------------- .. code-block:: python :name: test_group_snippets_different_names import pytest from pathlib import Path from pytest_codeblock.collector import CodeSnippet, group_snippets from pytest_codeblock.md import parse_markdown from pytest_codeblock.rst import ( parse_rst, resolve_literalinclude_path, get_literalinclude_content, ) # Snippets with different names are not grouped sn1 = CodeSnippet(name="foo", code="x=1", line=1) sn2 = CodeSnippet(name="bar", code="y=2", line=2) combined = group_snippets([sn1, sn2]) assert len(combined) == 2 assert combined[0].name.startswith("foo") assert combined[1].name.startswith("bar") ---- test_parse_markdown_simple -------------------------- .. code-block:: python :name: test_parse_markdown_simple import pytest from pathlib import Path from pytest_codeblock.collector import CodeSnippet, group_snippets from pytest_codeblock.md import parse_markdown from pytest_codeblock.rst import ( parse_rst, resolve_literalinclude_path, get_literalinclude_content, ) text = """ ```python name=test_example x=1 assert x==1 ```""" snippets = parse_markdown(text) assert len(snippets) == 1 sn = snippets[0] assert sn.name == "test_example" assert "x=1" in sn.code ---- test_parse_markdown_with_pytestmark ----------------------------------- .. code-block:: python :name: test_parse_markdown_with_pytestmark import pytest from pathlib import Path from pytest_codeblock.collector import CodeSnippet, group_snippets from pytest_codeblock.md import parse_markdown from pytest_codeblock.rst import ( parse_rst, resolve_literalinclude_path, get_literalinclude_content, ) text = """ ```python name=test_db from django.db import models ```""" snippets = parse_markdown(text) assert len(snippets) == 1 sn = snippets[0] # Should include both default and django_db marks assert "django_db" in sn.marks assert "codeblock" in sn.marks ---- test_pytest_fixtures -------------------- .. pytestfixture: tmp_path .. pytestfixture: http_request .. code-block:: python :name: test_pytest_fixtures_1 d = tmp_path / "sub" d.mkdir() # Create the directory assert d.is_dir() # Verify it was created and is a directory assert isinstance(http_request.GET, dict) ---- .. pytestfixture: tmp_path .. pytestfixture: http_request .. code-block:: python :name: test_pytest_fixtures_2 d = tmp_path / "sub" d.mkdir() # Create the directory assert d.is_dir() # Verify it was created and is a directory assert isinstance(http_request.GET, dict) ---- .. pytestfixture: tmp_path .. pytestfixture: http_request .. code-block:: python :name: test_pytest_fixtures_3 d = tmp_path / "sub" d.mkdir() # Create the directory assert d.is_dir() # Verify it was created and is a directory assert isinstance(http_request.GET, dict) ---- test_async_example ------------------ .. code-block:: python :name: test_async_example import asyncio result = await asyncio.sleep(0.1, result=42) assert result == 42 ---- test_group_snippets ------------------- .. code-block:: python :name: test_group_snippets text_2 = "Jude" Something in between .. continue: test_group_snippets .. code-block:: python :name: test_group_snippets_part_2 assert text_2 print(text_2) ---- test_pytestrun_marker --------------------- .. pytestmark: pytestrun .. code-block:: python :name: test_pytestrun_marker import pytest class TestSystemInfo: @pytest.fixture def system_name(self): return "Linux" @pytest.fixture def version_number(self): return 5 def test_combined_info(self, system_name, version_number): info = f"{system_name} v{version_number}" assert info == "Linux v5" def test_name_only(self, system_name): assert system_name.isalpha() ---- test_pytestrun_marker_and_conftest_fixtures ------------------------------------------- .. pytestmark: pytestrun .. code-block:: python :name: test_pytestrun_marker_and_conftest_fixtures import pytest class TestSystemInfo: def test_request(self, http_request): assert isinstance(http_request.GET, dict) ---- test_pytestrun_with_setup_teardown ---------------------------------- .. pytestmark: pytestrun .. code-block:: python :name: test_pytestrun_with_setup_teardown import pytest class TestWithSetup: def setup_method(self): self.value = 10 def test_value_set(self): assert self.value == 10 def test_value_modified(self): self.value += 5 assert self.value == 15 ---- test_pytestrun_with_parametrize ------------------------------- .. pytestmark: pytestrun .. code-block:: python :name: test_pytestrun_with_parametrize import pytest class TestParametrized: @pytest.mark.parametrize("input,expected", [(1, 2), (3, 4)]) def test_increment(self, input, expected): assert input + 1 == expected ---- test_pytestrun_nested_fixtures ------------------------------ .. pytestmark: pytestrun .. code-block:: python :name: test_pytestrun_nested_fixtures import pytest class TestNestedFixtures: @pytest.fixture def base_value(self): return 100 @pytest.fixture def derived_value(self, base_value): return base_value * 2 def test_derived(self, derived_value): assert derived_value == 200 ---- test_pytestrun_with_conftest_and_class_fixtures ----------------------------------------------- .. pytestmark: pytestrun .. code-block:: python :name: test_pytestrun_with_conftest_and_class_fixtures import pytest class TestMixedFixtures: @pytest.fixture def local_data(self): return {"key": "value"} def test_both_fixtures(self, http_request, local_data): assert isinstance(http_request.GET, dict) assert local_data["key"] == "value" ---- test_pytestrun_multiple_test_methods ------------------------------------ .. pytestmark: pytestrun .. code-block:: python :name: test_pytestrun_multiple_test_methods import pytest class TestMultipleMethods: @pytest.fixture def shared_list(self): return [1, 2, 3] def test_length(self, shared_list): assert len(shared_list) == 3 def test_first_element(self, shared_list): assert shared_list[0] == 1 def test_sum(self, shared_list): assert sum(shared_list) == 6 ---- test_pytestrun_multiple_test_methods_multiple_markers ----------------------------------------------------- .. pytestmark: pytestrun .. code-block:: python :name: test_pytestrun_multiple_test_methods_multiple_markers import pytest class TestMultipleMethodsMultipleMarkers: @pytest.fixture def letter_a(self): return "a" def test_class_level_fixture(self, letter_a): assert letter_a == "a" def test_pytest_built_in_fixture(self, tmp_path): d = tmp_path / "sub" d.mkdir() # Create the directory assert d.is_dir() # Verify it was created and is a directory def test_pytest_user_defined_fixture(self, http_request): assert isinstance(http_request.GET, dict) ---- test_updated_grouping --------------------- .. code-block:: python :name: test_updated_grouping names = ["Jude"] assert len(names) == 1 print(names) Something in between .. continue: test_updated_grouping .. code-block:: python :name: test_updated_grouping_part_2 assert names print(names) names.append("Lora") assert len(names) == 2 Something in between .. continue: test_updated_grouping .. code-block:: python :name: test_updated_grouping_part_3 assert names print(names) names.append("Alice") assert len(names) == 3 ---- test_updated_grouping_pytestrun_marker -------------------------------------- .. pytestmark: pytestrun .. code-block:: python :name: test_updated_grouping_pytestrun_marker import pytest class TestSample: @pytest.fixture def system_name(self): return "Linux" @pytest.fixture def version_number(self): return 5 def test_combined_info(self, system_name, version_number): info = f"{system_name} v{version_number}" assert info == "Linux v5" print(info) Some text in between .. continue: test_updated_grouping_pytestrun_marker .. pytestmark: pytestrun .. code-block:: python :name: test_updated_grouping_pytestrun_marker_part_2 class TestSample: @pytest.fixture def system_name(self): return "macOS" @pytest.fixture def version_number(self): return 17 def test_combined_info(self, system_name, version_number): info = f"{system_name} v{version_number}" assert info == "macOS v17" print(info)