pytest-report-stream - Live stream test reports

Authors:	Christos Liontos
Version:	0.1.0
Date:	2023-09-22
Download:	https://pypi.python.org/pypi/pytest-report-stream#downloads
Code:	https://github.com/kolitiri/pytest-report-stream

Welcome to pytest-report-stream!

pytest-report-stream is a pytest plugin which allows to stream test reports at runtime.

It is a simple plugin the leverages pytest's build-in hook pytest_runtest_makereport to intercept test execution and publish the status of the test run.

The plugin produces report events at specific moments: - One report in the beginning of the test run - One report in the end of each test - One report in the end of the test run

The report structure looks like the dictionary below:

{
    "test_run_tag": "My first test framework",
    "test_run_id": "5e080accaee748dc80619ee99245124e",
    "timestamps": {
        "started": datetime.datetime(2023, 10, 22, 17, 4, 43, 161646),
        "duration": 0.002042,
        "finished": datetime.datetime(2023, 10, 22, 17, 4, 43, 163688),
    },
    "summary": {"passed": 1, "failed": 1, "status": "in-progress"},
    "results": {
        "tests/test_module1": {
            "test_func_1": {
                "name": "test_func_1",
                "description": "Docstrings of function 1",
                "status": "passed",
                "error": "None",
            },
            "test_func_2": {
                "name": "test_func_2",
                "description": "Docstrings of function 2",
                "status": "failed",
                "error": "None",
            }
        }
    },
}

Use cases

The pytest-report-stream plugin can be particularly useful while running large and long lasting integration testing suites using pytest.

The plugin can be used to stream live report events to a remote service, allowing to monitor the progress and the status of the tests.

For example, Jenkins and other CI/CD tools are great, but there might be a requirement to aggregate test results from multiple builds.

Installation

Install the plugin as below.

pip install pytest-report-stream

Usage

The plugin is available after installation and can be enabled using the --stream-reports flag.

pytest --stream-reports

The plugin is using a report_client: ReportClient instance to generate the reports.

A default report_client is used to log the reports in STDOUT. (Note that in order to view the reports in STDOUT you will need to run pytest with the flag -s)

The default client can be overriden in your own pytest_configure with an implementation of the abstract pytest_report_stream::ReportClient class.

Synchronous test using the default STDOUT report client

# content of tests/test_my_module.py
def test_sync():
    pass

Aynchronous test using the default STDOUT report client

# content of tests/test_my_module.py
import pytest

@pytest.mark.asyncio
async def test_async():
    pass

Aynchronous test using a custom report client

# content of tests/conftest.py
import pytest
from pytest_report_stream import ReportClient, ReportStreamPlugin


class myCustomReportClient(ReportClient):
    async def publish_report(self, report_msg: dict) -> None:
        print('Some log comming from my custom report client')


def pytest_configure(config):
    if config.option.stream_reports:
        config._stream_reports = ReportStreamPlugin(
            report_client=myCustomReportClient()
        )
        config.pluginmanager.register(config._stream_reports)

# content of tests/test_my_module.py
import pytest

@pytest.mark.asyncio
async def test_async():
    pass

You can implement the publish_report function and do pretty much anything, such as publishing the events to a message broker.

Requirements

• pytest>=7.0.0
• pytest-asyncio

Contributing

Contributions are very welcome.

Tests can be run with tox, please ensure the coverage at least stays the same before you submit a pull request.

tox

License

Distributed under the terms of the MIT license, "pytest-report-stream" is free and open source software

Issues

If you encounter any problems, please file an issue along with a detailed description.