Initial release

.github/workflows/main.yml

@@ -0,0 +1,42 @@
+---
+name: main
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  style:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.9'
+
+      - name: Install tox
+        run: pip install tox
+
+      - name: Run style check
+        run: tox -e style
+
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ['3.9', '3.10', '3.11', '3.12']
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install tox
+        run: pip install tox
+
+      - name: Run tests
+        run: tox -e py

.gitignore

@@ -0,0 +1,77 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+env/
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*,cover
+.hypothesis/
+.pytest_cache
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+
+# Flask instance folder
+instance/
+
+# Sphinx documentation
+docs/_build/
+
+# MkDocs documentation
+/site/
+
+# PyBuilder
+target/
+
+# IPython Notebook
+.ipynb_checkpoints
+
+# pyenv
+.python-version
+
+
+# ruff
+.ruff_cache

.yamlfix.toml

@@ -0,0 +1,17 @@
+allow_duplicate_keys = false
+comments_min_spaces_from_content = 2
+comments_require_starting_space = true
+whitelines = 1
+comments_whitelines = 1
+section_whitelines = 1
+explicit_start = true
+sequence_style = "flow_style"
+indent_mapping = 2
+ident_offset = 2
+indent_sequence = 4
+line_length = 88
+none_representation = "null"
+quote_basic_values = false
+quote_keys_and_basic_values = false
+quote_representation = "'"
+preserve_quotes = false

README.md

@@ -1,2 +1,100 @@
 # pytest-nm-releng
-pytest plugin used by Release Engineering
+
+A pytest plugin offering various functionality for Neural Magic’s Release Engineering team.
+
+## Installation
+
+The Python package can be installed directly from this git repository from either a branch or tag:
+
+```shell
+# recommended: use a version tag (e.g., v0.1.0)
+pip install https://github.com/neuralmagic/pytest-nm-releng/archive/v0.1.0.tar.gz
+
+# alternative: install based on a branch (e.g., main)
+pip install https://github.com/neuralmagic/pytest-nm-releng/archive/main.tar.gz
+```
+
+## Features
+
+### Dynamically-named JUnit report files
+
+`pytest-nm-releng` can automatically generate unique, dynamically-named JUnit report files with an optional prefix. The report file is generated when the test run begins using Python’s `datetime.timestamp()` method (UTC).
+
+> [!NOTE]
+> This works by appending the `--junit-xml` flag after the command is run, meaning it will override any previously-specified instances of this flag.
+
+To enable this behavior, define the environment variable `NMRE_JUNIT_BASE` with a value to the path where the test files should be stored. This can be absolute or relative.
+
+The following examples will both write JUnit report files in a folder named "test-results" in the current working directory.
+
+```shell
+# example: prefixing a command
+NMRE_JUNIT_BASE=test-results pytest [...]
+
+# example: export the environment variable (useful if pytest is not being
+# invoked directly)
+export NMRE_JUNIT_BASE=test-results
+pytest [...]
+
+# after either example, a file named something like
+# `test-results/1735941024.348248.xml` will be created
+```
+
+Optionally, you can define `NMRE_JUNIT_PREFIX` with a value to be prefixed onto the file name. Note that no separator is used so you may want to include one.
+
+```shell
+export NMRE_JUNIT_BASE=test-results
+export NMRE_JUNIT_PREFIX="report-"
+pytest [...]
+
+# after either example, a file named something like
+# `test-results/report-1735941218.338192.xml` will be created
+```
+
+### Code coverage
+
+`pytest-nm-releng` can automatically add some code coverage flags as well (requires [pytest-cov]).
+
+To enable this behavior, define the `NMRE_COV_NAME` environment variable with a value of the project’s *_module_* name (e.g., the name that is used to import it within Python code).
+
+```shell
+# example: used with `nm-vllm-ent`, which is imported as `vllm`
+NMRE_COV_NAME=vllm pytest [...]
+
+# this will result in the following flags being appended:
+# --cov=vllm --cov-append --cov-report=html:coverage-html --cov-report=json:coverage.json
+```
+
+## Contributing
+
+To contribute, follow these general steps:
+
+1. Fork the repository
+1. Create a new branch
+1. Make your changes
+1. Install `tox`
+   ```shell
+   # example: using pipx
+   pipx install tox
+   # example: using uv
+   uv tool install tox --with tox-uv
+   ```
+1. Run quality checks and tests
+   ```shell
+   # apply available automatic style/formatting fixes
+   tox -e format
+   # check style/formatting
+   tox -e style
+   # run tests
+   tox -e py
+   ```
+1. Submit a pull request with your changes
+
+## Acknowledgements
+
+This pytest plugin was generated with [Cookiecutter] along with [@hackebrot]'s [cookiecutter-pytest-plugin] template.
+
+[@hackebrot]: https://github.com/hackebrot
+[cookiecutter]: https://github.com/audreyr/cookiecutter
+[cookiecutter-pytest-plugin]: https://github.com/pytest-dev/cookiecutter-pytest-plugin
+[pytest-cov]: https://github.com/pytest-dev/pytest-cov

pyproject.toml

@@ -0,0 +1,40 @@
+[build-system]
+requires = ["setuptools>=61.0.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "pytest-nm-releng"
+description = "A pytest plugin providing custom functionality for the Neural Magic release engineering team."
+version = "0.1.0"
+readme = "README.md"
+requires-python = ">=3.9"
+authors = [{ name = "Domenic Barbuzzi", email = "domenic@neuralmagic.com" }]
+maintainers = [{ name = "Domenic Barbuzzi", email = "domenic@neuralmagic.com" }]
+license = { file = "LICENSE" }
+classifiers = [
+    "Framework :: Pytest",
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Topic :: Software Development :: Testing",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3 :: Only",
+    "Programming Language :: Python :: Implementation :: CPython",
+    "License :: OSI Approved :: Apache Software License",
+]
+dependencies = [
+    "pytest>=8",
+]
+
+[project.urls]
+Repository = "https://github.com/neuralmagic/pytest-nm-releng"
+
+[project.entry-points.pytest11]
+nm-releng = "pytest_nm_releng.plugin"
+
+[tool.ruff.lint]
+extend-select = ["I"]

src/pytest_nm_releng/__init__.py

@@ -0,0 +1,13 @@
+# Copyright (c) 2025 - present / Neuralmagic, Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.

src/pytest_nm_releng/plugin.py

@@ -0,0 +1,53 @@
+# Copyright (c) 2025 - present / Neuralmagic, Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+import os
+from datetime import datetime, timezone
+from pathlib import Path
+
+
+def get_utc_timestamp() -> float:
+    return datetime.now(timezone.utc).timestamp()
+
+
+def generate_junit_flags() -> list[str]:
+    if not (junitxml_base_dir := os.getenv("NMRE_JUNIT_BASE")):
+        return []
+
+    junitxml_file = Path(junitxml_base_dir) / f"{get_utc_timestamp()}.xml"
+
+    if prefix := os.getenv("NMRE_JUNIT_PREFIX"):
+        junitxml_file = junitxml_file.with_name(f"{prefix}{junitxml_file.name}")
+
+    return [f"--junit-xml={junitxml_file.as_posix()}"]
+
+
+def generate_coverage_flags() -> list[str]:
+    if not (cc_package_name := os.getenv("NMRE_COV_NAME")):
+        return []
+
+    return [
+        f"--cov={cc_package_name}",
+        "--cov-append",
+        "--cov-report=html:coverage-html",
+        "--cov-report=json:coverage.json",
+    ]
+
+
+def pytest_load_initial_conftests(early_config, args: list[str], parser):
+    new_args: list[str] = []
+    new_args.extend(generate_junit_flags())
+    new_args.extend(generate_coverage_flags())
+    args[:] = [*args, *new_args]

tests/__init__.py

@@ -0,0 +1,13 @@
+# Copyright (c) 2025 - present / Neuralmagic, Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.

tests/conftest.py

@@ -0,0 +1,16 @@
+# Copyright (c) 2025 - present / Neuralmagic, Inc. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+pytest_plugins = "pytester"