Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify Manifester class to work with MockStub #4

Merged
merged 26 commits into from
Dec 15, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
14c039a
Modify Manifester class to work with MockStub
synkd Jun 1, 2022
0996b26
Rebase PR and address merge conflicts
synkd Nov 8, 2023
36ba344
Get initial tests passing
synkd Nov 14, 2023
ac11f42
Add test for get_manifest(), add docstrings
synkd Nov 15, 2023
c4880fa
Add test for delete_subscription_allocation
synkd Nov 16, 2023
d7c4861
Add negative test for simple retry's timeout
synkd Nov 17, 2023
6f34d36
Add test, modify mock method returns
synkd Nov 21, 2023
809bd13
Remove extraneous comments
synkd Nov 21, 2023
3202681
Revert max_timeout change
synkd Nov 21, 2023
1149cb7
Correct logic error in sat version evaluation
synkd Nov 21, 2023
f992b47
Use more realistic data in version check
synkd Nov 21, 2023
6a43c1c
Fix failing test for simple retry timeout
synkd Nov 21, 2023
f2a897e
Add test for reading manifest data from dictionary
synkd Nov 22, 2023
fb44056
Add test of subscription pool retrieval with offset
synkd Nov 28, 2023
7e45b04
Use realistic data to simplify mock handling
synkd Nov 28, 2023
4fe55e0
Add test for correct subs in allocation
synkd Nov 28, 2023
341cada
Tweak test_correct_subs_added_to_allocation
synkd Nov 28, 2023
83156c2
Add test for invalid sat_version
synkd Nov 29, 2023
f53bfaa
Add pre-commit hooks and run pre-commit
synkd Nov 29, 2023
55f3cda
Switch to ruff for linting/formatting
synkd Dec 4, 2023
6e1a3d2
Modify workflow to workaround CodeQL failure
synkd Dec 4, 2023
79a4b4e
Add pytest dep and correct typo in test manifest_data
synkd Dec 4, 2023
2af7760
Use manifest_data dict for all tests
synkd Dec 4, 2023
2bc2572
Modify checkout action to pull latest commit from PR
synkd Dec 5, 2023
39483fa
Address reviewer feedback; delete status code in one additional spot
synkd Dec 12, 2023
96e2640
Address settings issue and test flakiness
synkd Dec 12, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ on:
branches: [master]
pull_request:
types: [opened, synchronize, reopened]
paths-ignore:
- "*.md"
- "*.example"
- ".gitignore"

jobs:
analyze:
Expand All @@ -14,22 +18,22 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ["3.8", "3.9", "3.10"]
python-version: ["3.10", "3.11", "3.12"]

steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Initialize CodeQL
uses: github/codeql-action/init@v1
uses: github/codeql-action/init@v2
synkd marked this conversation as resolved.
Show resolved Hide resolved
with:
languages: ${{ matrix.language }}

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1
uses: github/codeql-action/analyze@v2

- name: Setup Python
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

Expand All @@ -39,4 +43,4 @@ jobs:
pip install -U .[test]
cp manifester_settings.yaml.example manifester_settings.yaml
manifester --help
# pytest -v tests/ --ignore tests/functional
pytest -v tests/
8 changes: 4 additions & 4 deletions .github/workflows/python-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,12 @@ jobs:
strategy:
matrix:
# build/push in lowest support python version
python-version: [ 3.8 ]
python-version: [ 3.10 ]

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4

- uses: actions/setup-python@v2
- uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

Expand All @@ -29,7 +29,7 @@ jobs:
python -m twine check dist/*

- name: Build and publish
uses: pypa/gh-action-pypi-publish@release/v1
uses: pypa/gh-action-pypi-publish@v1.8.11
with:
user: __token__
password: ${{ secrets.PYPI_TOKEN }}
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/update_manifester_image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,16 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v4

- name: Set up QEMU
uses: docker/setup-qemu-action@v1
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v3

- name: Login to Quay Container Registry
uses: docker/login-action@v1
uses: docker/login-action@v3
with:
registry: ${{ secrets.QUAY_SERVER }}
username: ${{ secrets.QUAY_USERNAME }}
Expand Down
9 changes: 3 additions & 6 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,13 @@ repos:
- id: trailing-whitespace
- id: check-yaml
- id: debug-statements
- repo: https://github.com/psf/black
rev: 22.10.0
hooks:
- id: black
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.0.277
rev: v0.1.6
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
- id: ruff-format
- repo: https://github.com/gitleaks/gitleaks
rev: v8.18.0
hooks:
- id: gitleaks
- id: gitleaks
3 changes: 3 additions & 0 deletions manifester/commands.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
"""Defines the CLI commands for Manifester."""
import click

from manifester import Manifester
Expand All @@ -6,6 +7,7 @@
# To do: add a command for returning subscription pools
@click.group
def cli():
"""Command-line interface for manifester."""
pass


Expand All @@ -17,6 +19,7 @@ def cli():
)
@click.option("--allocation_name", type=str, help="Name of upstream subscription allocation")
def get_manifest(manifest_category, allocation_name):
"""Return a subscription manifester based on the settings for the provided manifest_category."""
manifester = Manifester(manifest_category, allocation_name)
manifester.create_subscription_allocation()
for sub in manifester.subscription_data:
Expand Down
28 changes: 17 additions & 11 deletions manifester/helpers.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
"""Defines helper functions used by Manifester."""
from collections import UserDict
import random
import time
Expand All @@ -6,8 +7,7 @@


def simple_retry(cmd, cmd_args=None, cmd_kwargs=None, max_timeout=240, _cur_timeout=1):
"""Re(Try) a function given its args and kwargs up until a max timeout"""

"""Re(Try) a function given its args and kwargs up until a max timeout."""
cmd_args = cmd_args if cmd_args else []
cmd_kwargs = cmd_kwargs if cmd_kwargs else {}
# If additional debug information is needed, the following log entry can be modified to
Expand All @@ -27,41 +27,40 @@ def simple_retry(cmd, cmd_args=None, cmd_kwargs=None, max_timeout=240, _cur_time


def process_sat_version(sat_version, valid_sat_versions):
"""Ensure that the sat_version parameter is properly formatted for the RHSM API when creating
a subscription allocation with the 'POST allocations' endpoint"""
"""Ensure that the sat_version parameter is properly formatted for the RHSM API."""
expected_length = 8
synkd marked this conversation as resolved.
Show resolved Hide resolved
if sat_version not in valid_sat_versions:
# The valid values for the sat_version parameter when creating a subscription allocation
# are all 8 characters or less (e.g. 'sat-6.11'). Some data sources may include a Z-stream
# version (e.g. 'sat-6.11.0') when retrieving this value from settings. The conditional
# below assumes that, if the length of sat_version is greated than 8 characters, it includes
# a Z-stream version that should be removed.
if len(sat_version) > 8:
sat_version = sat_version.split('.')
if len(sat_version) > expected_length:
sat_version = sat_version.split(".")
sat_version = sat_version[0:2]
sat_version = ".".join(sat_version)
# If sat_version is still not valid, default to the latest valid version.
if sat_version not in valid_sat_versions:
valid_sat_versions.sort(
key=lambda i: int(i.split('-')[-1].split('.')[-1]), reverse=True
key=lambda i: int(i.split("-")[-1].split(".")[-1]), reverse=True
)
return valid_sat_versions[0]
return sat_version


def fake_http_response_code(good_codes=None, bad_codes=None, fail_rate=20):
"""Return an HTTP response code randomly selected from sets of good and bad codes"""

"""Return an HTTP response code randomly selected from sets of good and bad codes."""
if random.random() > (fail_rate / 100):
return random.choice(good_codes)
else:
return random.choice(bad_codes)


class MockStub(UserDict):
"""Test helper class. Allows for both arbitrary mocking and stubbing"""
"""Test helper class. Allows for both arbitrary mocking and stubbing."""

def __init__(self, in_dict=None):
"""Initialize the class and all nested dictionaries"""
"""Initialize the class and all nested dictionaries."""
if in_dict is None:
in_dict = {}
for key, value in in_dict.items():
Expand All @@ -78,9 +77,15 @@ def __init__(self, in_dict=None):
super().__init__(in_dict)

def __getattr__(self, name):
"""Fallback to returning self if attribute doesn't exist."""
return self

def __getitem__(self, key):
"""Get an item from the dictionary-like object.

If the key is a string, this method will attempt to get an attribute with that name.
If the key is not found, this method will return the object itself.
"""
if isinstance(key, str):
item = getattr(self, key, self)
try:
Expand All @@ -90,4 +95,5 @@ def __getitem__(self, key):
return item

def __call__(self, *args, **kwargs):
"""Allow MockStub to be used like a function."""
return self
13 changes: 5 additions & 8 deletions manifester/logger.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
"""Defines manifester's internal logging."""
import logging
from pathlib import Path

Expand All @@ -7,26 +8,22 @@


def setup_logzero(level="info", path="logs/manifester.log", silent=True):
"""Call logzero setup with the given settings."""
Path(path).parent.mkdir(parents=True, exist_ok=True)
log_fmt = "%(color)s[%(levelname)s %(asctime)s]%(end_color)s %(message)s"
debug_fmt = (
"%(color)s[%(levelname)1.1s %(asctime)s %(module)s:%(lineno)d]"
"%(end_color)s %(message)s"
"%(color)s[%(levelname)1.1s %(asctime)s %(module)s:%(lineno)d]%(end_color)s %(message)s"
)
log_level = getattr(logging, level.upper(), logging.INFO)
# formatter for terminal
formatter = logzero.LogFormatter(
fmt=debug_fmt if log_level is logging.DEBUG else log_fmt
)
formatter = logzero.LogFormatter(fmt=debug_fmt if log_level is logging.DEBUG else log_fmt)
logzero.setup_default_logger(formatter=formatter, disableStderrLogger=silent)
logzero.loglevel(log_level)
# formatter for file
formatter = logzero.LogFormatter(
fmt=debug_fmt if log_level is logging.DEBUG else log_fmt, color=False
)
logzero.logfile(
path, loglevel=log_level, maxBytes=1e9, backupCount=3, formatter=formatter
)
logzero.logfile(path, loglevel=log_level, maxBytes=1e9, backupCount=3, formatter=formatter)


setup_logzero(level=settings.get("log_level", "info"))
Loading
Loading