Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'make test' fails on Linux #192

Open
stucka opened this issue Sep 14, 2023 · 1 comment
Open

'make test' fails on Linux #192

stucka opened this issue Sep 14, 2023 · 1 comment

Comments

@stucka
Copy link
Contributor

stucka commented Sep 14, 2023

Tests fail in Linux, even when run from within pipenv, at what look like some pretty basic levels.

pipenv run make test
  __________
 |BIG🌲LOCAL|
 |&&& ======|
 |=== ======|  This is a Big Local News automation
 |=== == %%%|
 |[_] ======|         🤖 Running tests 🤖
 |=== ===!##|
 |__________|

=================================================================== test session starts ====================================================================
platform linux -- Python 3.9.17, pytest-7.4.0, pluggy-1.2.0
rootdir: /home/stucka/Desktop/data/warn-transformer
plugins: cov-4.1.0, vcr-1.0.2
collected 3 items                                                                                                                                          

tests/test_consolidate.py .                                                                                                                          [ 33%]
tests/test_download.py E                                                                                                                             [ 66%]
tests/test_integrate.py F                                                                                                                            [100%]

========================================================================== ERRORS ==========================================================================
_____________________________________________________________ ERROR at setup of test_download ______________________________________________________________

request = <SubRequest '_vcr_marker' for <Function test_download>>

    @pytest.fixture(autouse=True)
    def _vcr_marker(request):
        marker = request.node.get_closest_marker('vcr')
        if marker:
>           request.getfixturevalue('vcr_cassette')

../../../.local/share/virtualenvs/warn-transformer-1C5zOao7/lib/python3.9/site-packages/pytest_vcr.py:46: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../.local/share/virtualenvs/warn-transformer-1C5zOao7/lib/python3.9/site-packages/pytest_vcr.py:87: in vcr_cassette
    with vcr.use_cassette(vcr_cassette_name, **kwargs) as cassette:
../../../.local/share/virtualenvs/warn-transformer-1C5zOao7/lib/python3.9/site-packages/vcr/cassette.py:83: in __enter__
    self.__cassette = self.cls.load(**cassette_kwargs)
../../../.local/share/virtualenvs/warn-transformer-1C5zOao7/lib/python3.9/site-packages/vcr/cassette.py:171: in load
    new_cassette._load()
../../../.local/share/virtualenvs/warn-transformer-1C5zOao7/lib/python3.9/site-packages/vcr/cassette.py:344: in _load
    requests, responses = self._persister.load_cassette(self._path, serializer=self._serializer)
../../../.local/share/virtualenvs/warn-transformer-1C5zOao7/lib/python3.9/site-packages/vcr/persisters/filesystem.py:28: in load_cassette
    return deserialize(data, serializer)
../../../.local/share/virtualenvs/warn-transformer-1C5zOao7/lib/python3.9/site-packages/vcr/serialize.py:37: in deserialize
    data = serializer.deserialize(cassette_string)
../../../.local/share/virtualenvs/warn-transformer-1C5zOao7/lib/python3.9/site-packages/vcr/serializers/yamlserializer.py:12: in deserialize
    return yaml.load(cassette_string, Loader=Loader)
../../../.local/share/virtualenvs/warn-transformer-1C5zOao7/lib/python3.9/site-packages/yaml/__init__.py:81: in load
    return loader.get_single_data()
../../../.local/share/virtualenvs/warn-transformer-1C5zOao7/lib/python3.9/site-packages/yaml/constructor.py:49: in get_single_data
    node = self.get_single_node()
yaml/_yaml.pyx:673: in yaml._yaml.CParser.get_single_node
    ???
yaml/_yaml.pyx:687: in yaml._yaml.CParser._compose_document
    ???
yaml/_yaml.pyx:731: in yaml._yaml.CParser._compose_node
    ???
yaml/_yaml.pyx:845: in yaml._yaml.CParser._compose_mapping_node
    ???
yaml/_yaml.pyx:729: in yaml._yaml.CParser._compose_node
    ???
yaml/_yaml.pyx:806: in yaml._yaml.CParser._compose_sequence_node
    ???
yaml/_yaml.pyx:731: in yaml._yaml.CParser._compose_node
    ???
yaml/_yaml.pyx:845: in yaml._yaml.CParser._compose_mapping_node
    ???
yaml/_yaml.pyx:731: in yaml._yaml.CParser._compose_node
    ???
yaml/_yaml.pyx:845: in yaml._yaml.CParser._compose_mapping_node
    ???
yaml/_yaml.pyx:731: in yaml._yaml.CParser._compose_node
    ???
yaml/_yaml.pyx:847: in yaml._yaml.CParser._compose_mapping_node
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   yaml.reader.ReaderError: unacceptable character #x0080: control characters are not allowed
E     in "<unicode string>", position 1060035

yaml/_yaml.pyx:860: ReaderError
========================================================================= FAILURES =========================================================================
______________________________________________________________________ test_integrate ______________________________________________________________________

    @pytest.mark.runvcr
    @pytest.mark.vcr()
    def test_integrate():
        """Test integrate."""
        this_dir = Path(__file__).parent
        new_path = this_dir / "data" / "processed" / "consolidated.csv"
>       integrate.run(new_path, init_current_data=True)

tests/test_integrate.py:14: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
warn_transformer/integrate.py:34: in run
    current_data_list = get_current_data(init_current_data)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

init = True

    def get_current_data(init: bool = False) -> typing.List[typing.Dict[str, typing.Any]]:
        """Fetch the most recent published version of our integrated dataset.
    
        Args:
            init (bool): Set to True when you want to create a new integrated dataset from scratch. Default False.
    
        Returns a list of dictionaries ready for comparison against the new consolidated data file.
        """
        # Set which file to pull
        base_url = "https://raw.githubusercontent.com/biglocalnews/warn-github-flow/transformer/data/warn-transformer/processed/"
        if init:
            current_url = f"{base_url}consolidated.csv"
            logger.debug(f"Initializing new current file from {current_url}")
        else:
            current_url = f"{base_url}integrated.csv"
            logger.debug(f"Downloading most recent current file from {current_url}")
    
        # Download the current database
        current_r = requests.get(current_url)
>       current_data_str = current_r.content.decode("utf-8")
E       UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

warn_transformer/integrate.py:295: UnicodeDecodeError
===================================================================== warnings summary =====================================================================
tests/test_consolidate.py::test_consolidate
  /home/stucka/Desktop/data/warn-transformer/warn_transformer/schema.py:18: RemovedInMarshmallow4Warning: Passing field metadata as keyword arguments is deprecated. Use the explicit `metadata=...` argument instead. Additional metadata: {'max_length': 2}
    postal_code = fields.Str(max_length=2, required=True)

tests/test_consolidate.py::test_consolidate
tests/test_consolidate.py::test_consolidate
tests/test_consolidate.py::test_consolidate
  /home/stucka/.local/share/virtualenvs/warn-transformer-1C5zOao7/lib/python3.9/site-packages/marshmallow/fields.py:1176: RemovedInMarshmallow4Warning: The 'default' argument to fields is deprecated. Use 'dump_default' instead.
    super().__init__(**kwargs)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================= short test summary info ==================================================================
FAILED tests/test_integrate.py::test_integrate - UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
ERROR tests/test_download.py::test_download - yaml.reader.ReaderError: unacceptable character #x0080: control characters are not allowed
==================================================== 1 failed, 1 passed, 4 warnings, 1 error in 12.36s ===================================================

@stucka
Copy link
Contributor Author

stucka commented Sep 15, 2023

And yet Github's actually ... doing the same core OS for the same tests.

Operating System
Ubuntu
22.04.3
LTS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant