Skip to content

Latest commit

 

History

History
210 lines (174 loc) · 7.22 KB

cEP-0035.md

File metadata and controls

210 lines (174 loc) · 7.22 KB
Raw

Extending support for newer versions of Python

Metadata
cEP 0035
Version 1.0
Title Extending support for newer version of Python
Authors Anshal Shukla mailto:shukla.anshal85@gmail.com
Status Proposed
Type Process

Abstract

This cEP describes the upgrades and steps required to enable coala to support newer versions of Python. This project has been taken up as a GSoC'21 project.

Introduction

Lack of support for newer versions of Python has been a significant hurdle in the development of coala. coala, as of now, supports Python up to version 3.6, which is quite old. Version 3.6 was released back in 2016, and Python has made considerable progress since then. coala, on the other hand, has largely not been able to benefit from these upgrades because of the strict version limitations. Upgrading coala to support newer versions will accelerate the development of coala and will make it future proof.

Objectives

The project's primary goal is to enable coala to support newer Python versions and attain this in multiple smaller steps. The goal is not just to facilitate support for newer versions of Python but also to upgrade the coala to support more recent dependencies. Upgrades will be performed over multiple repositories as the dependencies across the project are closely interlinked. Will perform changes in coala, coala-bears, dependency_management, docker, mobans and coala-utils repository.

Proposed Approach

Upgrade dependency_management system

The dependency_management system provides a Python API for familiar package managers. It provides a generic method to install dependencies in the host operating system. It has been a while since it has been upgraded. Change it to support latest package manager versions.

Upgrade bears

The problems and procedure associated with upgrading bears are:

  • Fix bears to support pip version >10.
  • Pytest fails with 16 warnings, eight errors on pip 21.0.1, with total coverage dropping to 42.61%.
  • Pytest fails with 7 skipped, 26 warnings, ten errors on Python3.9 with coverage dropping to 22.95%
  • Python/pip upgrade breaks bear installation and compilation terminates. The list of dependencies with these issues are:
    1. wrapt: Current version is unsupported, newer version will be used instead.
    2. typed-ast: Current version is unsupported, newer version will be used instead. Required changes in bears for compatibility.
    3. aenum: Current version is deprecated for Python3, newer version will be used instead.
    4. lib2to3yaml: Current version is deprecated and may not be able to parse Python 3.10+. awpa can be used as an alternative.
    5. stevedore: Current version does not support Python 3.9, newer version will be used instead.
    6. libyaml: This package is not available in PyPI of Python 3.9. yaml2 can be used as an alternative.
  • There are a number of bears that fail due to such import errors caused by Python or pip upgrade. Some of which are:
    1. FormatBear
    2. RLintBear
    3. CPDBearTest
    4. PEP8NotebookBear
  • Import setup tools fail. This also is caused because of lack of dependencies.
  • Usage of dash-separated description file will not be supported in future versions, update these to underscore names.
  • Escape sequence at several places are deprecated(warning) in python 3.9, use raw strings to avoid warnings.

Upgrade coala

The problems and procedure associated with upgrading coala are:

  • pytest fails with ten warning and one error on python 3.9, with total coverage dropping to 23.28%

  • core, language definitions, results and bearlib have a missing portions that decreases the overall coverage drastically.

  • Replace "is" and "is not" by "==" and "!=" to avoid syntax warnings on Python 3.8+.

  • Fix invalid syntax error for Python 3.7:

    import asyncio

async def main():
print('Hello ...')
await asyncio.sleep(1)
print('... World!')

# Python 3.7+
asyncio.run(main())

Add GitHub Actions

GitHub actions will be added as an alternative CI. This will be performed in two steps

  1. Write configurations for GitHub Actions pipeline. Basic working structure for this is as follows:

  • sentinel workflow

    name: sentinel

on: [push]

jobs:
build:
    runs-on: ubuntu-16.04
    env:
    BEARS_ZIP_URL: https://codeload.github.com/coala/coala-bears/zip
    steps:
    - uses: actions/checkout@v2
    - name: Setup Python
        uses: actions/setup-python@v2
        with:
        python-version: '3.6.9'
    - name: Setup JDK
        uses: AdoptOpenJDK/install-jdk@v1
        with:
        version: '8'
    - name: before_install
        run: |
        pip install --prefer-binary cffi lxml
        pip uninstall setuptools --yes
        pip install pip==9.0.3 setuptools==21.2.2
        python .misc/check_setuptools.py
        cp requirements.txt requirements.orig
        cat test-requirements.txt docs-requirements.txt >> requirements.txt
        sed -i.bak '/^-r/d' requirements.txt
        npm install --no-save
        sudo npm i -g csslint
    - name: install
        run: |
        pip install -r requirements.txt
    - name: before_script
        run: |
        mv requirements.orig requirements.txt
        python .misc/check_setuptools.py
        sed -i.bak '/^-r/d' requirements.txt
    - name: script
        run: |
        py.test
        python setup.py bdist_wheel
        pip install ./dist/*.whl
        curl -fsSL -o coala-bears.zip $BEARS_ZIP_URL/master
        pip install coala-bears.zip[alldeps]
        coala --non-interactive
        python setup.py docs
        .ci/check_man.sh
    - name: after_script
        run: codecov
        ```

  • moban workflow

    name: Moban Sync

on: [push]

jobs:
build:
    runs-on: ubuntu-16.04
    steps:
    - uses: actions/checkout@v2
    - name: Setup Python
        uses: actions/setup-python@v2
        with:
        python-version: '3.6'
    - name: Install Dependencies
        run: pip install moban~=0.5.0 gitfs2 pypifs
    - name: script
        run: |
        moban
        git diff --exit-code
  1. Add moban template for this new pipeline.

Remove Circle CI and AppVeyor CI

Circle CI and AppVeyor were used to ensure Windows and MacOS support, these were also maintained as a fallback if any of the CI solution gets paid alternatively other can be used. But now with Virtual Machines we don't have to ensure Windows and Mac Support, also we now have GitHub actions which is available free of cost.

Update Docker Image

Fix docker image to serve the latest version of coala and bears.