BUG: Poetry 2.0.0 release leads to dependency resolution failure with Habushu plugin in CI build

[cpointe-ibllanos]

Description

The last CI build for aiSSEMBLE failed. The issue seems to be relating to the 2.0.0 release of poetry and misaligned versioning: where Habushu expects to run 1.x.x, aiSSEMBLE is pulling 2.0.0.

Definition of done

• Plan A: Support Poetry 2.0.0 on Habushu

  1. Update Habushu with poetry 2.0.0 support
  2. Build locally with tests passing mvn clean install -Pbootstrap then mvn clean install
  3. Build Habushu on CI using GH action
  4. Cut a new Habushu release
  5. Upgrade Habushu on aiSSEMBLE's parent pom
  6. Make sure this passes on local aiSSEMBLE build mvn clean install
  7. Repeat above for aiSSEMBLE CI build
  8. Run through aiSSEMBLE regression testing

• Plan B: Downgrade aiSSEMBLE Poetry version to be <2.0.0

  1. Upgrade aiSSEMBLE components to support Poetry <2.0.0
  2. Build locally
  3. Build on CI via GH Actions

In weighing the two options, we have decided to go with Plan B since the latest version of Poetry could introduce breaking changes within Habushu. We will spin off a separate ticket to handle that upgrade.

Steps to Reproduce

Clear, specific, and detailed steps taken to enable reproduction of the bug for investigation.

1. Go to Actions
2. Click on "Build aiSSEMBLE"
3. Click "Run Workflow"
4. Scroll down to "Build aiSSEMBLE" in the workflow logs after it fails
5. See error:

[Poetry version 2.0.0 was installed - Habushu requires that installed version of Poetry satisfies ^1.5.0.  Please update Poetry by executing 'poetry self update' or visit https://python-poetry.org/docs/#installation for more information]

Alternatively, you may see the following instead during the docker versioning build:

[INFO] DOCKER> 19.65 Because poetry-plugin-bundle (1.3.0) depends on poetry (>=1.5.0,<2.0.0)
[INFO] DOCKER> 19.65  and poetry-instance depends on poetry (2.0.0), poetry-plugin-bundle is forbidden.
[INFO] DOCKER> 19.65 So, because poetry-instance depends on poetry-plugin-bundle (1.3.0), version solving failed.
[INFO] DOCKER> ------
[INFO] DOCKER> Dockerfile:4
[INFO] DOCKER> --------------------
[INFO] DOCKER>    3 |     # Poetry and supporting plugin installations
[INFO] DOCKER>    4 | >>> RUN python -m ensurepip --upgrade && \
[INFO] DOCKER>    5 | >>>     pip install poetry && \
[INFO] DOCKER>    6 | >>>     poetry self add poetry-monorepo-dependency-plugin && \
[INFO] DOCKER>    7 | >>>     poetry self add poetry-plugin-bundle@1.3.0
[INFO] DOCKER>    8 |     
[INFO] DOCKER> --------------------
[INFO] DOCKER> ERROR: failed to solve: process "/bin/sh -c python -m ensurepip --upgrade &&     pip install poetry &&     poetry self add poetry-monorepo-dependency-plugin &&     poetry self add poetry-plugin-bundle@1.3.0" did not complete successfully: exit code: 1
[ERROR] DOCKER> Error status (1) when building

Expected Behavior

A successful build in GitHub Actions

Actual Behavior

Build fails due to above error

[cpointe-ibllanos]

Reassigning as I have an aging effort I'm working on simultaneously.

Additionally, moving away from the approach of upgrading Habushu for now as some of the default behaviors and syntax used in that project have been altered with the new poetry release:

Change the default behavior of poetry lock to --no-update and introduce a --regenerate option for the old default behavior (#9327).
Replace virtualenvs.prefer-active-python by the inverse setting virtualenvs.use-poetry-python and prefer the active Python by default (#9786).
Deprecate poetry install --sync in favor of poetry sync (#9801)

Will simply downgrade the GH environment's poetry version for the pipeline for now -- saving upgrading habushu for a potential follow-up effort.

[meliz19]

Completed OTS with @chang-annie & @cpointe-ibllanos

[meliz19]

CI build was successful.

Final test completed by @JeffreyRoss.