Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pantsbuild: Make exported venv useful for running nose #6141

Merged
merged 6 commits into from
Feb 20, 2024
Merged

Conversation

cognifloyd
Copy link
Member

@cognifloyd cognifloyd commented Feb 16, 2024

With this change, you can run pants export --resolve=st2 to get a venv under dist/export/python/virtualenvs/st2/<python version>. The venv includes:

  • editable installs of all of our st2* (st2api, st2auth, ...) and runners
  • everything in lockfiles/st2.lock:
    • nose so that you can run the tests.
    • versions of pymongo and mongoengine that is compatible with our code
      • All unit tests pass with this

Pex (used by pants) will create the venv with the first interpreter on the path that matches the interpreter_constraints in pants.toml. If you want to generate a venv with a particular python binary use this:

pants export --resolve=st2 --python-bootstrap-search-path=[] --python-bootstrap-search-path=/usr/bin/python3.9

Replace /usr/bin/python3.9 with the interpreter you want pants to use.

This PR does not attempt to get the Makefile to use the venv. This PR just extracts the bits I'm confident about from #6130.

Summary of changes in the Lockfile:

==                    Upgraded dependencies                     ==

  certifi                        2023.11.17   -->   2024.2.2
  cryptography                   42.0.1       -->   42.0.3
  gitpython                      3.1.41       -->   3.1.42
  netaddr                        0.10.1       -->   1.1.0
  orjson                         3.9.12       -->   3.9.14
  pip                            23.3.2       -->   24.0
  platformdirs                   4.1.0        -->   4.2.0
  pytz                           2023.4       -->   2024.1
  setuptools                     69.0.3       -->   69.1.0
  ruamel-yaml                    0.18.5       -->   0.18.6
  tzdata                         2023.4       -->   2024.1
  urllib3                        2.1.0        -->   2.2.0
  voluptuous                     0.14.1       -->   0.14.2
  waitress                       2.1.2        -->   3.0.0

==                      Added dependencies                      ==

  colorama                       0.4.6
  nose                           1.3.7
  nose-parallel                  0.4.0
  nose-timer                     1.0.1
  rednose                        1.3.0
  termstyle                      0.1.11

==                !! Downgraded dependencies !!                 ==

  mongoengine                    0.27.0       -->   0.23.1
  pymongo                        4.6.1        -->   3.12.3

Lockfile diff: lockfiles/st2.lock [st2]

==                    Upgraded dependencies                     ==

  certifi                        2023.11.17   -->   2024.2.2
  cryptography                   42.0.1       -->   42.0.2
  orjson                         3.9.12       -->   3.9.13
  pip                            23.3.2       -->   24.0
  platformdirs                   4.1.0        -->   4.2.0
  pytz                           2023.4       -->   2024.1
  ruamel-yaml                    0.18.5       -->   0.18.6
  urllib3                        2.1.0        -->   2.2.0
  voluptuous                     0.14.1       -->   0.14.2
  waitress                       2.1.2        -->   3.0.0

==                      Added dependencies                      ==

  colorama                       0.4.6
  nose                           1.3.7
  nose-parallel                  0.4.0
  nose-timer                     1.0.1
  rednose                        1.3.0
  termstyle                      0.1.11
@cognifloyd cognifloyd added this to the pants milestone Feb 16, 2024
@cognifloyd cognifloyd requested a review from a team February 16, 2024 20:57
@cognifloyd cognifloyd self-assigned this Feb 16, 2024
@pull-request-size pull-request-size bot added the size/XL PR that changes 500-999 lines. Consider splitting work into several ones that easier to review. label Feb 16, 2024
Lockfile diff: lockfiles/st2.lock [st2]

==                    Upgraded dependencies                     ==

  cryptography                   42.0.2       -->   42.0.3
  gitpython                      3.1.41       -->   3.1.42
  netaddr                        0.10.1       -->   1.1.0
  orjson                         3.9.13       -->   3.9.14
  setuptools                     69.0.3       -->   69.1.0
  tzdata                         2023.4       -->   2024.1

==                !! Downgraded dependencies !!                 ==

  mongoengine                    0.27.0       -->   0.23.1
  pymongo                        4.6.1        -->   3.12.3
@cognifloyd cognifloyd requested review from a team and nzlosh February 20, 2024 14:13
@@ -45,7 +46,8 @@ prettytable
# For st2client: prompt-toolkit v2+ does not have prompt_toolkit.token.Token
prompt-toolkit<2
psutil
pymongo
# pymongo 3.13 has backports of APIs from pymongo 4 to help w/ migration
pymongo>=3.11.0,<3.13.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So 3.13 comes already with breaking changes? I'll study their changelog to get a better understanding. Besides that the PR looks good!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It at least has a bunch of deprecations that are likely to cause (many?) extra warnings in the logs.

@cognifloyd cognifloyd merged commit d987658 into master Feb 20, 2024
29 checks passed
@cognifloyd cognifloyd deleted the pants-venv branch February 20, 2024 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
external dependency pantsbuild size/XL PR that changes 500-999 lines. Consider splitting work into several ones that easier to review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants