Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[workflows] Add workflow for building revive in a debian container. #55

Merged
merged 1 commit into from
Sep 30, 2024

Conversation

wpt967
Copy link
Collaborator

@wpt967 wpt967 commented Sep 27, 2024

Makefile: Add target 'install-revive' to build revive with the installation path specified by variable REVIVE_INSTALL_DIR.

Add utils directory with scripts for building revive in a container.

Add utils/build-revive.sh taking option argument '-o ' to build revive with the specified install directory.

Add utils/revive-builder-debian.dockerfile to make a docker container for building revive in a Debian environment.

@wpt967 wpt967 force-pushed the revive-debian-x86-build branch 3 times, most recently from b7edfd9 to 83af5f5 Compare September 27, 2024 13:39

- name: build-container
run: |
(cd utils && ./${{ env.DEBIAN_CONTAINER_BUILDER}} . )
Copy link
Collaborator

@smiasojed smiasojed Sep 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did you decide to use your own docker. With this ci-unified
you have rust already installed and you can control the versions using docker tags.
The only issue which I see is that they are using debian 11 and remix backend is based on debian 12.

Copy link
Member

@xermicus xermicus Sep 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good point, the debian version in the builder should probably match the version where the binary is run. By keeping the images being used in sync we achieve it for free.

Note that if we settle on ci-unifed the supported rust version is 1.77 which is quite old. It'll probably able to build revive but I didn't try that.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the issue with installing rust in a Debian container?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any issue; I just prefer to avoid maintaining additional tools/images if I have something ready to use.

run: |
(cd utils && ./${{ env.DEBIAN_CONTAINER_BUILDER}} --build-arg RUST_VERSION=${{ env.RUST_VERSION}} . )

- name: build-revive-debian
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be good to print the versions of the tools being used. Please take a look here

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Member

@xermicus xermicus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

DEBIAN_CONTAINER_RUNNER: run-debian-builder.sh
REVIVE_DEBIAN_INSTALL: ${{ github.workspace }}/target/release
REVIVE_DEBIAN_BINARY: resolc
RUST_VERSION: stable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem here is stable will change over time, but we want to have manual control over it. We can stick to 1.80 (slightly older version should do fine too).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to 1.80

libmpfr-dev libgmp-dev libmpc-dev ncurses-dev \
git curl
EOF
ARG RUST_VERSION=stable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, I think sticking to specific version should be better.

What is missing is a rust version in the top level Cargo.toml:

rust-version = "1.80.0"

Could you add this here and to the top level Cargo.toml?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The value in the Dockerfile is a default value, the value used is set in in the workflow file.

Changed the Cargo.toml file to specify the rust-version is 1.80

REVIVE_INSTALL_DIR=$OPTARG
;;
\?) echo "Error: Invalid option"
exit;;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably return non-zero value here?

Suggested change
exit;;
exit 1;;

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

done
echo "Installing to ${REVIVE_INSTALL_DIR}"

$(pwd)/build-llvm.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To get rid of doing this every time we release, we can have another workflow (could even be a separate repository).

The idea is that this separate workflow only builds and releases LLVM, and here we can just download the LLVM release. The assumption is that we do not change the LLVM version often, should be way less often than we release revive.

However it can be done later, it shouldn't block us for now.

jobs:
build-revive-debian-x86:
name: debian-container-x86
runs-on: ubuntu-latest
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the run takes more than half an hour, maybe we should use beefier runners. I remember we had ubuntu-latest-8-cores and ubuntu-latest-16-cores runners.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we should use the more beefy ones. But building resolc itself takes seconds to minutes even on potato machines. I guess the main problem is LLVM and this is solved by having a dedicated release workflow for LLVM. So I think we should try both: Beefy runners + more economic release workflows.

Makefile: Add target 'install-revive' to build revive with the
installation path specified by variable REVIVE_INSTALL_DIR.

Add utils directory with scripts for building revive in a container.

Add utils/build-revive.sh taking option argument '-o <install-dir>' to
build revive with the specified install directory.

Add utils/revive-builder-debian.dockerfile to make a docker container
for building revive in a Debian environment.
@wpt967 wpt967 merged commit e8fcd61 into paritytech:main Sep 30, 2024
2 checks passed
@wpt967 wpt967 deleted the revive-debian-x86-build branch September 30, 2024 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants