Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install julia via pixi #878

Merged
merged 26 commits into from
Dec 13, 2023
Merged

Install julia via pixi #878

merged 26 commits into from
Dec 13, 2023

Conversation

deltamarnix
Copy link
Contributor

@deltamarnix deltamarnix commented Dec 8, 2023

One step less in installing your development environment. Juliaup is now installed via pixi and the julia executable itself with the supported version is installed in <project>/juliaup. The packages that we use within julia are still shared, as they are not precompiled in ~/.julia.

This enables us to build our packages without using the pre-installed versions of julia on our Teamcity build agents.

Julia is only installed once via pixi on Github Actions, it then performs the tests over pixi run test-ribasim-core-cov.

Part of #93

@deltamarnix deltamarnix marked this pull request as ready for review December 8, 2023 10:07
Comment on lines +173 to +174
[system-requirements]
linux = "3.10.0"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was needed for the linux build. We could leave it out of this PR, I don't know if there is anyone with a linux system that wants to test if it is working for ubuntu.

Copy link
Contributor

@Hofer-Julian Hofer-Julian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR works locally on my machine

@deltamarnix
Copy link
Contributor Author

deltamarnix commented Dec 8, 2023

Julia tests are failing, since the steps are not compliant to what we do in pixi. We need to use the correct environment and no need to install julia here.

    steps:
      - uses: actions/checkout@v4
      - uses: prefix-dev/setup-pixi@v0.4.1
        with:
          pixi-version: "latest"
      - name: Prepare pixi
        run: pixi run install-without-pre-commit
      - name: Prepare model input
        run: |
          pixi run generate-testmodels
      - uses: julia-actions/setup-julia@v1
        with:
          version: ${{ matrix.julia_version }}
          arch: ${{ matrix.arch }}
      - uses: julia-actions/cache@v1
        with:
          cache-compiled: "true"
          cache-registries: "true"
      - uses: julia-actions/julia-buildpkg@v1
        with:
          project: core
      - uses: julia-actions/julia-runtest@v1
        with:
          project: core
      - uses: julia-actions/julia-processcoverage@v1
        with:
          directories: core/src
      - uses: codecov/codecov-action@v3
        with:
          files: lcov.info

@deltamarnix
Copy link
Contributor Author

deltamarnix commented Dec 8, 2023

@visr My branch is still failing, even after a simple retry. This is the error

 ERROR: The following 1 direct dependency failed to precompile:

Ribasim [aac5e3d9-0b8f-4d4f-8241-b1a7a9632635]

Failed to precompile Ribasim [aac5e3d9-0b8f-4d4f-8241-b1a7a9632635] to "C:\\Users\\runneradmin\\.julia\\compiled\\v1.10\\Ribasim\\jl_A69F.tmp".
ERROR: LoadError: InitError: could not load library "C:\Users\runneradmin\.julia\artifacts\3fb1f0562f4bfbe9d22f7f2e506095c800807a9d\bin\libhighs.dll"
Access is denied. 

@deltamarnix
Copy link
Contributor Author

Also, github action now builds julia twice, which is unnecessary, we should fix that in a new PR. I will make a ticket for it when this branch is merged.

@@ -76,6 +76,10 @@ test-ribasim-core = { cmd = "julia --project=core --eval 'using Pkg; Pkg.test()'
"instantiate-julia",
"generate-testmodels",
] }
test-ribasim-core-cov = { cmd = "julia --project=core --eval 'using Pkg; Pkg.test(coverage=true, julia_args=[\"--check-bounds=yes\"])'", depends_on = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if the check-bounds flag is needed. I couldn't find a single @inbounds in our code.
What do you say @visr?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would be fine with disabling it. Our dependencies do use that I assume, but probably they are testing with check-bounds.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not really in the way, is it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My idea was to reduce the number of options we pass. I am fine with keeping it if you prefer that

Copy link
Contributor

@Hofer-Julian Hofer-Julian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR improves things on so many fronts. Great work!

@Hofer-Julian Hofer-Julian merged commit 9531488 into main Dec 13, 2023
@Hofer-Julian Hofer-Julian deleted the feat/linux-build branch December 13, 2023 08:50
visr added a commit that referenced this pull request Jul 8, 2024
In #878 we started installing julia via pixi. We kept the packages
shared under `~/.julia` to avoid unnecessary duplication. We did put the
julia installation under `utils/juliaup` by setting JULIAUP_DEPOT_PATH.
This way we could set a default channel for our juliaup.

However if we can manage to set the default channel for our repository
having the julia installations in the normal place is preferred, because
it won't have to download julia versions that are already installed.
Looking at [`juliaup
override`](https://github.com/JuliaLang/juliaup?tab=readme-ov-file#overrides)
we can achieve this with `juliaup override set 1.10.4`. This also has
the advantage that you get the desired channel with both `pixi run
julia` and `julia`. That's nice because my global default channel is
1.11.

```
❯ juliaup override status
 Path  Channel
---------------

Ribasim on  home-depot [$!?] via ஃ v1.10.4 via 🐍
❯ pixi run install-julia
✨ Pixi task (install-julia in dev): juliaup add 1.10.4 && juliaup override set 1.10.4
'1.10.4' is already installed.

Ribasim on  home-depot [$!?] via ஃ v1.10.4 via 🐍 took 3s
❯ juliaup override status
 Path                     Channel
----------------------------------
 D:\repo\ribasim\Ribasim  1.10.4
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants