Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CURA 10831 #1

Merged
merged 89 commits into from
Dec 8, 2023
Merged
Show file tree
Hide file tree
Changes from 88 commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
797598c
Add script to retrieve Conan broadcast data
jellespijker Nov 10, 2023
b25335f
use wget to download runner_scripts
jellespijker Nov 10, 2023
46e6ccb
Ensure every arg has a value
jellespijker Nov 10, 2023
b83bc79
Ensure that repo's without tags start with version 0.0.1-alpha
jellespijker Nov 10, 2023
9ad7f28
Add script to remove Conan binaries
jellespijker Nov 11, 2023
ae7695d
Add presentation about the buildsystem
jellespijker Nov 11, 2023
9c7a52d
Refactor get_conan_broadcast_data to use conandata.yml
jellespijker Nov 11, 2023
1cf3c9d
Only checkout conandata.yml
jellespijker Nov 11, 2023
a77ee56
No external deps
jellespijker Nov 11, 2023
53f7c46
No pip install needed
jellespijker Nov 11, 2023
5ea29d7
add yaml as dependency
jellespijker Nov 11, 2023
c5a2643
Don't use `release` key, value from conandata.yml
jellespijker Nov 11, 2023
1946b9a
Don't require release
jellespijker Nov 11, 2023
03693b7
Added Conan recipe export workflow
jellespijker Nov 11, 2023
1e1ef03
Use `CONAN_PRIVATE_ARTIFACTORY_URL`
jellespijker Nov 11, 2023
5f3c072
Allow write to the contents
jellespijker Nov 11, 2023
7e7928f
Use the correct pip req file
jellespijker Nov 11, 2023
83d7114
only try to upload package to cura-private when internal
jellespijker Nov 11, 2023
bd455f4
Only create stable packages on release branch
jellespijker Nov 11, 2023
433b434
Update Conan recipe export workflow
jellespijker Nov 12, 2023
a221680
Don't decouple update system and export binaries
jellespijker Nov 13, 2023
7927882
Fix caching of Conan data and downloads
jellespijker Nov 14, 2023
f148d06
Add Conan package create workflows for Linux, MacOS, and Windows
jellespijker Nov 14, 2023
20089f9
Use curl on Windows
jellespijker Nov 14, 2023
109ca5c
Remove creating the alias (latest)
jellespijker Nov 14, 2023
44c1317
flip slashes
jellespijker Nov 14, 2023
37d8dee
fix empty reqs
jellespijker Nov 14, 2023
3e81434
move packages
jellespijker Nov 17, 2023
a6886e7
Allow version override
jellespijker Nov 27, 2023
e77eba8
Add `libxcb-cursor-dev` as dependency
jellespijker Nov 29, 2023
16bed12
Add conan-ultimaker-index upload script
jellespijker Nov 30, 2023
fbdbc55
Use branch to determine channel
jellespijker Nov 30, 2023
c2f4850
use recipes folder
jellespijker Nov 30, 2023
64a074e
use correct package name
jellespijker Nov 30, 2023
7c389f0
use relative abs path
jellespijker Nov 30, 2023
767a4df
use abs path for cwd
jellespijker Nov 30, 2023
852bce2
fix configs
jellespijker Nov 30, 2023
e9f8b4e
use str instead of Path
jellespijker Nov 30, 2023
21e964f
use str instead of Path
jellespijker Nov 30, 2023
8aa3437
use lowercase names for channel
jellespijker Nov 30, 2023
05ba6cf
Port benchmark workflows
jellespijker Dec 1, 2023
d0372be
Add process-pull-request workflow
jellespijker Dec 1, 2023
cd7998a
Add Slack notify workflow
jellespijker Dec 4, 2023
4344a44
Moving linux installer workflow over
jellespijker Dec 4, 2023
c4313da
Set Filename parameters with args
jellespijker Dec 4, 2023
784062c
Use Filename as args
jellespijker Dec 4, 2023
71e4cdc
Use correct environment variables
jellespijker Dec 4, 2023
79d1663
Port the Windows installer workflow
jellespijker Dec 4, 2023
4b8ec22
Port the MacOS installer workflow
jellespijker Dec 4, 2023
b2c1d8a
fix `wget` not present on self-hosted mac runners
jellespijker Dec 4, 2023
ce472eb
Source the AppImage script
jellespijker Dec 4, 2023
2f3e807
Use curl to get pip requirements
jellespijker Dec 4, 2023
b6d1857
Install AppImage deps
jellespijker Dec 4, 2023
39fa6fa
Use curl instead of wget
jellespijker Dec 4, 2023
31b8c74
Use correct slahes for Mac
jellespijker Dec 4, 2023
678ba25
Fix path for macos_setup.sh
jellespijker Dec 4, 2023
df41e2c
Use actual installer_filename as heading value
jellespijker Dec 4, 2023
22b065b
Use the force
jellespijker Dec 4, 2023
10972b1
Fix private remopte
jellespijker Dec 4, 2023
c84e926
Port default values to cura-workflows
jellespijker Dec 4, 2023
0a7aa22
Only upload to cura-private when internal
jellespijker Dec 4, 2023
7ba7865
Use `runner/internal` branch form conan-config
jellespijker Dec 4, 2023
c16845f
Build the installer internally when required
jellespijker Dec 4, 2023
4ec0134
Port unit-test to cura-workflows
jellespijker Dec 5, 2023
305906c
Activate all activate_github_action envs
jellespijker Dec 5, 2023
4c7679f
Ensure PATH is known to runner
jellespijker Dec 5, 2023
74c97ce
Post the unit test result on PR's
jellespijker Dec 5, 2023
67cd4f8
Give write permission on QT post
jellespijker Dec 5, 2023
672f3b4
Merge branch 'main' into CURA-10831
jellespijker Dec 6, 2023
219978d
Add a conan-package-release workflow
jellespijker Dec 6, 2023
d14b48a
Fix inputs
jellespijker Dec 6, 2023
1e35eb3
Use `inputs` instead of `github.event.inputs`
jellespijker Dec 6, 2023
e64726f
optional `latest` alias
jellespijker Dec 6, 2023
60b2842
Remove unused env var `CONAN_PRIVATE_ARTIFACTORY_URL`
jellespijker Dec 6, 2023
7d9818a
Allow setting user/channel
jellespijker Dec 6, 2023
d02bd79
Fix missing arg channel
jellespijker Dec 6, 2023
d24e393
fix indent
jellespijker Dec 6, 2023
6b1c47f
Shorten descriptions
jellespijker Dec 6, 2023
a0e7288
Use sip<=6.7.12
jellespijker Dec 6, 2023
f471956
Also build the binaries after an export
jellespijker Dec 7, 2023
408369e
Fix to many inputs
jellespijker Dec 7, 2023
2f02908
allow runner_scripts directory to be present
jellespijker Dec 7, 2023
6c45a7b
Revert "Fix to many inputs"
jellespijker Dec 7, 2023
52f8fa4
Revert "Also build the binaries after an export"
jellespijker Dec 7, 2023
630d36c
remove private parts and upload later
jellespijker Dec 8, 2023
847492e
Upload binaries to Sentry
jellespijker Dec 8, 2023
3f8b3c4
Update debug symbols upload paths in GitHub workflows
jellespijker Dec 8, 2023
45774c3
Apply suggestions from code review
jellespijker Dec 8, 2023
ab43f57
Update runner_scripts/cura_installer_filename.py
jellespijker Dec 8, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 142 additions & 0 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
name: Benchmark

on:
workflow_call:
inputs:
recipe_id_full:
required: true
type: string

conan_extra_args:
required: false
default: ""
type: string

benchmark_cmd:
required: true
type: string

name:
required: true
type: string

output_file_path:
required: true
type: string

data_dir:
required: true
type: string

tool:
required: true
type: string

permissions:
contents: write
deployments: write

env:
CONAN_LOGIN_USERNAME: ${{ secrets.CONAN_USER }}
CONAN_PASSWORD: ${{ secrets.CONAN_PASS }}

jobs:
benchmark:
name: Run benchmark
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v4
if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
with:
fetch-depth: 1
ref: ${{ github.head_ref }}

- name: Checkout repo PR
uses: actions/checkout@v4
if: ${{ github.event.pull_request.head.repo.full_name != github.repository }}
with:
fetch-depth: 1
ref: ${{ github.base_ref }}

- name: Sync pip requirements
run: wget https://raw.githubusercontent.com/Ultimaker/cura-workflows/main/.github/workflows/requirements-runner.txt -O .github/workflows/requirements-runner.txt

- name: Setup Python and pip
uses: actions/setup-python@v4
with:
python-version: 3.11.x
cache: pip
cache-dependency-path: .github/workflows/requirements-runner.txt

- name: Install Python requirements and Create default Conan profile
run: pip install -r .github/workflows/requirements-runner.txt

- name: Install Linux system requirements for building
run: |
mkdir runner_scripts
wget https://raw.githubusercontent.com/Ultimaker/cura-workflows/main/runner_scripts/ubuntu_setup.sh -O runner_scripts/ubuntu_setup.sh
chmod +x runner_scripts/ubuntu_setup.sh
sudo ./runner_scripts/ubuntu_setup.sh

- name: Setup pipeline caches
run: |
mkdir -p /home/runner/.conan/downloads
mkdir -p /home/runner/.conan/data

- name: Create default Conan profile
run: conan profile new default --detect

- name: Get Conan configuration
run: |
conan config install https://github.com/Ultimaker/conan-config.git
conan config install https://github.com/Ultimaker/conan-config.git -a "-b runner/${{ runner.os }}/${{ runner.arch }}"

- name: Cache Conan packages
uses: actions/cache@v3
with:
path: /home/runner/.conan/data
key: ${{ runner.os }}-conan-data-${{ github.run_id }}
restore-keys: |
${{ runner.os }}-conan-data-

- name: Cache Conan downloads
uses: actions/cache@v3
with:
path: /home/runner/.conan/downloads
key: ${{ runner.os }}-conan-downloads-${{ github.run_id }}
restore-keys: |
${{ runner.os }}-conan-downloads-

- name: Install dependencies
run: conan install . ${{ inputs.recipe_id_full }} -s build_type=Release --build=missing --update -g GitHubActionsRunEnv -g GitHubActionsBuildEnv ${{ inputs.conan_extra_args }}

- name: Set Environment variables from Conan install (bash)
if: ${{ runner.os != 'Windows' }}
run: |
. ./activate_github_actions_runenv.sh
. ./activate_github_actions_buildenv.sh
working-directory: build/Release/generators

- name: Build
run: |
cmake --preset release
cmake --build --preset release

- name: Run benchmark
id: run-test
run: ${{ inputs.benchmark_cmd }}
working-directory: build/Release

- name: Store benchmark result
uses: benchmark-action/github-action-benchmark@v1
with:
name: ${{ inputs.name }}
output-file-path: ${{ inputs.output_file_path }}
gh-repository: github.com/Ultimaker/CuraEngineBenchmarks
gh-pages-branch: main
benchmark-data-dir-path: ${{ inputs.data_dir }}
tool: ${{ inputs.tool }}
github-token: ${{ secrets.CURA_BENCHMARK_PAT }}
auto-push: true
max-items-in-chart: 250
34 changes: 34 additions & 0 deletions .github/workflows/check-actor.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: Check Actor

on:
workflow_call:
outputs:
proceed:
description: "Is workflow started by a legit actor? No bots or forks"
value: ${{ jobs.check_actor.outputs.proceed }}

permissions:
contents: read

env:
CONAN_LOGIN_USERNAME: ${{ secrets.CONAN_USER }}
CONAN_PASSWORD: ${{ secrets.CONAN_PASS }}

jobs:
check_actor:
runs-on: ubuntu-latest
outputs:
proceed: ${{ steps.skip_check.outputs.proceed }}
steps:
- id: skip_check
run: |
if [[ "${{ github.actor }}" == *"[bot]"* ]]; then
echo "proceed=true" >> $GITHUB_OUTPUT
elif [[ "${{ github.event.pull_request }}" == "" ]]; then
echo "proceed=true" >> $GITHUB_OUTPUT
elif [[ "${{ github.event.pull_request.head.repo.fork }}" == "false" ]]; then
echo "proceed=true" >> $GITHUB_OUTPUT
else
echo "proceed=false" >> $GITHUB_OUTPUT
fi
shell: bash
110 changes: 110 additions & 0 deletions .github/workflows/conan-package-create-linux.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
name: Conan package create Linux

on:
workflow_call:
inputs:
recipe_id_full:
required: true
type: string

conan_internal:
required: false
default: false
type: boolean

conan_extra_args:
required: false
default: ""
type: string

permissions:
contents: read

env:
CONAN_LOGIN_USERNAME: ${{ secrets.CONAN_USER }}
CONAN_PASSWORD: ${{ secrets.CONAN_PASS }}

jobs:
package-create:
runs-on: ubuntu-latest

steps:
- name: Checkout repo
uses: actions/checkout@v4
if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
with:
fetch-depth: 1
ref: ${{ github.head_ref }}

- name: Checkout repo PR
uses: actions/checkout@v4
if: ${{ github.event.pull_request.head.repo.full_name != github.repository }}
with:
fetch-depth: 1
ref: ${{ github.base_ref }}

- name: Sync pip requirements
run: wget https://raw.githubusercontent.com/Ultimaker/cura-workflows/main/.github/workflows/requirements-runner.txt -O .github/workflows/requirements-runner.txt

- name: Setup Python and pip
uses: actions/setup-python@v4
with:
python-version: 3.11.x
cache: pip
cache-dependency-path: .github/workflows/requirements-runner.txt

- name: Install Python requirements and Create default Conan profile
run: pip install -r .github/workflows/requirements-runner.txt

- name: Install Linux system requirements for building
run: |
mkdir -p runner_scripts
wget https://raw.githubusercontent.com/Ultimaker/cura-workflows/main/runner_scripts/ubuntu_setup.sh -O runner_scripts/ubuntu_setup.sh
chmod +x runner_scripts/ubuntu_setup.sh
sudo ./runner_scripts/ubuntu_setup.sh

- name: Setup pipeline caches
run: |
mkdir -p /home/runner/.conan/downloads
mkdir -p /home/runner/.conan/data

- name: Create default Conan profile
run: conan profile new default --detect

- name: Get Conan configuration
run: |
conan config install https://github.com/Ultimaker/conan-config.git
conan config install https://github.com/Ultimaker/conan-config.git -a "-b runner/${{ runner.os }}/${{ runner.arch }}"

- name: Add Cura private Artifactory remote
if: ${{ inputs.conan_internal }}
run: conan config install https://github.com/Ultimaker/conan-config.git -a "-b runner/internal"

- name: Cache Conan packages
uses: actions/cache@v3
with:
path: /home/runner/.conan/data
key: ${{ runner.os }}-conan-data-${{ github.run_id }}
restore-keys: |
${{ runner.os }}-conan-data-

- name: Cache Conan downloads
uses: actions/cache@v3
with:
path: /home/runner/.conan/downloads
key: ${{ runner.os }}-conan-downloads-${{ github.run_id }}
restore-keys: |
${{ runner.os }}-conan-downloads-

- name: Export the Package (binaries)
run: conan create . ${{ inputs.recipe_id_full }} --build=missing --update ${{ inputs.conan_extra_args }}

- name: Upload the Package(s)
if: ${{ always() && !inputs.conan_internal }}
run: |
conan upload ${{ inputs.recipe_id_full }} -r cura --all -c

- name: Upload the Package(s) to the private Artifactory
if: ${{ always() && inputs.conan_internal }}
run: |
conan upload ${{ inputs.recipe_id_full }} -r cura-private --all -c
Loading