Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Build system improvements and fixes #546

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
e83774f
docs typo and submodule version bump
RedLeader962 Dec 12, 2023
bb447bc
submodule version bump
RedLeader962 Dec 12, 2023
53e2749
docs: improve README.md, fix typo, mod changelog version to 1.4.0.
RedLeader962 Dec 12, 2023
c6a0779
refactor: bump LPM version number in C++ header to 1.4.0
RedLeader962 Dec 12, 2023
08e515a
docs: README.md improvement to the *Contributing* section
RedLeader962 Dec 12, 2023
88786f0
Modify Licence year
boxanm Dec 13, 2023
e4804da
Merge branch 'develop' into release
RedLeader962 Dec 14, 2023
8371f36
fix: make run_bats_core_test_in_n2st.bash script able to automaticall…
RedLeader962 Dec 14, 2023
e66b0d1
refactor: fix LPM build system root path resolution. Added .env.libp…
RedLeader962 Dec 15, 2023
4630521
docs: add build system shield.io and update git submodule instructions
RedLeader962 Dec 15, 2023
9cba0f8
Merge branch 'release' into dev-refactor-simplify-the-dockerfile-impl…
RedLeader962 Dec 15, 2023
71f8989
chore: add github code owner entries and add instructions
RedLeader962 Dec 17, 2023
476a28c
style: clean CODEOWNERS file
RedLeader962 Dec 17, 2023
76d0e24
chore: added a PR template and a commit msg references file
RedLeader962 Dec 17, 2023
5530c4f
docs: added readme related to _libpointmatcher-build-system_ and upda…
RedLeader962 Dec 17, 2023
3cc38ef
docs: readme contributing instructions + versions and updated PR temp…
RedLeader962 Dec 18, 2023
4d8caea
build: update submodule version to latest
RedLeader962 Dec 18, 2023
7fa11e9
chore: update IDE run configuration
RedLeader962 Dec 18, 2023
9c1d6a4
test: add tests for `lpm_crawl_libpointmatcher_build_matrix.bleeding.…
RedLeader962 Dec 18, 2023
70e1d73
feat: added lpm script to execute nbs crawler script
RedLeader962 Dec 18, 2023
57c6ee6
build: update submodule to latest
RedLeader962 Dec 18, 2023
7d4ef00
doc: update hyperlink to release branch and clean comment
RedLeader962 Dec 18, 2023
5ccb99d
Merge branch 'develop' into dev-refactor-simplify-the-dockerfile-impl…
RedLeader962 Dec 18, 2023
1517c1c
build: update NBS submodule version to lates
RedLeader962 Dec 18, 2023
cb91642
build: update submodules version to latest
RedLeader962 Dec 18, 2023
9048f30
Merge branch 'develop' into dev-refactor-simplify-the-dockerfile-impl…
RedLeader962 Dec 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .env.libpointmatcher
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# =================================================================================================
#
# Set project related environment variables. Those are available for convenience
# and are also required by 'norlab-shell-script-tools' library.
#
# Usage:
#
# Important! Source this file from 'libpointmatcher' repository root
# $ cd <path/to/libpointmatcher/>
# $ set -o allexport && source .env.libpointmatcher && set +o allexport
#
# =================================================================================================

PROJECT_PROMPT_NAME='LPM'

# ....Programaticaly fetch source code information.................................................
PROJECT_GIT_REMOTE_URL="https://github.com/norlab-ulaval/libpointmatcher"
PROJECT_GIT_NAME=$( basename "${PROJECT_GIT_REMOTE_URL}" .git )
PROJECT_PATH=$( git rev-parse --show-toplevel )
PROJECT_SRC_NAME="$( basename ${PROJECT_PATH} )"

# ....Set LPM related environment variable with their own prefix...................................
# Note: Those with "PROJECT_" prefix will get eventualy overiden in the case where N2ST is used
# as a library. Using generic testing logic require that environment variables with
# "PROJECT_" prefix be available.
LPM_PROMPT_NAME="${PROJECT_PROMPT_NAME"
LPM_GIT_REMOTE_URL="${PROJECT_GIT_REMOTE_URL}"
LPM_GIT_NAME="${PROJECT_GIT_NAME}"
LPM_PATH="${PROJECT_PATH}"
LPM_SRC_NAME="${PROJECT_SRC_NAME}"

# ....Set dependencies path........................................................................
LPM_BUILD_SYSTEM_PATH="${PROJECT_PATH}/build_system"
N2ST_PATH="${PROJECT_PATH}/build_system/utilities/norlab-shell-script-tools"
NBS_PATH="${PROJECT_PATH}/build_system/utilities/norlab-build-system"
30 changes: 23 additions & 7 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -1,13 +1,29 @@
# =================================================================================================
# Files or directories with designated code owner.
# Branch with "Require review from Code Owners" will automaticaly trigger a request to a designated code owner
#
# Note:
# - Each line is a file pattern followed by one or more owners.
# - Branch with branch protection rule "Require review from Code Owners"
# will automaticaly trigger a request to a designated code owner
# =================================================================================================

# DevOps related
/build_system/ @RedLeader962
/.github/ @RedLeader962
/.dockerignore @RedLeader962
/.gitignore @RedLeader962
# ....Repository wide code owner...................................................................
# These owners will be the default owners for everything in the repo.
* @pomerlef

# Core
# ....Core directories and files...................................................................
/README.md @pomerlef
/LICENSE @pomerlef
/pointmatcher @boxanm @simonpierredeschenes @aguenette
/python @boxanm @simonpierredeschenes @aguenette
/utest @boxanm @simonpierredeschenes @aguenette
/.env.libpointmatcher @RedLeader962

# ....DevOps related...............................................................................
/.github/ @RedLeader962
/build_system/ @RedLeader962
/build_system/utilities @RedLeader962
/.gitmodules @RedLeader962
/.gitignore @RedLeader962
/.dockerignore @RedLeader962

47 changes: 47 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Description
### Summary:






### Changes and type of changes:

-
-
-


---

# Checklist:

### Code related
- [ ] I have made corresponding changes to the documentation (i.e.: function, class, script header,
README.md)
- [ ] I have commented hard-to-understand code
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] All tests pass locally with my changes (Check [README.md #Contributing](https://github.com/norlab-ulaval/libpointmatcher/tree/release#contributing) for local testing procedure using _libpointmatcher-build-system_)

### PR creation related
- [ ] My pull request `base ref` branch is set to the `develop` branch (the _build-system_ won't be triggered otherwise)
- [ ] My pull request branch is up-to-date with the `develop` branch (the _build-system_ will reject it otherwise)

### PR description related
- [ ] I have included a quick summary of the changes
- [ ] I have included a high-level list of changes and their corresponding type
- Types: `feat` `fix` `docs` `style` `refactor` `perf` `test` `build` `ci` `chore` `revert`
- Breaking changes: `<type>!`
- Reference:
- See [commit_msg_reference.md](https://github.com/norlab-ulaval/libpointmatcher/blob/release/commit_msg_reference.md) in the repository root for details
- https://www.conventionalcommits.org
- [ ] I have indicated the related issue's id with `# <issue-id>` if changes are of type `fix`

## Note for repository admins
### Release PR related
- Only repository admins have the privilege to `push/merge` on the default branch (ie: `master`) and the `release` branch.
- Keep PR in `draft` mode until all the release reviewers are ready to push the release.
- Once a PR from `release` -> `master` branch is created (not in draft mode),
- it triggers the _build-system_ test
- (in-progress) and it triggers the _semantic release automation_
2 changes: 1 addition & 1 deletion CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Changelog for package libpointmatcher
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

2.0.0 (2023-12-15)
1.4.0 (2023-12-15)
-----------------
* fix: N2ST path resolution in dependencies-doc docker image
* refactor: move libpointmatcher build-system logic to norlab-build-system submodule
Expand Down
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
BSD 3-Clause License

Copyright (c) 2010-2021, the libpointmatcher authors
Copyright (c) 2010-2023, the libpointmatcher authors
All rights reserved.

Redistribution and use in source and binary forms, with or without
Expand Down
78 changes: 53 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,37 @@


---
`libpointmatcher` is currently tested on our build system with the following architecture and OS


`libpointmatcher` is tested on our build system under the following architecture and OS:
- x86 and arm64/v8
- Ubuntu bionic (18.04) and focal (20.04)
- Ubuntu bionic (18.04) and focal (20.04)

Note:
- support for Ubuntu jammy (22.04) comming soon
- `libpointmatcher` reportedly works on MacOs OsX (latest) and Windows (latest)

<div align="center">
<img src="https://img.shields.io/static/v1?label=powered by JetBrains TeamCity&message=CI/CD&color=green?style=plastic&logo=teamcity" />
</div>

<br>
<br>

<div style="padding: 20px;padding-top: 5px;padding-bottom: 5px; margin-top:20px;margin-bottom:20px; border: 2px solid LightGray; border-radius:10px;">

### ★ Version `1.4.0` release note (important)
This release of _libpointmatcher_ introduces the integration of [norlab-build-system (NBS)](https://github.com/norlab-ulaval/norlab-build-system) as a _git submodule_ for codebase development and testing.

Execute the following to clone the repository with its submodule:
```shell
git clone --recurse-submodules https://github.com/norlab-ulaval/libpointmatcher.git
```
If _libpointmatcher_ was previously cloned, execute the following to fetch its new submodule
```shell
git submodule update --remote --recursive --init
```
</div>

# Documentation and Tutorials

Expand All @@ -23,17 +51,10 @@ It is now maintained by the Northern Robotics Laboratory ([Norlab](https://norla

You can read the latest changes in the [release notes](doc/ReleaseNotes.md).

# Quick Start

Clone the repository with its submodule
```shell
git clone --recurse-submodules https://github.com/norlab-ulaval/libpointmatcher.git

# If libpointmatcher is already cloned, fetch its new submodule
git submodule update --remote --recursive --init
```


# Quick Start
Although we suggest to use the [tutorials](doc/index.md), here is a quick version of it:

The library has a light dependency list:
Expand Down Expand Up @@ -95,18 +116,22 @@ described [here](doc/ICPIntro.md) in more detail.

Please use our [github's issue tracker](http://github.com/ethz-asl/libpointmatcher/issues) to report bugs. If you are running the library on Ubuntu, copy-paste the output of the script [listVersionsUbuntu.sh](https://github.com/norlab-ulaval/libpointmatcher/blob/master/utest/listVersionsUbuntu.sh) to simplify the search of an answer.

## Codebase
## Codebase development

Libpointmatcher codebase now integrate [norlab-build-system (NBS)](https://github.com/norlab-ulaval/norlab-build-system) and [norlab-shell-script-tools (N2ST)](https://github.com/norlab-ulaval/norlab-shell-script-tools).
`NBS` is a build-infrastructure-agnostic build system custom-made to meet our needs in robotic software engineering at NorLab and `N2ST` is a library of shell script functions as well as a shell testing tools leveraging _**bats-core**_ and _**docker**_ .
`N2ST` purpose is to speed up shell script development and improve reliability.

Libpointmatcher codebase now integrate [norlab-build-system (NBS)](https://github.com/norlab-ulaval/norlab-build-system) and [norlab-shell-script-tools (N2ST)](https://github.com/norlab-ulaval/norlab-shell-script-tools). `NBS` is a build-infrastructure-agnostic build system custom-made for our need at NorLab and `N2ST` is library of shell script, function for shell script development and a shell testing tools leveraging `bats-core`.
`NBS` is deployed on our [TeamCity](https://www.jetbrains.com/teamcity/) continuous integration/deployment server and oversees protected branches of the [libpointmatcher](https://github.com/norlab-ulaval/libpointmatcher) GitHub repository:

NBS is deployed on our [TeamCity](https://www.jetbrains.com/teamcity/) continuous integration/deployment server and oversees both protected branch of the [libpointmatcher](https://github.com/norlab-ulaval/libpointmatcher) GitHub repository: the `master` branch and the `develop` branch.
- The `develop` branch can only be merged through a pull-request from any `<feature>` branches. Any contributor can submit a pull request to the `develop` branch;
- the `release` branch is a revision and preparation branch where we can freeze the codebase in a given state without stalling to `develop` branch progression;
- The `master` branch can only be merged through a pull-request from the `release` branch. Only repository admin can submit a PR to the `master` branch.

- The `develop` branch can only be merged through a pull-request from any `<feature>` branchs. Any contributor can submit a pull request to the `develop` branch;
- The `master` branch can only be merged from the `release` branch through a pull-request by a repository admin.
In both cases submiting a pull request will trigger a build configuration on our build system and the pull request will be granted if the build/test run succeede.
In any cases, submitting a pull request to `develop` or `master` will trigger a build/test configuration on our build system and the pull request will be granted if the build/test run succeed.

**Current build matrix:**
`[latest] x [x86, arm64] x [ubuntu] x [bionic, focal, jammy] x [Release, RelWithDebInfo, MinSizeRel]`
`[latest] x [x86, arm64] x [ubuntu] x [bionic, focal] x [Release, RelWithDebInfo, MinSizeRel]`

### Development workflow

Expand All @@ -133,16 +158,19 @@ bash lpm_create_multiarch_docker_builder.bash
```shell
cd <path/to/libpointmatcher>/build_system

# Run the build matrix as specified in ".env.build_matrix.libpointmatcher" on native architecture using "ci_PR" service
# Run the build matrix as specified in ".env.build_matrix.libpointmatcher"
# on native architecture using "ci_PR" service
bash lpm_crawl_libpointmatcher_build_matrix.bash --fail-fast -- build ci_PR

# Run a specific case using build flags with multi-architecture virtualization using "ci_PR_amd64" and "ci_PR_arm64v8" services
bash lpm_crawl_libpointmatcher_build_matrix.bash --repository-version-build-matrix-override latest \
--os-name-build-matrix-override ubuntu \
--cmake-build-type-build-matrix-override RelWithDebInfo \
--ubuntu-version-build-matrix-override focal \
--fail-fast \
-- build ci_PR_amd64 ci_PR_arm64v8
# Run a specific case using build flags with multi-architecture
# virtualization using "ci_PR_amd64" and "ci_PR_arm64v8" services
bash lpm_crawl_libpointmatcher_build_matrix.bash \
--repository-version-build-matrix-override latest \
--os-name-build-matrix-override ubuntu \
--cmake-build-type-build-matrix-override RelWithDebInfo \
--ubuntu-version-build-matrix-override focal \
--fail-fast \
-- build ci_PR_amd64 ci_PR_arm64v8

# Read the help for details
bash lpm_crawl_libpointmatcher_build_matrix.bash --help
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="(HELP) bash lpm_execute_compose.* › --help menu" type="ShConfigurationType" folderName="[LPM] B | dry-run">
<option name="SCRIPT_TEXT" value="bash lpm_crawl_libpointmatcher_build_matrix.bash --help &amp;&amp; bash nbs_execute_compose.bash --help" />
<configuration default="false" name="(HELP) bash lpm_crawl.* › --help menu" type="ShConfigurationType" folderName="[LPM] B | dry-run">
<option name="SCRIPT_TEXT" value="bash lpm_crawl_libpointmatcher_build_matrix.bash --help" />
<option name="INDEPENDENT_SCRIPT_PATH" value="true" />
<option name="SCRIPT_PATH" value="" />
<option name="SCRIPT_OPTIONS" value="" />
Expand Down

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="bash lpm_crawl_libpointmatcher_build_matrix.bash › build &lt;service&gt; &lt;popup&gt;" type="ShConfigurationType" folderName="[LPM] A | Execute compose over build matrix" editBeforeRun="true">
<configuration default="false" name="bash lpm_crawl_libpointmatcher_build_matrix.bash › build &lt;service&gt; &lt;popup&gt;" type="ShConfigurationType" folderName="[LPM] C | Execute compose over build matrix" editBeforeRun="true">
<option name="SCRIPT_TEXT" value="bash lpm_crawl_libpointmatcher_build_matrix.bash --repository-version-build-matrix-override latest --os-name-build-matrix-override ubuntu --fail-fast -- build ci_PR_arm64v8" />
<option name="INDEPENDENT_SCRIPT_PATH" value="true" />
<option name="SCRIPT_PATH" value="" />
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="bash lpm_crawl_libpointmatcher_build_matrix.bash › build ci_PR_arm64v8 &lt;popup&gt;" type="ShConfigurationType" folderName="[LPM] A | Execute compose over build matrix" editBeforeRun="true">
<configuration default="false" name="bash lpm_crawl_libpointmatcher_build_matrix.bash › build ci_PR_arm64v8 &lt;popup&gt;" type="ShConfigurationType" folderName="[LPM] C | Execute compose over build matrix" editBeforeRun="true">
<option name="SCRIPT_TEXT" value="bash lpm_crawl_libpointmatcher_build_matrix.bash --repository-version-build-matrix-override latest --os-name-build-matrix-override ubuntu --cmake-build-type-build-matrix-override RelWithDebInfo --ubuntu-version-build-matrix-override focal --fail-fast -- build ci_PR_arm64v8" />
<option name="INDEPENDENT_SCRIPT_PATH" value="true" />
<option name="SCRIPT_PATH" value="" />
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="bash lpm_crawl_dependencies_build_matrix.bash › build dependencies-general dependencies (BUILD MATRIX SUBSET)" type="ShConfigurationType" folderName="[LPM] A | Execute compose over build matrix">
<configuration default="false" name="bash lpm_crawl_dependencies_build_matrix.bash › build dependencies-general dependencies (BUILD MATRIX SUBSET)" type="ShConfigurationType" folderName="[LPM] C | Execute compose over build matrix">
<option name="SCRIPT_TEXT" value="bash lpm_crawl_dependencies_build_matrix.bash --fail-fast --cmake-build-type-build-matrix-override None --os-name-build-matrix-override ubuntu --ubuntu-version-build-matrix-override focal -- build dependencies-general dependencies dependencies-doc" />
<option name="INDEPENDENT_SCRIPT_PATH" value="true" />
<option name="SCRIPT_PATH" value="" />
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="bash lpm_crawl_libpointmatcher_build_matrix.bash › build test_compilation_* (compilation test)" type="ShConfigurationType" folderName="[LPM] A | Execute compose over build matrix">
<configuration default="false" name="bash lpm_crawl_libpointmatcher_build_matrix.bash › build test_compilation_* (compilation test)" type="ShConfigurationType" folderName="[LPM] C | Execute compose over build matrix">
<option name="SCRIPT_TEXT" value="bash lpm_crawl_libpointmatcher_build_matrix.bash --fail-fast --cmake-build-type-build-matrix-override Release --os-name-build-matrix-override ubuntu -- build test_compilation_prefixed_path test_compilation_auto_path_resolution" />
<option name="INDEPENDENT_SCRIPT_PATH" value="true" />
<option name="SCRIPT_PATH" value="" />
Expand Down
Loading