Test/improve deploy and CI methods

[mayel]

No description provided.

[fishinthecalculator]

Hi I'd love to help with this. I have expertise on Linux systems and releasing free software products. Depending on the requirements and priorities some things that, off the top of my head, could use some love (imo) are:

• I love that flexibility allowed in setting up a development environment or in the deploy of Bonfire (and I will probably contribute to it if there's interest with this Guix System service I implemented. It kinda works in my experience). Nonetheless I believe there should be one "blessed" supported and tested way of running both development and production instances of Bonfire (unless there's bandwidth to take care, maintain and test all of the current 4 ways of setting up environments).
• with this in place a CI running automated tests the "blessed" way for each push on main could be implemented to track and prevent regressions.
• Implementing an automated, reproducible and documented release workflow is pretty useful. In my opinion it should at least build bonfire, run automated tests if possible, change the version in mix.exs , tag the current commit and support dry run executions . here you can see something similar I implemented for a Python project.
• with that in place the release CI could be standardized to run only for tags (or pushs to a given release/vX.Y.Z branch, or whatever this can be decided later) , run the usual test suite and maybe in the future even an additional more extensive test suite, and proceed with the current docker build

probably you have already some solutions and opinions on the above topics, I'd love to hear your thoughts on this.

At last I'm not sure if any of you will attend Reclaim The Tech this May in Bologna. I'm based there and I was pondering the idea of bringing a small talk/workshop on how to build a Bonfire extension. I still don't know Elixir very well so I'm not sure if I will be able to, but if you are interested in my proposal above we could setup a meeting (I believe also @chobeat may be interested) where we define better the needs of Bonfire in terms of CI and release workflow and you show me how you currently build and run Bonfire on your environment :)

WDYT?

[ivanminutillo]

Hey @fishinthecalculator this sounds really great! totally up to dicuss it further over a call.
We will be in bologna and would love to help you with your workshop, hopefully we will collect data to improve / extend our tutorials.

[fishinthecalculator]

Awesome. I'm sending you a poll via email @ivanminutillo and @mayel . thank you for your help!

[mayel]

Nonetheless I believe there should be one "blessed" supported and tested way of running both development and production instances of Bonfire.

Of course this is up for discussion but I'd say our blessed ways so far are:

• For dev: Docker "partial" mode, meaning the DB and any other bg services run in Docker but Elixir/OTP/JS tooling all run outside of docker since that gives a nice balance between ease of setup and efficiency.
• For prod: Coop-cloud (a wrapper around Docker Swarm which while it requires initial buy-in / setup makes subsequent deploys/upgrades/backups etc much easier)

[mayel]

CI running automated tests the "blessed" way for each push on main could be implemented to track and prevent regressions.

There's an initial test workflow here: https://github.com/bonfire-networks/bonfire-app/blob/main/.github/workflows/test.yaml

Implementing an automated, reproducible and documented release workflow is pretty useful. In my opinion it should at least build bonfire, run automated tests if possible, change the version in mix.exs , tag the current commit and support dry run executions

There's just contrib-release to

• prompt to commit/push any pending changes
• update extensions (ie. update lockfiles)
• change the version in mix.exs
• push that to trigger CI

There's then https://github.com/bonfire-networks/bonfire-app/blob/main/.github/workflows/release.yaml which builds docker images for different flavours (only for amd64 for now since there others were too slow, but we should try adding arm64 again so it can produce a multiarch image).

We can also build a release locally instead using just rel-build-release (useful for building on non-amd64 arch) and can publish that image with just rel-push.

Additionally there's https://github.com/bonfire-networks/bonfire-app/blob/main/.github/workflows/docs.yaml which builds docs and publishes them to https://doc.bonfirenetworks.org/readme.html

[mayel]

Anyway, I look forward to chatting next week, just thought it'd be useful to see what we have and where the gaps or room for improvement are beforehand :)

[fishinthecalculator]

Hi @mayel @ivanminutillo

I'd like your opinion on this: CI for the main branch is worth insofar as its breakage is rare or at least is solved quickly and all changes to the main branch should require a MR (or equivalent, like a branch) with green CI. this is not really urgent right now but I think it is pretty important for a stable 1.0 software, so starting setting up some process in this sense could be helpful.

I'm reasonably sure that the CI in #856 is working, but right now on main there's some dependency conflict so it would still be red upon merge. I think to prevent failures we should at least:

• lock extension dependencies in the lockfile. i'm still learning mix but i believe pushing a breaking commit on an extension repo shouldn't break the main build. i'm still not sure this is the case and how extension dependencies are referenced as they are not listed like other dependencies in the mix.exs but I'd like to fix this if you give me some pointers.
• test should work . I was able to try to run them on a commit from last week but they appeared not to work or I did some error . anyway we should be able to disable broken ones. I think make extension tests runnable (eg. in CI) individually, outside of the top level app #851 is the relevant issue, I'd love to help with this also.

how can we stabilize the main branch (and the extensions repositories) development workflow so that CI "never" breaks? can I help?

my only experience is with a branching policy that has develop/main branches with feature branches going into develop and releases done on main after merging a stable develop commit (the develop name is just an example). maybe the whole develop/main circus is too much bureaucracy, and feature branches against main are sufficient? what do you think? one this is figured out I can proceed with the release CI i believe

[mayel]

I've been thinking about this too since our chat, and while I don't have any definitive answers, as you've seen with #851 it seems like ideally tests should run within each extension as much as possible, so the main app repo prioritise running extension integration tests (ones that depend on several extensions working together), maybe only running the full suite before a release. This means having a CI workflow in each extension, but first I need to get tests to actually run within extension (finishing #851)...

Speaking of releases, we should start using semantic versioning not only on the app but also for each extension, so dependencies can be specified with a version number rather than pointing to main branch. Ideally we'd publish the extensions to https://hex.pm as well and only use git for development.

Anyway, more to think about and discuss...

[mayel]

To answer your question about where extensions are specified:

• each flavour defines its deps:
  • from hex.pm https://github.com/bonfire-networks/bonfire-app/blob/main/flavours/classic/config/deps.flavour.hex
  • from git: https://github.com/bonfire-networks/bonfire-app/blob/main/flavours/classic/config/deps.flavour.git
• bonfire_spark repo specifies ones that are shared by all extensions:
  • from hex: https://github.com/bonfire-networks/bonfire_spark/blob/main/deps.hex
  • from git: https://github.com/bonfire-networks/bonfire_spark/blob/main/deps.git
• each extension can also specify deps, for example:
  • https://github.com/bonfire-networks/bonfire_common/blob/main/deps.hex
  • https://github.com/bonfire-networks/bonfire_common/blob/main/deps.git

[mayel]

@fishinthecalculator alright, it required some tweaks (on the elixir and justfile sides) but tests are now running :)

[fishinthecalculator]

cool thanks. I'll look into the release pipeline :)

[mayel]

@fishinthecalculator alright closed #851 meaning tests are now runnable in isolated fashion in the https://github.com/bonfire-networks/bonfire_ui_me extension, so you could use that one for prototyping and then we can replicate to the others

[mayel]

See bonfire-networks/bonfire_fail#1 (comment) for a discussion of our branching/commit/PR workflow...

[mayel]

@fishinthecalculator following that discussion, I have translated jungle into elixir (for fetching all cloned extensions/forks in parallel and same for messctl is in progress

[mayel]

Alright, still needs some more testing, but here's Mess.Manager with which you can install hex/git deps or clone git ones to use locally: bonfire-networks/mess@a60a4fb#diff-dfa6f4ed74c90e5d4fda283d547d366586e690387289bcfd473e3fa5f9ace308

[mayel]

@fishinthecalculator @ivanminutillo here's a first attempt at auto-branching-and-PR, but would love to have a higher level discussion about ideal workflows first: https://github.com/bonfire-networks/bonfire-app/pull/1071/files (yes, that branch and PR where created by the script in the PR)

[mayel]

After discussing with @ivanminutillo here's a workflow that seems to balance the pros/cons for us:

• Each repo having a main branch and stable branch
• Contributors make PRs on the main branch (on which CI tests run before they are merged)
• Maintainers can push directly to main (on which CI tests run too)
• We write a dedicated script to prepare a release, which makes a PR to each repo which has changes in main not present in stable (maybe with an option to specify the last commit to include, skipping any changes made after it), upon which CI tests run on each extension + on the parent app before the release is published (to stable branch and hex.pm)
• Releases use the stable branch (or last hex version) of extensions
• Devs should have a way to toggle all extensions between stable and main (or could be between git deps and hex deps)

[chobeat]

this sounds quite close to established practices and a huge step forward compared to the current structure so 👍 for me.

How would the last item look like in practice?

[mayel]

Well ideally the release script (or CI) would also increment the version and publish to hex once tests pass, so actually releases would use that instead of the stable branch, which thanks to Mess would be quite easy to toggle

[fishinthecalculator]

Hi @mayel @ivanminutillo @chobeat , apologies for the delay. I agree on the main/stable branching model, thank you for your help!

[fishinthecalculator]

Also what would be the next steps? Can I help somehow? Otherwise I plan to add CI to the following extensions in this order:

1. bonfire-networks/bonfire_ui_common
2. bonfire-networks/bonfire_ui_me
3. bonfire-networks/bonfire

[mayel]

Well, haven't given it much thought, but in terms of which extensions would benefit first, it's worth knowing that the UI ones have the most failing tests (some from changes in features/UX where tests haven't been updated, some from changes in HTML/CSS which points to fragile tests, some from bugs) whereas the backend extensions (like bonfire_me, bonfire_social, bonfire_posts, bonfire_social_graph, etc) are more likely to have little to no broken tests

[ivanminutillo]

hey, wonderful improvements :) imo this could be a good moment to start adding CI to UI extensions, now that features and UX are become stable, in particular :

• bonfire-networks/bonfire_ui_boundaries
• bonfire-networks/bonfire_ui_me
• bonfire-networks/bonfire_ui_social

[fishinthecalculator]

Hi @mayel @ivanminutillo , let me recap the requirements so that you can help me prioritize them.

The need is to have development supported through process improvements, CI and automation scripts directed to the goal of stabilizing the Bonfire umbrella app and extension codebases. We'll need to have some kind of measure of success to define when this task can be considered complete, it can be a minimal set of required repositories that have to be covered and a set of nice to haves or something else entirely. Let me know your thoughts about this, thanks.

Bonfire development model

Below I'll describe the development model proposed by @mayel some comments above with some changes I'm proposing. Once we reach consensus this will probably need to be documented somewhere.

• Each repo has a main branch and stable branch
• CI is run for pushes and PRs to both the main and stable branches. (For this to make sense CI for PRs should only detect the diff of failures, until we get there additional care must be put in reviews)
• Contributors make PRs on the main branch, Maintainers can push directly to main
• Releases are git tags. They depend only on other releases of extensions (there can exceptions) and require green CI to be made
• Devs should have a way to toggle all extensions between stable and main (or could be between git deps and hex deps)
• We write a dedicated script to prepare a release, which makes a PR to each repo which has changes in main not present in stable (maybe with an option to specify the last commit to include, skipping any changes made after it)
• "Stabilization PRs" live in their own branch, based on main and targeting stable. Extensions will require additional commits before being ready for the stable branch, as building inside an umbrella app gives different results than building on their own. Once the PR CI is green, changes can be brought from main to stable and a new release can be executed (by git tagging) on the stable branch.
• A release for the umbrella app (i.e. a Bonfire flavor) requires released versions for all extensions (not necessarily from hex.pm). This should ensure that only tested extension releases are included in the new flavor app release.
• hex.pm publishing can be automated but should not happen on Github infrastructure imho. Either someone from the development team runs the automation from their machine or a trusted building environment should be setup somehow that publishes releases to hex.pm. we can definitely discuss this further as I don't have an easy solution in mind.

Priority of the extensions

An extension is considered covered once it has CI running on the main branch, and a stable branch with a new release on it with CI enabled as well that follows the development model that we agree upon. This is a proposed sorting of the extensions you mentioned in this issue, if you don't have concerns I think I'll start from bonfire_ui_common as I have already started working on that. Also in some cases it could be easier to cover dependencies of a given extension, before covering the extension itself, so it may happen that the final order will be partly determined by technical needs.

1. bonfire_ui_common
2. bonfire_ui_boundaries
3. bonfire_ui_me
4. bonfire_ui_social
5. bonfire_me
6. bonfire_social
7. bonfire_posts
8. bonfire_social_graph

Do you have a list of extensions that must be covered before 1.0? What do you think about this proposal? All of the above is obviously doomed to be exposed to change if some technical issue arises but I think it's a good start. Definitely let me know if I need to elaborate further. Thank you

[mayel]

I think we've got a good approach, and a working structure https://github.com/bonfire-networks/bonfire-extension-boilerplate and its justfile that can update itself and other files in extension repos (such as CI scripts). Closing this issue and we can follow it up by more specific ones as needs arise.