WIP: Document using pantsbuild

[cognifloyd]

Background

This is part of introducing pants, as discussed in the TSC Meetings on 12 July 2022 and 02 Aug 2022. Pants has fine-grained per-file caching of results for lint, fmt (like black), test, etc. It also has lockfiles that work well for monorepos that have multiple python packages. With these lockfiles CI should not break when any of our dependencies or our transitive dependencies release new versions, because CI will continue to use the locked version until we explicitly relock with updates.

Overview of this PR

As we replace the Makefile and other parts of our CI and development infrastructure, the development and contributing guidelines need to be updated. As we integrate pantsbuild in the main st2 repo, we can document changed processes in the pantsbuild branch (this PR). Any of the TSC members can push to this branch, so we can collaboratively update the documentation.

PRs in StackStorm/st2

• Initialize pantsbuild st2#5713
• Configure pantsbuild source roots st2#5724
• Enable pantsbuild language backends st2#5725

[cognifloyd]

I've looked at the developer docs, and it's really hard to make sense of what should be where. What follows is a bit of a braindump about how to proceed.

So, I'm taking a step back to try and use the Diátaxis framework evaluate what we might need*.

Diátaxis: A systematic framework for technical documentation authoring.

There are four kinds of documentation in Diátaxis: Tutorials, How-to Guides, Reference, Explanation. Looking specifically at the Development branch of our documentation--the section that targets StackStorm contributors and TSC members instead of StackStorm users--we have all four of these on the same page. Mapping out exactly where everything should be feels a bit overwhelming, but apparently we don't need to (see: How to Use Diátaxis):

Although structure is key to documentation, using Diátaxis means not spending energy trying to get its structure correct.
...
In practice, this means that getting started with Diátaxis doesn’t require thinking about dividing up your documentation into the four sections, or writing out headings to put material under.
...
Diátaxis strongly prescribes a structure, but whatever the state of your existing documentation - even if it’s a complete mess by any standards - it’s always possible to improve it, iteratively.
...
Although Diátaxis is intended to provide a big picture of documentation, don’t try to work on the big picture. It’s both unnecessary and unhelpful. Diátaxis is designed to guide small steps; keep taking small steps to arrive where you want to go.

If you’re tidying up a huge mess, the temptation is to tear it all down and start again. Again, avoid it. ...

We have docs that serve multiple purposes mixed together in the dev docs, and we should be able to take iterative steps towards reorganizing it. We also need to document how to do various things with pants, because that should simplify the dev processes and make them more discoverable (w/ ./pants help). I expect to replace a lot of the existing dev docs

So, as I write new documentation about pants, I should add it in a new page, referring to Diátaxis to figure out which purpose that page should serve and where it should go. Then, figure out if that any existing docs can be moved into that page, or it the page replaces any docs sections so it can be removed or updated to refer to the new page.

*: I forget where I first heard about this framework, but I watched this talk before I found the website: https://youtu.be/t4vKPhjcMZg

[arm4b]

I think we'll need a small section somewhere after Pre commit hook

[arm4b]

Just having a couple of commands hinting how to build things with pants for someone residing in the stackstorm repo would be sufficient or perhaps even a more high-level overview of the 2-3 steps that pants does referring to their official docs.

There is no need of a big write-up and explanation and we should stick with the format/size that's already used, like an existing "Code Style Guide".