Vox Pupuli organization

[ekohl]

Today Vox Pupuli has some organization in terms of teams, but I think it's not all well defined and/or documented. I also have concerns about scalability, especially if we end up adopting the community maintenance of Puppet itself (voxpupuli/plumbing#294). This is meant to start a discussion how we should organize ourselves in the future.

Where are we today?

Starting with the good parts, the PMC is clearly defined: https://voxpupuli.org/elections/

Then we have various GitHub teams. I explicitly didn't use @ to avoid mentioning the entire org.

• collaborators grants permissions on most repositories
• modules I just created this as a parent for specific module sets
  • php Already existed as modules/php. I've renamed it and put it under modules
• Project-Maintainers The organization admins
• teams A parent team to group organization specific teams. For example, there is a team for theforeman which grants permissions on all repositories the Foreman project needs. There are others here. The idea is to make it easier for companies to focus on a specific set of repositories they use without granting permissions on everything (avoiding notification overload)
• tools Much like the teams team, a parent for tooling specific groups. I'll name them all since there's less of a privacy concern here and the names should be fairly self-explanatory.
  • beaker
  • containerimages
  • github-action
  • librarian
  • lint
  • onceover
  • python
  • ruby-json-schema
  • vox-pupuli-tasks
• tools/$tool/admin Specific admin groups for tools. Can't be nested because they would grant permissions on all members

Where do I think we should go to

Regarding the tools/$tool/admin groups, I doubt there's benefit to it. We should be able to grant maintainer permissions on tools/$tool and I'd think that's sufficient. If you want more specific modifications, you can contact the Project-Maintainers group.

Then I'd like to drop the collaborators group and instead assign people to more specific groups. Either modules, tools or a specific tool. Considering today's world of supply chain attacks, I'd like to lazily add people instead of blindly give them permissions. Right now collaborators has 170 members and grants permissions on 267 repositories (some archived) and a lot of people have moved on from the Puppet ecosystem.

Speaking of removing permissions, this is less concrete, but perhaps there should be something where you need to be active to keep your permissions. Here's where security and being open collide. On the one hand, we want to be welcoming to people but that also poses a security threat.

I also think a security team is missing. Today we have listed Julien on our security page but I don't think this scales well and a security team should help improve response times. Again, if we become even more crucial by adopting the Puppet fork there will be an increased demand. I'd also like them to take a critical look at the security procedures, for example around releasing modules, gems, Python packages etc. There should be a transparent process defined (like we have for the PMC; perhaps not exactly the same process, but written out) on membership and responsibilities.

Then related to the security team, I think we should have an infra team. You could argue today's Project Maintainers group has this role, but there's no clearly defined process around membership and responsibilities.

I hope this was more than just me rambling but at the very least I'd like to spark a discussion.

[wyardley]

I had been thinking about this recently as well, especially after the article about the Puppet supply chain attack, and had written (or at least inteded to write) with some similar ideas a little while back on Slack / IRC. As you say, there are currently a lot of collaborators. When was the last time we reviewed that list and confirmed if all those folks still want to be part of / are active? I would imagine a lot of folks who were members at one point are not heavily involved with Puppet related stuff these days. For example, I haven't actively used Puppet since about 2017 or so. I'd say it's worth looking at:

• Rotate any tokens / credentials we use for publishing to the Forge and / or interacting with GitHub (and set some kind of schedule for doing this on an ongoing basis in any cases where we must use explicit credentials vs. something like OIDC). Even if we don't believe they've been compromised, this seems like a good idea overall.
• Are there collaborators who could / should be removed? It might seem awkward or weird to have a recertification process (maybe a form with some kind of deadline to request continued access), but I think it might be good to err on the side of removing most people, and work on adding people back as needed.
• If there are any shared / service accounts that require 2 factor auth, look at using 1password or a similar tool.
• In the case of GitHub
• If someone only needs to help with one or two modules (for example, the original author of a module), there might be a case for setting up some repo level permissions, possibly with some kind of central tracking / management, vs. having them in a team that has permissions on all repos
• Are there modules which Vox can't safely maintain with the number of people involved total
• In the case where modules are archived / deprecated, is there a process to unpublish them from the Forge after some amount of delay?
• Look at teams and see if all of the existing ones are needed (do we still run Icinga?)

Also, if there hasn't already, as you also mention, having a review of CI setups and tooling to preemptively check for things which could lead to a supply chain attack would be a good idea.

I feel like at one point there was a security team? There is the security managers group in GH, and I want to say @smortex was also involved with security at one point? I totally think all of this is a great idea. But, given that Puppet, and traditional config management in general, is less of a thing these days, and, on top of that, there may be some additional fragmentation of the community over time, or organizations continuing to shift away from using Puppet, with the licensing changes by Perforce, I am a little skeptical that there will be enough people in a volunteer ecosystem to take on these kinds of tasks in such a formal way.

It's possible that the organization trying to fork Puppet itself might be able to help -- my concern is that even the OpenTofu fork, which has quite a few companies sponsoring its development, doesn't have large amounts of staffing, and I'm concerned that something focused on Puppet would have even less funding / involvement.

IMO, it could be smart to also try and figure out ways to safely retire / deprecate some of the large number of modules that Vox maintains. That may be a little bit of a challenge to line up with our mission, but even with all the automation and tools like modulesync, there are a lot of challenges / issues with modules that aren't actively maintained.

[genebean]

FWIW, 1Password is free for FOSS teams. I helped get this setup for SCaLE not long ago.

[ekohl]

If there are any shared / service accounts that require 2 factor auth, look at using 1password or a similar tool.

Today it's stored in https://github.com/voxpupuli/plumbing/ as a single GPG encrypted file. I have thought about changing this. In the Foreman project we use gopass with private git repositories (one for infra, one for releases). I'm very pleased with it.

Are there modules which Vox can't safely maintain with the number of people involved total

There are some modules which are very popular (puppet/systemd comes to mind) making them a bigger target so I'd say yes.

In the case where modules are archived / deprecated, is there a process to unpublish them from the Forge after some amount of delay?

There is a process defined: https://voxpupuli.org/docs/deprecated_and_archived_modules/

Look at teams and see if all of the existing ones are needed (do we still run Icinga?)

We don't run Icinga and I'm not sure if we ever did, but recently the icinga team transferred their modules. They still maintain them, but wanted to use Vox Pupuli as an umbrella organization to benefit from easier collaboration. Perhaps you're confusing that?

It's possible that the organization trying to fork Puppet itself might be able to help -- my concern is that even the OpenTofu fork, which has quite a few companies sponsoring its development, doesn't have large amounts of staffing, and I'm concerned that something focused on Puppet would have even less funding / involvement.

I don't think you need a huge staff, but in my experience clearly defining processes and responsibilities helps to make it easier for a smaller staff to execute them while also being more transparent to the community.

[genebean]

I think the structure you have outlined makes a lot of sense. Assuming Vox adopts the puppet code / puppet fork, I would advocate for the lead(s) of any future security team to be people employed with this as a focus of their job by relevant companies in the community so as to ensure timely focus on the needs of such a team. Company-wise, this could easily encompass Overlook InfraTech (@binford2k's company), betadots GmbH (@tuxmea's company), or any of the others whose names are not coming to mind at this moment. In no way am I saying only these paid people should be on security, just that ideally the lead(s) has it as part of their job instead of as a side project.

[wyardley]

If there are any shared / service accounts that require 2 factor auth, look at using 1password or a similar tool.

Today it's stored in https://github.com/voxpupuli/plumbing/ as a single GPG encrypted file.

Right, I'm aware of the credentials file. I was mostly thinking around the 2 factor auth issue specifically, though agree there's probably room for improvement.

Perhaps you're confusing that?

Yes, I didn't look at the team description too carefully, but I can see that now. Having a team at org level even for a niche case like this has some advantages vs. just marking people as external collaborators on a couple of repos, maybe we could do some nested teams to make the team structure a little less messy.

In the case where modules are archived / deprecated, is there a process to unpublish them from the Forge after some amount of delay?

There is a process defined

Right, there's a process for doing it and getting buy-in... I meant more, is there a top level process for being a bit more proactive about figuring out which modules are a) actively in use and b) actively maintained, and trying to cull some of the ones that are neither. I could be wrong (I haven't been paying too much attention), but it seems to me that we're quick to add modules, and quick to deprecate supported versions of OSes / Puppet itself, but not always very active in actually retiring modules.

The fact that we have the special "puppet" namespace is also one reason we might be a target for supply chain attacks. It will be interesting if we keep that namespace if Vox does get involved in whatever fork may be planned.

[smortex]

All this make sense.

Speaking of removing permissions, this is less concrete, but perhaps there should be something where you need to be active to keep your permissions.

Safekeeping permissions of users after a period of inactivity sounds the right thing to do.

I feel like at one point there was a security team?

As spotted the security info is inconsistent across different places, I guess they should all point to a single place to limit duplication and inconsistencies (RFC9116 suggest /.well-known/security.txt for this purpose; https://securitytxt.org/ for the TL;DR). Having a team composed of multiple persons looks indeed necessary.