Dependency cycle in tag-driven CI/CD

[haraldatbmw]

I just came up with a dependency cycle which I want to discuss here.

1. In my app-repo I store both my app sources and my k8s manifests with kustomization overlays.
2. A commit on my app-repo master branch triggers a build which ends with a new docker-image (tagged with build-number, e.g. 1.0.0-build25) in my docker-registry.
3. Flux image-reflector-controller and image-automation-controller detect the new image-tag in the registry and updates+pushes my kustomization.yaml with the new image-tag which again trigges my build-pipeline... (2.)

So I created an endless loop by accident.

To fix this I need to move all k8s manifests with kustomization overlays to my git-ops repo where no Jenkins build-pipeline is triggered. So this should work.

Are there any other options to bypass such a situation?

[stefanprodan]

Most CI tools will ignore commits with [skip ci] as the prefix of commit message, if Jenkins doesn't do that, I think it's easy to add it.

I think you can add [skip ci] to the commit message inside the image automation resource cc @squaremo

[haraldatbmw]

Just to make it clear the CI logic has to do this:

pseudo-code:

...
var commitUser = extractCommitUser();

if (commitUser == 'flux-image-automation') {
  var tagName = extractTagNameFromCommit();

  if (isPreReleaseTag(tagName) || isRelease(tagName)) {
    createAndPushTag(tagName);
  }
}
...

[squaremo]

I missed the bit where the automation decides what the tag name is -- does it get named for the image that's been updated, or bumped from the last such tag, or ..?

[haraldatbmw]

In my case the app-repo git-tag is also the image-tag which can be used to create the tag in the ops-repo.
So the automation knows the image-tag?

[squaremo]

This feels like quite a specific use case*, and I'm reluctant to add specialised features to a controller. But I do want to get to a point where you have a plausible way forward @haraldatbmw. (*The specific bit being: the deployments are driven by a git tag which is exactly the image tag of a particular image.)

If we could make it possible for you to set up a CI action something like the pseudo-code you posted above, would that be good enough do you think? I am thinking mainly about being able to reliably get the image tag in question from the commit message; e.g., by providing template values for each image that was updated in the commit, so you can include them.

[haraldatbmw]

@squaremo I think the CI action should be fine for me. It would be nice if the commit-message of the image-automation-commit would contain the value of the replaced image-policy in the yaml file. So my CI action can parse this info and reuse it for tagging.