Better docs around patches broadly (new guide)

[kingdonb]

We have a lot of different places that you can apply patches and a veritable diaspora of information spread around all the various locations that document various components of Flux and adjacent tools like Kustomize

We have this guide for Flux's built-in Kustomization patches:
https://fluxcd.io/flux/components/kustomize/kustomization/#patches

Kustomize has various examples for patches of different types:

• https://github.com/kubernetes-sigs/kustomize/blob/master/examples/jsonpatch.md
• https://github.com/kubernetes-sigs/kustomize/blob/master/examples/inlinePatch.md
• https://github.com/kubernetes-sigs/kustomize/blob/master/examples/patchMultipleObjects.md

There are some differences between Flux's patches and Kustomize CLI patches that make it somewhat difficult to narrow down what is the correct syntax to use in "this instance of patching" whatever it is you're about to try, and there are some also apparently disjoint implementations apparently owing to deprecations that have not been fully deprecated, like Flux Kustomization's patchesStrategicMerge field.

• https://github.com/kubernetes-sigs/kustomize/blob/master/examples/patchMultipleObjects.md

So depending on where you are when you want to patch your Flux-controlled resources, when you need them patched, and where else you might be able to move laterally to from there (eg. sometimes Flux Kustomization.spec.patches may be your only option, but other times you can edit kustomization.yaml and use a patch file.)... it would be great to have one guide that covers all the options, that we can point people towards when they have trouble related to patching.

This latest issue report from fluxcd/terraform-provider-flux (fluxcd/terraform-provider-flux#333) that we discussed at today's Bug Scrub drove the point home for me, (I need to spend some more time getting familiar with Flux+terraform before I'll be ready to write something like this.)

[stefanprodan]

@kingdonb Flux TF provider does not support patching at all. Even if you add a patch, it will not be executed at install time. Only after Flux is installed from the original files, the patch will be applied by kustomize-controller, which is too late in most case.

[kingdonb]

Thanks for that detail; it might be too late if the gitrepo is protected by a secret that you need to decrypt from KMS. All of this worth making part of the patching guide.

(You just saved me going down a rabbit hole later on when I get to that step in my tf learning goals to achieve relative to my talks before KubeCon, so double thanks 😄)

[stefanprodan]

Thanks for that detail; it might be too late if the gitrepo is protected by a secret that you need to decrypt from KMS. All of this worth making part of the patching guide.

The flux-system Kustomization can't deal with decryption, nor can flux bootstrap. This was never supported, so how come you're bringing this up?

[stefanprodan]

Let me give you an example when you need to patch the Flux controllers before installation. Let's assume all your nodes are tainted, if you run terraform apply the Flux pods will have no place to run and the bootstrap will fail. If TF would know (like the Flux CLI) to patch the manifests locally before it applies them on the cluster, then the pods will run because the tolerations are in-place.

But if you want to enable KMS access to some other Kustomization than flux-system, you can do that after install, as kustomize-controller will update its own service account when it sees the SA patch in Git.

[kingdonb]

The flux-system Kustomization can't deal with decryption

I was suggesting that you might have the flux-system secret encrypted – but you're right, that would never work with Flux, it would have to be fetched from an external store by some terraform module in order to make the deploy key usable with terraform-provider-flux. I had in mind that you might have stored your deploy keys in vault or KMS to avoid recreating them, but that won't work with Flux patching as explained.

So, that whole idea that: once Flux is installed with terraform, as long as the bootstrap is successful then at that point, Flux can use kustomization.yaml to patch itself and upgrade to use a service account, so KMS decryption that requires the patch can succeed (but that you should only enable this use on others than flux-system, else you will block the bootstrap which cannot use decryption here.)

That's all information that seems like it would definitely be in scope for a guide about patching. 👍