You may or may not be joking, but gitops has been the cause of outages for us in quite a few instances. I have been unable to stress what a terrible idea arbitrarily pulling and applying from git is. I am unable to stress it because that's the overwhelming way that many k8s setups are done, so clearly they cannot be wrong.
When I started out with Kubernetes Gitops hadn't really gained any momentum. We just used normal CI/CD pipeline tooling. For our own apps the pipeline simply built the docker image, pushed it, and then ran kubectl apply. No manual labour, no magic.
For third party stuff (like helm charts for ELK or whatever) it was the same but with helm cli/kubectl, without building the image.
I don't know why really, but gitops have never seemed that nice for me. It's perhaps kinda useful for third party stuff. But for your own applications, where you need to actually build the docker image and then either manually bump the tag or have some rube goldberg machinery committing to your repos, it just seems annoying.
Wanna see the state / source of truth? Use kubectl (or some other tool), we have this whole cluster just for keeping track of the state. Wanna see how/why/by who something was deployed? Look at the CI/CD tooling history.
What's really a True Scotsman? But I wouldn't really say so, no. I think [0] represent what I would consider a gitops pushed based approach, the way we did it differed in two main ways:
- (1) We didn't have any "environment repository". The manifest files were in the same repository as the application code
- (2) Perhaps more importantly: The manifest files did not _exactly_ represent what was deployed. We had a template-variable in the Deployment yaml file, where the Github action substituted the tag that had just been built. To see which version was deployed you either had to look in the cluster, or the Github Action logs.
If you use something like ArgoCD (and maybe also Argo Rollouts) you can do the diffing from Git automatically but either put a manual validation step where you have a chance to review the diff or implement some gradual rollout strategy.
Also, it's probably wise to use a branch/tagging strategy and not read from head.
Bottom line is: GitOps means the source of truth is Git and automation makes sure to avoid drifts. You still have to have a rollout strategy and schedule that makes sense for your usecase.
> Bottom line is: GitOps means the source of truth is Git and automation makes sure to avoid drifts. You still have to have a rollout strategy and schedule that makes sense for your usecase.
