My biggest problem with Rancher fleet currently is that when something doesn't work, it's very hard to locate an error message telling you what actually is holding things up. You might see the fleet agent giving a 401, or a bunch of resources in state "missing", while they are definitely in the gitops repo.

But to list some others:

· · Web · 6 · 0 · 0

Bundles (fleet's crd for resources to apply) are not a useful abstraction. They can contain anything from a single config map all the way to a helm chart and several different yamls. So deleting or modifying a bundle can affect an arbitrary amount of resources.

Bundle names, which are based on folders in the gitops repo cap out at 63 characters, which means that deeply nested gitops repos just don't work.

For some reason, resources are applied alphabetically. So fleet will fail to start the yaml called app-that-needs-sql.yaml, because it's missing the sql-details.yaml configmap, even though both are in the same folder in the gitops repo. I don't know how, but flux has no problem with such a thing, so I classify it as poor implementation rather than a limitation.

The fleet.yaml files control what is in a bundle, they have to be littered everywhere.

It tries to figure out what the applied resources should look like but it can't account for things like mutating webhooks so it's almost always wrong, and when that happens the bundles go into state "modified" which makes it look like something is wrong. I suspect this could be fixed by doing a server-side apply, or just not peering as deeply into applied yamls as it does.

Their solution for this is JSON diff blocks, that tell fleet to ignore changes to a specific part of the yaml. This however is super anti-gitops, because it's not the repo telling the cluster what the state should be. It's the cluster (fleet-agent) telling a human what the repo should be. Moreover, this is impossible to predict for an arbitrary resource, and so is antithetical to automating management/generation of the gitops repo.

Sign in to participate in the conversation

Welcome to my Blog! I mostly talk about DevOps, Cloud, Linux and Kubernetes. Huge Tech, Chess and Outdoor Fan.