Fleet is a Kubernetes-native GitOps tool that allows you to manage multiple AKS clusters from a single control plane. Think of it as a meta-cluster that orchestrates other clusters.
Here’s Fleet in action. Let’s say you have three AKS clusters: eastus-prod, westus-staging, and europe-dev. You want to deploy a common set of applications (like Prometheus and Grafana) to all of them, but with slightly different configurations for each environment.
First, you’d set up your Fleet control plane. This is typically a dedicated Kubernetes cluster where Fleet components run.
apiVersion: fleet.hazelcast.com/v1alpha1
kind: Fleet
metadata:
name: fleet-manager
namespace: fleet-system
spec:
# Fleet specific configurations here
# ...
Next, you’d register your AKS clusters as "members" of your Fleet. This involves creating Cluster resources in the Fleet control plane.
apiVersion: fleet.hazelcast.com/v1alpha1
kind: Cluster
metadata:
name: eastus-prod
spec:
# Details for connecting to the eastus-prod AKS cluster
# This might involve a kubeconfig or a service account
# ...
---
apiVersion: fleet.hazelcast.com/v1alpha1
kind: Cluster
metadata:
name: westus-staging
spec:
# Details for connecting to the westus-staging AKS cluster
# ...
---
apiVersion: fleet.hazelcast.com/v1alpha1
kind: Cluster
metadata:
name: europe-dev
spec:
# Details for connecting to the europe-dev AKS cluster
# ...
Once your clusters are registered, you can define "Workloads." A Workload is essentially a Kubernetes manifest or a Helm chart that you want to deploy. For example, to deploy Prometheus:
apiVersion: fleet.hazelcast.com/v1alpha1
kind: Workload
metadata:
name: prometheus-operator
spec:
helm:
chart: prometheus-operator
repo: https://prometheus-community.github.io/helm-charts
version: "46.3.0"
namespace: monitoring
values: |
alertmanager:
enabled: true
prometheus:
enabled: true
grafana:
enabled: true
Now, the magic happens with "Deployments." A Deployment in Fleet specifies which Workloads should be deployed to which Clusters, and critically, how those Workloads should be configured per cluster.
apiVersion: fleet.hazelcast.com/v1alpha1
kind: Deployment
metadata:
name: monitoring-stack
spec:
workloads:
- name: prometheus-operator
clusters:
- name: eastus-prod
namespace: monitoring # Override namespace for this cluster
values: | # Override Helm values for this cluster
prometheus:
retention: 24h # Keep data for 24 hours in production
- name: westus-staging
namespace: monitoring
values: |
prometheus:
retention: 7d # Keep data for 7 days in staging
- name: europe-dev
namespace: monitoring
values: |
prometheus:
retention: 1d # Keep data for 1 day in dev
When you apply this Deployment resource, Fleet will reconcile the state. It will ensure that the prometheus-operator Workload is deployed to eastus-prod, westus-staging, and europe-dev. For each cluster, it will use the specified namespace and apply the corresponding values override. If a Cluster resource is added or removed, Fleet will automatically provision or de-provision the Workloads accordingly.
This approach solves the problem of managing application deployments and configurations across a fleet of Kubernetes clusters consistently and declaratively. You define your desired state in Git, and Fleet ensures that all your registered clusters conform to that state. It abstracts away the complexity of individual cluster connections and applies GitOps principles at a meta-cluster level.
A key aspect of Fleet’s operation is how it handles cluster registration and the subsequent GitOps reconciliation. When you register a cluster, Fleet doesn’t just store its endpoint; it establishes a connection (often via a service account or kubeconfig) and deploys a lightweight agent within that member cluster. This agent is responsible for receiving instructions from the Fleet control plane and applying the desired state, whether that’s deploying a Helm chart or a raw Kubernetes manifest. The Cluster resource in the Fleet control plane is thus more than just an endpoint; it represents the identity and operational status of a managed cluster within the Fleet system.
The next step is understanding how to implement advanced deployment strategies like canary releases or blue/green deployments across your managed clusters using Fleet’s capabilities.