How to accelerate the Kubernetes cluster configuration using FluxCD – Part1

Have you ever had to create a new kubernetes cluster and thought “Oh no, where is the helm chart I used to install and configure the operator X in the current cluster?”. On a enterprise scale, having to provision new kubernetes clusters can be a recurring and painful task. Whether it’s for Disaster Recovery tests, segregating departments, experimenting with different features/operators/versions, there is always a baseline of working stuff you have to configure before the caller is able to start pushing workloads into it.

It’s true that the “resource creation” part is easily solved with Infra as Code (using tools such Terraform, CloudFormation or Biceps), but after that, we still have a “configuration” part where we have to install an significant amount of tools to make the user experience smoother.

Most companies does the cluster configuration with a bunch of scripts and automations that are often outdated compared to the actual cluster, due to version updates, feature changes and etc. That is where the GitOps framework and FluxCD comes in! The idea today is to show how to use GitOps beyond the workload-specific configuration and show how it can accelerate the cluster configuration process with FluxCD.

The FluxCD GitOps tool

FluxCD is a tool for automatically deploying and managing applications on Kubernetes using GitOps principles. It continuously monitors Git repositories for changes and applies them to the Kubernetes cluster, enabling declarative and automated infrastructure management through version-controlled configuration stored in Git repositories. To achieve this, the operator mainly provides two new Kubernetes resources:

1. GitRepository: This resource is responsible for monitoring a Git repository where your manifests are stored. It ensures that FluxCD always has the latest version of your files.
2. Kustomization: This resource allows you to customize and deploy the manifests in the kubernetes cluster.

The operator also adds support for HelmCharts installation with HelmRepository and HelmRelease resources, which allow us to benefit of the GitOps concept for more than the basics and automate the cluster configuration in a declarative way instead of performing helm CLI commands.

Organizing the components into manifests

First of all, we need to create a Git Repository to add the manifests we want to apply to our cluster. For this lab, I’ve created the k8s-base-config repository on GitHub with two main folders:

1. global-config: the manifests included in this folder will be applied automatically during flux installation
2. extra-capabilities: the manifests included in this folder will only be applied if they are explicitly declared

As an example, we are considering that KEDA is an operator that must be activated in all the clusters in which FluxCD will be installed. To do this, a folder called “keda-operator” was created inside the global-config folder with the following files:

#### helmrepository.yml
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
  ## Name of the repository.
  ## Same value used on "helm repo add [NAME]" command.
  name: keda-operator 
  namespace: flux-system 
spec:
  ## Interval that you want flux search for new versions.
  interval: 5m0s
  ## Url field must be the endpoint of the helmrepository. 
  ## Same value used on "helm repo add [NAME] [URL]" command.
  url: https://kedacore.github.io/charts 
  
#### helmrelease.yml
apiVersion: helm.toolkit.fluxcd.io/v2beta2
kind: HelmRelease
metadata:
  ## Name of the helm installation. 
  ## Same value used on "helm install [NAME]" command.
  name: keda-operator 
  namespace: flux-system
spec:
  ## Interval that you want to apply the lastest config in case new version is available.
  interval: 5m0s 
  chart:
    spec:
      ## Chart field must be the name of the helmchart. 
      ## Same value used on "helm install [NAME] [CHART]" command.
      chart: keda 
      ## Version field must be the chart version you want to install
      ## Same value used on "--version" parameter.
      version: '2.*'
      ## SourceRef must match the same name used in the helmrepository.yml file
      sourceRef:
        kind: HelmRepository
        name: keda-operator
    install:
      createNamespace: true
  targetNamespace: keda

### kustomization.yml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- helmrepository.yml
- helmrelease.yml

If you want to install any extra components, you only need to create a new folder and add the manifests and flux will apply them.

Installing FluxCD and the global components

FluxCD must be installed on your cluster using FluxCLI. To install the CLI, perform the following command:

brew install fluxcd/tap/flux

In case you don’t use homebrew, you can find more installation options here: Install the FluxCLI

Once the CLI is installed, the flux bootstrap command can be used to install the operator. As we’re using GitHub as repository, we’ll perform the following command to start the installation:

### A PAT with read and write access to the repository must be provided.
export GITHUB_TOKEN=<myPAT> 

flux bootstrap github \
  --token-auth \
  --owner=diego7marques \
  --repository=k8s-base-config \
  --branch=main \
  --path=global-config \
  --personal

And just like that, FluxCD and the componentes you declared inside the global-config folder will be installed. You can see how fast it can be in the following example:

Conclusion

As we can see, FluxCD proves to be a powerful tool to help us to accelerate the bootstrap process of a new Kubernetes clusters, ensuring consistency and continuous compliance on cluster configuration and management. With all the components declared in a repository, you can guarantee the state of the cluster and manage the entire lifecycle of your configurations in a much simpler and transparent way.

To avoid a long and tedious post, I’ve decided to split it into two parts. In the second section, we’ll cover how to manage and enable specific capabilities with more fine-grained control. Wait for it 🙂