GitOps on Amazon EKS using FluxCD, Docker and AWS CDK

By Irshad A Buchh

Elevator Pitch

Companies want to go fast, they need to develop and deploy applications more often, more reliably, and preferably with less overhead. GitOps is a fast and secure method for developers and operation teams to manage and update complex applications and infrastructure running in Kubernetes.

Description

GitOps is an operations and application deployment workflow and a set of best practices for managing both infrastructure and deployments for cloud-native applications. This talk is divided into two parts. In the first part, I shall provide you with the history of GitOps as well as a description of how it works and what the benefits are. In the second part, I shall show you a demo that describes how to set up a continuous deployment pipeline with Flux and Docker to Amazon Elastic Kubernetes Service (Amazon EKS). The demo is implemented using AWS Cloud Development Kit (AWS CDK).

The AWS CDK is an open source software development framework to define your cloud application resources using familiar programming languages. Provisioning cloud applications can be a challenging process that requires you to perform manual actions, write custom scripts, maintain templates, or learn domain-specific languages. AWS CDK uses the familiarity and expressive power of programming languages for modeling your applications.

Notes

This talk will focus on the following core principals of GitOps:

1. Your entire system described declaratively

Kubernetes is one of many modern cloud native tools out there that is managed with declarative configurations. Declarative means that configuration is guaranteed by a set of facts instead of by a set of instructions. This means that configuration can be treated as code and kept in Git alongside your application source code. But more importantly, with your entire system’s declarative configuration under source control, you have a single source of truth of your system’s desired state, providing a number of benefits like the ability to simultaneously manage infrastructure and application code.

2. A desired system state version controlled in Git

With the declarative definitions of your system stored in Git, and serving as your canonical source of truth, there is a single place from which your cluster can be managed. This also trivializes rollbacks and roll forwards to take you back to a ‘good state’ if needed. With Git’s excellent security guarantees, SSH key signed commits to enforce strong security guarantees about authorship as well as the code’s provenance.

3. The ability for changes to be automatically applied

Once the desired system state is kept in Git, the next step is the ability to automatically reconcile changes with your system. What’s significant about this is that you don’t need specific cluster credentials to make a change. With GitOps, there is a separated environment that the state definition lives outside of. This allows your team to separate what they actually do, for example, deploying code from how they are going to do it, for example, configuring the cluster to deploy the code.

4. Software agents that verify correct system state and alert on divergence

With the desired state of your entire system kept under version control, and running in the cluster, you can now employ software controllers to bring the actual state of the cluster in alignment with that desired state, and inform you whenever reality doesn’t match your expectations. The use of a GitOps controller like Flux in combination with these software agents ensures that your entire system is self-healing. And by self-healing, we don’t mean when nodes or pods fail, those are handled by Kubernetes, but in a broader sense, in the case of human error, for example. In this case, software agents act as the feedback and control loop for your operations.