kubernetes

In this tutorial, we are going to spin up a CockroachDB secure cluster running in Kubernetes with certificates managed by HashiCorp Vault and issued by cert-manager. Before we get to the final state, we are going to evolve the installation step by step to understand how each component is contributing to the setup and what we are gaining.

We’ll focus on the execution, assuming you are minimally familiar with these components. Links will be provided for additional context when necessary.

This tutorial can be executed from your local machine, no cloud resources are necessary.

We all know the benefits of using managed Kubernetes services like GKE, EKS, AKS, etc. Given the complexity of managing the cluster infrastructure and its core components (control plane, auto-scaling, monitoring, networking, storage, etc.), using a managed Kubernetes service is generally the first choice when running workloads in production.

However, in some situations, provisioning and managing the Kubernetes cluster from scratch might be necessary. Specific product features, security & compliance, costs, vendor independency, etc. are some factors that usually justify the decision of running Kubernetes by yourself. Of course, many challenges come with managing a Kubernetes cluster and the discussion around which route - self-managed vs. managed - to take is far from simple. I want to keep this discussion out of the scope of this tutorial since it would require special attention.

Currently, the tool of choice when considering provisioning and managing a Kubernetes cluster is Cluster API. From the docs:

Cluster API is a Kubernetes sub-project focused on providing declarative APIs and tooling to simplify provisioning, upgrading, and operating multiple Kubernetes clusters.

[…] The supporting infrastructure, like virtual machines, networks, load balancers, and VPCs, as well as the Kubernetes cluster configuration are all defined in the same way that application developers operate deploying and managing their workloads. This enables consistent and repeatable cluster deployments across a wide variety of infrastructure environments.

In this post, I want to share the approach I have been using to scope and make decisions when defending a Kubernetes environment. The approach presented here defines ten high-level security concerns and is used with other frameworks and benchmarks to contextualize and guide the security efforts. For each security concern, I present an informal and opinionated description based on my experience.

Infrastructure as Code (IaC) is an essential piece of the modern software development landscape. IaC tools allow teams to define their infrastructure in a reproducible-automated fashion, increasing speed and consistency while preventing errors, misconfiguration, and configuration drifts.

IaC also plays an important role in security. It gives visibility over the whole infrastructure, including its security aspects and components, even before they have been deployed. Development and/or security and operations teams can define sensible security defaults and a security baseline enforcing best practices that are automated and applied via the CICD pipeline, where changes can be reviewed and approved following the separation of duties principle.

In this tutorial, we will deploy Cilium to an Amazon EKS cluster and limit traffic to Pods using Kubernetes Network Policies.

Network Policies are not available in Kubernetes out-of-the-box. To leverage Network Policies, you must use a network plugin that implements such a feature - here is where Cilium comes into play.

It’s worth noting that while this is the scope of this tutorial, Cilium is not limited to Network Policies and not even limited to Kubernetes. Check the Introduction to Cilium to learn more about it and the Kubernetes Integration doc about how Cilium works with Kubernetes.

In this tutorial, we are going to play with the Google Kubernetes Engine Dataplane V2 and check how we can use it along with Kubernetes Network Policies to limit traffic to Pods and to obtain real-time visibility on cluster network activity.

Dataplane V2 is a recent feature in GKE, with GA starting on version 1.20.6-gke.700 as of May 10, 2021. It uses Cilium to process network packets in-kernel using Kubernetes-specific metadata without relying on the kube-proxy and iptables for service routing, resulting in performance improvements. Dataplane V2 brings some exciting features for cluster operations and security, such as:

• Built-in Network Policies enforcement without the need of Calico and;
• Real-time visibility, enabling cluster networking troubleshooting, auditing, and alerting.

In this tutorial, we are going to configure and explore the HashiCorp Vault AWS Auth method with Amazon EKS. We will start performing the Vault authentication using the EC2 instances (Kubernetes nodes) identity and later we will use a Kubernetes service account to impersonate an AWS IAM Role and have more fine-grained control at the Pod level.

In the source code for this tutorial, we extend the Getting started with Cloud Endpoints for GKE with ESP documentation guide to provide an example of how to configure HTTPS between the LB and the ESP.

In this tutorial, we’re going to go through the Workload Identity feature and see how it helps to improve the way we manage access to Google Services and APIs from applications running in Google Kubernetes Engine (GKE).

Workload Identity is the recommended way to access Google Cloud APIs from within GKE due to its improved security properties and manageability. With Workload Identity you can control access to APIs using Google service accounts and IAM roles without deploying static service account JSON keys to Pods and without relying on the node’s service account.