Multi Environment Workflows in Kustomize

This article explains how to set up a multi environment workflow in Kustomize step by step, with detailed examples.

Managing Kubernetes configurations for multiple environments such as development, staging, and production can be challenging. Kustomize simplifies this by allowing you to reuse and customize configurations without duplicating code.

Prerequisites:

1. Kubernetes Cluster

1. A Kubernetes cluster to deploy resources.

2. Kustomize Installed

• Install Kustomize on your system.

3. Basic Knowledge of Kubernetes

• Familiarity with Kubernetes objects like Deployment, Service, etc.

What is a Multi-Environment Workflow in Kustomize?

A multi-environment workflow refers to organizing and managing Kubernetes configurations for different environments by:

1. Defining a shared base configuration that contains common resources.
2. Customizing overlays for environment-specific modifications.

This approach ensures:

• Code Reusability: Shared configurations are reused.
• Customizability: Each environment has tailored configurations.
• Simplicity: Maintenance becomes easier as shared changes propagate automatically.

Directory Structure in Kustomize

A typical directory structure for multi-environment workflows in Kustomize looks like this:

my-app/
├── base/
│   ├── deployment.yaml
│   ├── service.yaml
│   └── kustomization.yaml
├── overlays/
│   ├── dev/
│   │   ├── kustomization.yaml
│   │   └── patch.yaml
│   ├── staging/
│   │   ├── kustomization.yaml
│   │   └── patch.yaml
│   └── prod/
│       ├── kustomization.yaml
│       └── patch.yaml

Explanation of Directories

• base/: Contains common Kubernetes manifests shared across environments (e.g., Deployment, Service).
• overlays/: Contains customizations specific to each environment (e.g., dev, staging, prod).
  • Each environment has its own kustomization.yaml and a patch.yaml for environment-specific changes.

#1.Set Up the Base Directory

The base/ directory contains the shared Kubernetes manifests.

1. Create the base/ directory:

mkdir -p my-app/base

2. Add shared resources to the base/ directory.

Example: base/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  labels:
    app: my-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app
        image: nginx:latest
        ports:
        - containerPort: 80

Example: base/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: my-app
  labels:
    app: my-app
spec:
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: my-app

3. Create base/kustomization.yaml to reference these resources:

Example: base/kustomization.yaml

resources:
  - deployment.yaml
  - service.yaml

#2.Set Up Overlays for Each Environment

Create separate overlays for each environment (e.g., dev, staging, prod).

1. Create directories for each environment:

mkdir -p my-app/overlays/dev my-app/overlays/staging my-app/overlays/prod

2. Add a kustomization.yaml file in each overlay directory to reference the base and apply patches.

Example: overlays/dev/kustomization.yaml

resources:
  - ../../base

patchesStrategicMerge:
  - patch.yaml

#3.Add Environment-Specific Patches

Each environment will have a patch.yaml to define customizations.

Example: Development Patch (overlays/dev/patch.yaml)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 2
  template:
    spec:
      containers:
      - name: my-app
        env:
        - name: ENV
          value: "development"

Example: Staging Patch (overlays/staging/patch.yaml)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  template:
    spec:
      containers:
      - name: my-app
        env:
        - name: ENV
          value: "staging"

Example: Production Patch (overlays/prod/patch.yaml)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 5
  template:
    spec:
      containers:
      - name: my-app
        env:
        - name: ENV
          value: "production"

#4.Build and Apply Configurations

You can build and apply environment-specific configurations using the kustomize CLI.

Build the Configuration

• Development:

kustomize build my-app/overlays/dev

Development Environment Result:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: my-app
  name: my-app
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: my-app
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: my-app
  name: my-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - env:
        - name: ENV
          value: development
        image: nginx:latest
        name: my-app
        ports:
        - containerPort: 80

• Staging:

kustomize build my-app/overlays/staging

Staging Environment Result:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: my-app
  name: my-app
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: my-app
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: my-app
  name: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - env:
        - name: ENV
          value: staging
        image: nginx:latest
        name: my-app
        ports:
        - containerPort: 80

• Production:

kustomize build my-app/overlays/prod

Production Environment Result:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: my-app
  name: my-app
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: my-app
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: my-app
  name: my-app
spec:
  replicas: 5
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - env:
        - name: ENV
          value: production
        image: nginx:latest
        name: my-app
        ports:
        - containerPort: 80

Apply the Configuration to Kubernetes

• Development:

kustomize build my-app/overlays/dev | kubectl apply -f -

• Staging:

kustomize build my-app/overlays/staging | kubectl apply -f -

• Production:

kustomize build my-app/overlays/prod | kubectl apply -f -

5. Verify the Deployment

After applying the manifests, check the deployed resources:

kubectl get deployments
kubectl get services

Benefits of Multi-Environment Workflows in Kustomize

1. Code Reusability: The base configuration is shared across environments.
2. Customizability: Each environment can have tailored settings.
3. Scalability: Changes to the base automatically propagate to all overlays.
4. Simplicity: Easier maintenance and reduced redundancy.

Conclusion:

Multi-environment workflows in Kustomize provide an efficient way to manage Kubernetes configurations for different environments like development, staging, and production. By leveraging a shared base and environment-specific overlays, you can ensure consistency, reduce redundancy, and improve maintainability. With the step-by-step guide outlined in this article, you can implement these workflows to simplify your Kubernetes deployments and scale your applications with ease.

Related Articles:

Strategic Merge Patches in Kubernetes using Kustomize

Reference:

kustomize official documentation