Packaging Kubernetes Applications with Helm Charts

kubernetes

Master the art of packaging Kubernetes applications using Helm charts for streamlined deployment and management across development, staging, and production environments. This guide covers chart structure, templating, value overrides, and best practices.

Manually applying Kubernetes YAML files for each environment (development, staging, production) often involves repetitive copying and modifying values. A more efficient approach is to package your Kubernetes application as a Helm chart, leveraging templating for easy customization and management across various environments.

The Challenge

The goal is to create a robust Helm chart for your application. This involves:

  • Packaging the deployment, service, and ConfigMap resources within the chart.
  • Utilizing templating mechanisms for environment-specific values.
  • Supporting multiple environments through dedicated values files.
  • Including proper chart metadata.
  • Thoroughly testing the chart deployment process.

Upon successful completion, you will have:

  • A working Helm chart structure.
  • Templated Kubernetes resources.
  • Specific values files for development, staging, and production environments.
  • A successfully validated Helm chart.
  • A chart that deploys correctly to a Kubernetes cluster.

Sample Application Chart Directory Structure

demo-app/ ├── Chart.yaml # Chart metadata ├── values.yaml # Default values ├── values-dev.yaml # Development overrides ├── values-staging.yaml # Staging overrides ├── values-prod.yaml # Production overrides ├── templates/ │ ├── deployment.yaml │ ├── service.yaml │ ├── configmap.yaml │ ├── ingress.yaml │ ├── hpa.yaml │ ├── _helpers.tpl # Template helpers │ └── NOTES.txt # Post-install notes ├── charts/ # Dependency charts └── .helmignore # Files to ignore

Solution: Chart Files

Below are examples of the core Helm chart files and their configurations.

Chart.yaml

apiVersion: v2
name: demo-app
description: A Helm chart for deploying the demo application
type: application
version: 1.0.0
appVersion: "1.0.0"
keywords:
  - demo
  - nodejs
  - example
maintainers:
  - name: DevOps Team
    email: [email protected]
home: https://github.com/yourorg/demo-app
sources:
  - https://github.com/yourorg/demo-app
dependencies: []
annotations:
  category: Application
licenses: MIT

values.yaml (Default Values)

# Default values for demo-app
replicaCount: 2
image:
  repository: your-registry/demo-app
  pullPolicy: IfNotPresent
  tag: "" # Defaults to chart appVersion
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
serviceAccount:
  create: true
  annotations: {}
  name: ""
podAnnotations: {}
podSecurityContext:
  runAsNonRoot: true
  runAsUser: 1000
  fsGroup: 1000
securityContext:
  allowPrivilegeEscalation: false
  capabilities:
    drop:
      - ALL
  readOnlyRootFilesystem: true
service:
  type: ClusterIP
  port: 80
  targetPort: 3000
  annotations: {}
ingress:
  enabled: false
  className: nginx
  annotations: {}
  hosts:
    - host: demo-app.local
      paths:
        - path: /
          pathType: Prefix
tls: []
resources:
  limits:
    cpu: 200m
    memory: 256Mi
  requests:
    cpu: 100m
    memory: 128Mi
autoscaling:
  enabled: false
  minReplicas: 2
  maxReplicas: 10
  targetCPUUtilizationPercentage: 70
  targetMemoryUtilizationPercentage: 80
nodeSelector: {}
tolerations: []
affinity: {}
config:
  appName: "Demo App"
  logLevel: "info"
  nodeEnv: "production"
env:
  - name: PORT
    value: "3000"
healthCheck:
  liveness:
    enabled: true
    path: /health
    initialDelaySeconds: 10
    periodSeconds: 10
  readiness:
    enabled: true
    path: /ready
    initialDelaySeconds: 5
    periodSeconds: 5

templates/_helpers.tpl

{{- /*
Expand the name of the chart.
*/ -}}
{{- define "demo-app.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- end -}}

{{- /*
Create a default fully qualified app name.
*/ -}}
{{- define "demo-app.fullname" -}}
{{- if .Values.fullnameOverride -}}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- if contains $name .Release.Name -}}
{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- end -}}
{{- end -}}

{{- /*
Create chart name and version as used by the chart label.
*/ -}}
{{- define "demo-app.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
{{- end -}}

{{- /*
Common labels
*/ -}}
{{- define "demo-app.labels" -}}
helm.sh/chart: {{ include "demo-app.chart" . }}
{{ include "demo-app.selectorLabels" . }}
{{- if .Chart.AppVersion -}}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote -}}
{{- end -}}
app.kubernetes.io/managed-by: {{ .Release.Service -}}
{{- end -}}

{{- /*
Selector labels
*/ -}}
{{- define "demo-app.selectorLabels" -}}
app.kubernetes.io/name: {{ include "demo-app.name" . }}
app.kubernetes.io/instance: {{ .Release.Name -}}
{{- end -}}

{{- /*
Create the name of the service account to use
*/ -}}
{{- define "demo-app.serviceAccountName" -}}
{{- if .Values.serviceAccount.create -}}
{{- default (include "demo-app.fullname" .) .Values.serviceAccount.name -}}
{{- else -}}
{{- default "default" .Values.serviceAccount.name -}}
{{- end -}}
{{- end -}}

{{- /*
Image name
*/ -}}
{{- define "demo-app.image" -}}
{{- $tag := .Values.image.tag | default .Chart.AppVersion -}}
{{- printf "%s:%s" .Values.image.repository $tag -}}
{{- end -}}

templates/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "demo-app.fullname" . }}
  labels:
    {{- include "demo-app.labels" . | nindent 4 }}
spec:
  {{- if not .Values.autoscaling.enabled }}
  replicas: {{ .Values.replicaCount }}
  {{- end }}
  selector:
    matchLabels:
      {{- include "demo-app.selectorLabels" . | nindent 6 }}
  template:
    metadata:
      annotations:
        checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml" ) . | sha256sum }}
        {{- with .Values.podAnnotations }}
        {{- toYaml . | nindent 8 }}
        {{- end }}
      labels:
        {{- include "demo-app.selectorLabels" . | nindent 8 }}
    spec:
      {{- with .Values.imagePullSecrets }}
      imagePullSecrets:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      serviceAccountName: {{ include "demo-app.serviceAccountName" . }}
      securityContext:
        {{- toYaml .Values.podSecurityContext | nindent 8 }}
      containers:
        - name: {{ .Chart.Name }}
          securityContext:
            {{- toYaml .Values.securityContext | nindent 12 }}
          image: {{ include "demo-app.image" . }}
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          ports:
            - name: http
              containerPort: {{ .Values.service.targetPort }}
              protocol: TCP
          env:
            {{- range .Values.env }}
            - name: {{ .name }}
              value: {{ .value | quote }}
            {{- end }}
          envFrom:
            - configMapRef:
                name: {{ include "demo-app.fullname" . }}
          {{- if .Values.healthCheck.liveness.enabled }}
          livenessProbe:
            httpGet:
              path: {{ .Values.healthCheck.liveness.path }}
              port: http
            initialDelaySeconds: {{ .Values.healthCheck.liveness.initialDelaySeconds }}
            periodSeconds: {{ .Values.healthCheck.liveness.periodSeconds }}
          {{- end }}
          {{- if .Values.healthCheck.readiness.enabled }}
          readinessProbe:
            httpGet:
              path: {{ .Values.healthCheck.readiness.path }}
              port: http
            initialDelaySeconds: {{ .Values.healthCheck.readiness.initialDelaySeconds }}
            periodSeconds: {{ .Values.healthCheck.readiness.periodSeconds }}
          {{- end }}
          resources:
            {{- toYaml .Values.resources | nindent 12 }}
      {{- with .Values.nodeSelector }}
      nodeSelector:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.affinity }}
affinity:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.tolerations }}
tolerations:
        {{- toYaml . | nindent 8 }}
      {{- end }}

templates/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: {{ include "demo-app.fullname" . }}
  labels:
    {{- include "demo-app.labels" . | nindent 4 }}
  {{- with .Values.service.annotations }}
  annotations:
    {{- toYaml . | nindent 4 }}
  {{- end }}
spec:
  type: {{ .Values.service.type }}
  ports:
    - port: {{ .Values.service.port }}
      targetPort: http
      protocol: TCP
      name: http
  selector:
    {{- include "demo-app.selectorLabels" . | nindent 4 }}

templates/configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ include "demo-app.fullname" . }}
  labels:
    {{- include "demo-app.labels" . | nindent 4 }}
data:
  APP_NAME: {{ .Values.config.appName | quote }}
  LOG_LEVEL: {{ .Values.config.logLevel | quote }}
  NODE_ENV: {{ .Values.config.nodeEnv | quote }}

templates/ingress.yaml

{{- if .Values.ingress.enabled -}}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{ include "demo-app.fullname" . }}
  labels:
    {{- include "demo-app.labels" . | nindent 4 }}
  {{- with .Values.ingress.annotations }}
  annotations:
    {{- toYaml . | nindent 4 }}
  {{- end }}
spec:
  {{- if .Values.ingress.className }}
  ingressClassName: {{ .Values.ingress.className }}
  {{- end }}
  {{- if .Values.ingress.tls }}
tls:
    {{- range .Values.ingress.tls }}
    - hosts:
      {{- range .hosts }}
      - {{ . | quote }}
      {{- end }}
      secretName: {{ .secretName }}
    {{- end }}
  {{- end }}
rules:
  {{- range .Values.ingress.hosts }}
  - host: {{ .host | quote }}
    http:
      paths:
      {{- range .paths }}
      - path: {{ .path }}
        pathType: {{ .pathType }}
        backend:
          service:
            name: {{ include "demo-app.fullname" $ }}
            port:
              number: {{ $.Values.service.port }}
      {{- end }}
  {{- end }}
{{- end }}

templates/hpa.yaml

{{- if .Values.autoscaling.enabled }}
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: {{ include "demo-app.fullname" . }}
  labels:
    {{- include "demo-app.labels" . | nindent 4 }}
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: {{ include "demo-app.fullname" . }}
  minReplicas: {{ .Values.autoscaling.minReplicas }}
  maxReplicas: {{ .Values.autoscaling.maxReplicas }}
  metrics:
    {{- if .Values.autoscaling.targetCPUUtilizationPercentage }}
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: {{ .Values.autoscaling.targetCPUUtilizationPercentage }}
    {{- end }}
    {{- if .Values.autoscaling.targetMemoryUtilizationPercentage }}
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }}
    {{- end }}
{{- end }}

templates/NOTES.txt

1. Get the application URL by running these commands:
{{- if .Values.ingress.enabled }}
{{- range $host := .Values.ingress.hosts }}
  {{- range .paths }}
  http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ .path }}
  {{- end }}
{{- end }}
{{- else if contains "NodePort" .Values.service.type }}
  export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "demo-app.fullname" . }})
  export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
  echo http://$NODE_IP:$NODE_PORT
{{- else if contains "LoadBalancer" .Values.service.type }}
     NOTE: It may take a few minutes for the LoadBalancer IP to be available.
           You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "demo-app.fullname" . }}'
  export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "demo-app.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
  echo http://$SERVICE_IP:{{ .Values.service.port }}
{{- else if contains "ClusterIP" .Values.service.type }}
  export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "demo-app.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
  export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT
{{- end }}

Environment-Specific Values

Helm allows overriding default values using environment-specific values files.

values-dev.yaml (Development)

replicaCount: 1
image:
  tag: "dev-latest"
  pullPolicy: Always
resources:
  limits:
    cpu: 100m
    memory: 128Mi
  requests:
    cpu: 50m
    memory: 64Mi
config:
  appName: "Demo App - DEV"
  logLevel: "debug"
  nodeEnv: "development"
ingress:
  enabled: true
  hosts:
    - host: demo-dev.example.com
      paths:
        - path: /
          pathType: Prefix
autoscaling:
  enabled: false

values-prod.yaml (Production)

replicaCount: 5
image:
  tag: "1.0.0"
  pullPolicy: IfNotPresent
resources:
  limits:
    cpu: 500m
    memory: 512Mi
  requests:
    cpu: 250m
    memory: 256Mi
config:
  appName: "Demo App"
  logLevel: "warn"
  nodeEnv: "production"
service:
  type: LoadBalancer
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
ingress:
  enabled: true
  className: nginx
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
  hosts:
    - host: demo.example.com
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: demo-app-tls
      hosts:
        - demo.example.com
autoscaling:
  enabled: true
  minReplicas: 3
  maxReplicas: 10
  targetCPUUtilizationPercentage: 70
affinity:
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchExpressions:
              - key: app.kubernetes.io/name
                operator: In
                values:
                  - demo-app
          topologyKey: kubernetes.io/hostname

Explanation: Helm Concepts

  1. Templates Helm uses Go templating with various built-in functions. For example:

    name: {{ include "demo-app.fullname" . }}
    • {{ }}: Denotes a template action.
    • .Values: Accesses values defined in values.yaml or override files.
    • .Chart: Provides access to chart metadata (e.g., name, version).
    • .Release: Contains information about the Helm release.
    • include: Inserts the content of a named template helper.

  2. Values Hierarchy Values are merged in a specific order, with later sources overriding earlier ones:

    • Default values.yaml
    • Environment-specific values-<env>.yaml files (e.g., values-prod.yaml)
    • Command-line --set flags (e.g., helm install my-app ./demo-app -f values-prod.yaml --set replicaCount=10)

  3. Helpers (_helpers.tpl) _helpers.tpl files contain reusable template snippets, often used for common labels or name generation, ensuring consistency and promoting the DRY (Don't Repeat Yourself) principle.

    Example helper for common labels:

    {{- define "demo-app.labels" -}}
app: {{ .Chart.Name }}
version: {{ .Chart.Version }}
{{- end -}}

    This helper can then be used in other templates with:

    {{- include "demo-app.labels" . | nindent 4 -}}

Chart Operations

Install the Chart

To begin, lint the chart for syntax errors, then perform a dry run to inspect the generated Kubernetes manifests before actual deployment.

# Lint the chart
helm lint demo-app/

# Dry run to see generated manifests
helm install my-app demo-app/ --dry-run --debug

# Install to the development environment
helm install my-app-dev demo-app/ \
  -f demo-app/values-dev.yaml \
  --namespace demo-dev \
  --create-namespace

Expected output for installation:

NAME: my-app-dev
LAST DEPLOYED: Fri Dec  8 12:00:00 2023
NAMESPACE: demo-dev
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
  export POD_NAME=$(kubectl get pods --namespace demo-dev -l "app.kubernetes.io/name=demo-app" -o jsonpath="{.items[0].metadata.name}")
  kubectl --namespace demo-dev port-forward $POD_NAME 8080:3000

Verify the deployment:

# List releases
helm list -n demo-dev

# Check deployment
kubectl get all -n demo-dev

Upgrade Release

To modify deployed applications, update the values and perform an upgrade. For example, to change replicaCount:

# Modify values (e.g., in values-dev.yaml or via --set)
echo "replicaCount: 3" >> values-dev.yaml

# Upgrade the release
helm upgrade my-app-dev demo-app/ \
  -f demo-app/values-dev.yaml \
  -n demo-dev

# Check the release history
helm history my-app-dev -n demo-dev

Rollback

If an upgrade introduces issues, you can easily roll back to a previous revision.

# Rollback to the previous version
helm rollback my-app-dev -n demo-dev

# Rollback to a specific revision (e.g., revision 1)
helm rollback my-app-dev 1 -n demo-dev

Validation: Testing Checklist

Follow these steps to thoroughly validate your Helm chart:

  1. Lint the chart:

    helm lint demo-app/
# Expected: 1 chart(s) linted, 0 chart(s) failed

  2. Template validation: Generate manifests and perform a client-side dry run.

    helm template my-app demo-app/ -f demo-app/values-dev.yaml > /tmp/rendered.yaml
kubectl apply --dry-run=client -f /tmp/rendered.yaml
# Expected: Successful validation without errors

  3. Install and verify: Install a test release and check pod status.

    helm install test-release demo-app/ -f demo-app/values-dev.yaml --namespace test --create-namespace
kubectl get pods -n test
# Expected: Pods should be in a 'Running' state

  4. Test upgrade: Perform an upgrade with a new value.

    helm upgrade test-release demo-app/ --set replicaCount=2 -n test
# Expected: Upgrade should succeed

  5. Verify values applied: Confirm the new value is reflected.

    kubectl get deployment -n test -o jsonpath='{.items[0].spec.replicas}'
# Expected: Returns '2'

  6. Cleanup: Uninstall the test release and delete the namespace.

    helm uninstall test-release -n test
kubectl delete namespace test

Best Practices for Helm Charts

Do's:

  • Use helpers: Keep your templates DRY (Don't Repeat Yourself) by creating reusable snippets in _helpers.tpl.
  • Version charts: Adopt semantic versioning for your charts (Major.Minor.Patch) to manage changes effectively.
  • Default values: Provide sensible default values in values.yaml.
  • Document values: Include comments for all options in your values.yaml to guide users.
  • Test thoroughly: Always lint, dry-run template, and test actual deployments.
  • Use checksum annotations: Add annotations to deployments (e.g., checksum/config) to force pod restarts when ConfigMaps or Secrets change.

Don'ts:

  • Don't hardcode: Avoid hardcoding values; use the values.yaml mechanism for all configurable parameters.
  • Don't skip linting: Always run helm lint early in your development cycle to catch errors.
  • Don't ignore NOTES.txt: Provide helpful post-install instructions in NOTES.txt to guide users.
  • Don't forget .helmignore: Use .helmignore to exclude unnecessary files from your chart package, keeping it lean.
  • Don't make charts too complex: Keep charts focused on a single application or microservice for better maintainability.

Further Reading