Horizontal Pod Autoscaling in Kubernetes

Horizontal Pod Autoscaling (HPA) automatically adjusts the number of Pod replicas based on observed metrics. When traffic increases and CPU usage rises, HPA adds more Pods. When traffic drops, it removes them. This is the primary mechanism for elastic scaling in Kubernetes, and getting it right means the difference between wasting money on idle Pods and dropping requests during traffic spikes.

How HPA works

The HPA controller runs a control loop every 15 seconds (configurable). In each iteration, it:

1. Fetches the current metric values for all Pods in the target Deployment
2. Calculates the desired replica count using the formula: desired = ceil(current * (currentMetricValue / targetMetricValue))
3. Scales the Deployment if the desired count differs from the current count

Current: 4 replicas, each at 75% CPU
Target: 50% CPU

Desired = ceil(4 * (75 / 50)) = ceil(4 * 1.5) = ceil(6) = 6 replicas

The HPA scales the Deployment to 6 replicas. On the next check, if CPU drops to 50%, no scaling happens. If it drops to 25%, the HPA calculates ceil(6 * (25 / 50)) = 3 and scales down.

Prerequisites: Metrics Server

HPA requires a metrics source. For CPU and memory metrics, you need Metrics Server installed in the cluster.

# Check if metrics-server is running
kubectl get deployment metrics-server -n kube-system

# Install metrics-server (if not present)
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

# Verify it works
kubectl top nodes
kubectl top pods

On local clusters (minikube, kind), you may need to add --kubelet-insecure-tls to the metrics-server deployment args.

# For minikube
minikube addons enable metrics-server

Basic HPA: CPU-based scaling

First, your Deployment must have CPU resource requests defined. Without requests, HPA cannot calculate utilization percentages.

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
        - name: web
          image: my-web-app:latest
          ports:
            - containerPort: 8080
          resources:
            requests:
              cpu: "200m"     # 0.2 CPU cores
              memory: "256Mi"
            limits:
              cpu: "500m"
              memory: "512Mi"

# hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 50  # Target 50% CPU utilization

kubectl apply -f deployment.yaml
kubectl apply -f hpa.yaml

# Or create with kubectl directly
kubectl autoscale deployment web-app \
  --min=2 --max=10 --cpu-percent=50

# Monitor HPA status
kubectl get hpa web-app-hpa -w

The output shows current utilization, target, and replica count:

NAME          REFERENCE            TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
web-app-hpa   Deployment/web-app   23%/50%   2         10        2          5m

Memory-based scaling

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 50
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 70

When multiple metrics are specified, HPA calculates the desired replica count for each metric and uses the highest value. If CPU says 4 replicas and memory says 6, HPA scales to 6.

Memory-based scaling can be tricky because many applications (especially JVM-based) allocate memory at startup and never release it. This means the HPA might scale up but never scale down. Use memory scaling cautiously and prefer CPU for most workloads.

Absolute value targets

Instead of utilization percentages, you can target absolute metric values.

metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: AverageValue
        averageValue: 100m  # Target 100 millicores per pod
  - type: Resource
    resource:
      name: memory
      target:
        type: AverageValue
        averageValue: 200Mi  # Target 200Mi per pod

Absolute targets are useful when you want consistent per-pod resource usage regardless of how big the requests are.

Custom metrics

For more sophisticated scaling, you can use custom metrics from your application. This requires a custom metrics adapter (like Prometheus Adapter).

Setting up Prometheus Adapter

helm install prometheus-adapter prometheus-community/prometheus-adapter \
  --namespace monitoring \
  --set prometheus.url=http://prometheus-server.monitoring.svc \
  --set prometheus.port=9090

# prometheus-adapter-config.yaml (simplified)
rules:
  - seriesQuery: 'http_requests_total{namespace!="",pod!=""}'
    resources:
      overrides:
        namespace: { resource: "namespace" }
        pod: { resource: "pod" }
    name:
      matches: "^(.*)_total$"
      as: "${1}_per_second"
    metricsQuery: 'rate(<<.Series>>{<<.LabelMatchers>>}[2m])'

HPA with custom metrics

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 2
  maxReplicas: 20
  metrics:
    # Scale based on requests per second per pod
    - type: Pods
      pods:
        metric:
          name: http_requests_per_second
        target:
          type: AverageValue
          averageValue: "100"  # Target 100 RPS per pod

External metrics

Scale based on metrics that are not tied to Kubernetes resources, like a message queue depth.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: queue-worker-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: queue-worker
  minReplicas: 1
  maxReplicas: 50
  metrics:
    - type: External
      external:
        metric:
          name: sqs_queue_length
          selector:
            matchLabels:
              queue: "processing-queue"
        target:
          type: AverageValue
          averageValue: "10"  # 10 messages per pod

When the queue has 500 messages, HPA calculates 500 / 10 = 50 and scales to 50 pods (capped at maxReplicas).

Scaling behavior

The behavior field (autoscaling/v2) gives fine-grained control over how fast scaling happens.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 2
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 50
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 60  # Wait 60s before scaling up
      policies:
        - type: Percent
          value: 100        # Double the replicas
          periodSeconds: 60
        - type: Pods
          value: 4          # Or add 4 pods
          periodSeconds: 60
      selectPolicy: Max     # Use whichever adds more pods
    scaleDown:
      stabilizationWindowSeconds: 300  # Wait 5 min before scaling down
      policies:
        - type: Percent
          value: 10         # Remove 10% of replicas
          periodSeconds: 60
      selectPolicy: Min     # Conservative: remove fewer pods

Key settings:

• stabilizationWindowSeconds — How long the HPA waits before acting on a scale decision. Prevents flapping. Default: 0 for scale up, 300 for scale down.
• policies — Rate limits for scaling. Multiple policies use selectPolicy to choose.
• selectPolicy — Max (most aggressive), Min (most conservative), or Disabled (no scaling in this direction).

Preventing scale-down flapping

The default 5-minute stabilization window for scale-down is often too short for applications with bursty traffic. Increase it for workloads with irregular patterns.

behavior:
  scaleDown:
    stabilizationWindowSeconds: 600  # 10 minutes
    policies:
      - type: Pods
        value: 1
        periodSeconds: 300  # Remove at most 1 pod every 5 minutes

Aggressive scale-up for latency-sensitive apps

behavior:
  scaleUp:
    stabilizationWindowSeconds: 0  # Scale up immediately
    policies:
      - type: Percent
        value: 200          # Triple the replicas if needed
        periodSeconds: 30

Testing HPA with load generation

# Generate CPU load to trigger scaling
kubectl run load-generator --image=busybox:1.36 --restart=Never -- \
  /bin/sh -c "while true; do wget -q -O- http://web-app:8080; done"

# Watch the HPA respond
kubectl get hpa web-app-hpa -w

# Clean up
kubectl delete pod load-generator

Troubleshooting

# Check HPA status and events
kubectl describe hpa web-app-hpa

# Common issues:
# "unable to fetch metrics" — Metrics Server not running
# "missing request for cpu" — No CPU requests on containers
# "<unknown>/50%" — Metrics not available yet (wait ~60s)

# Verify metrics are available
kubectl top pods -l app=web-app

# Check metrics API directly
kubectl get --raw "/apis/metrics.k8s.io/v1beta1/namespaces/default/pods" | jq .

Best practices

1. Always set resource requests. HPA cannot calculate utilization without them.
2. Start with CPU-based scaling. It is the most predictable and well-understood metric.
3. Set sensible min/max bounds. minReplicas should handle your baseline traffic. maxReplicas should be capped by your budget and cluster capacity.
4. Use aggressive scale-up, conservative scale-down. It is better to have a few extra pods for a few minutes than to drop traffic during a spike.
5. Combine HPA with PodDisruptionBudgets. Ensure that scale-down does not violate your availability requirements.
6. Do not use HPA with Deployments that also have manual replica counts managed by CI/CD. They will fight each other.

HPA is the standard mechanism for elastic scaling in Kubernetes. Start simple with CPU-based scaling, add custom metrics as your observability matures, and tune the scaling behavior to match your application’s traffic patterns.