Kubernetes Resource Limits, Requests, and QoS Classes

Every container in Kubernetes can specify how much CPU and memory it needs (requests) and how much it is allowed to use (limits). Getting these values wrong leads to OOMKilled pods, CPU throttling, unschedulable workloads, or wasted cluster resources. This article explains how requests and limits work, how they affect scheduling and QoS, and how to set them correctly.

Requests vs Limits

Requests tell the scheduler how much resource a pod needs. The scheduler uses requests to find a node with enough capacity. Requests are a guarantee — the kubelet reserves that amount for your container.

Limits are a hard ceiling. If a container tries to use more memory than its limit, it is killed (OOMKilled). If it tries to use more CPU than its limit, it is throttled (slowed down) but not killed.

apiVersion: v1
kind: Pod
metadata:
  name: api
spec:
  containers:
    - name: api
      image: myapp:1.0.0
      resources:
        requests:
          cpu: "250m"
          memory: "256Mi"
        limits:
          cpu: "1000m"
          memory: "512Mi"

CPU is measured in millicores. 250m is a quarter of one CPU core. Memory is measured in bytes, with Mi for mebibytes and Gi for gibibytes.

How the Scheduler Uses Requests

When you create a pod, the scheduler looks at all nodes and finds one where the sum of existing requests plus your pod’s requests does not exceed the node’s allocatable capacity.

Node capacity: 4 CPU, 8Gi memory
Existing pod requests: 2.5 CPU, 5Gi memory
Your pod requests: 1 CPU, 2Gi memory
Total after scheduling: 3.5 CPU, 7Gi memory --> Fits

If no node has enough capacity, the pod stays in Pending state with a FailedScheduling event.

Important: the scheduler uses requests, not limits. If your requests are too low and limits are high, you can overcommit the node. This works fine until all containers actually use their limits simultaneously, at which point the node runs out of resources and the kubelet starts evicting pods.

What Happens When Limits Are Exceeded

Memory: the container is killed immediately with OOMKilled status. The kernel’s OOM killer terminates the process. If the pod has a restart policy, Kubernetes restarts it. Check for OOMKilled:

kubectl get pod api -o jsonpath='{.status.containerStatuses[0].lastState.terminated.reason}'

CPU: the container is throttled. The kernel’s CFS (Completely Fair Scheduler) limits how much CPU time the process gets. The container is not killed, but it runs slower. Throttling is silent — there are no events or status changes. You detect it through metrics:

kubectl top pod api

QoS Classes

Kubernetes assigns each pod a Quality of Service class based on its resource configuration. The QoS class determines which pods are evicted first when a node runs out of resources.

Guaranteed

All containers have requests equal to limits for both CPU and memory:

resources:
  requests:
    cpu: "500m"
    memory: "256Mi"
  limits:
    cpu: "500m"
    memory: "256Mi"

Guaranteed pods are evicted last. Use this for critical workloads like databases or payment services.

Burstable

At least one container has a request or limit set, but they are not equal:

resources:
  requests:
    cpu: "250m"
    memory: "128Mi"
  limits:
    cpu: "1000m"
    memory: "512Mi"

Burstable pods can use more than their request up to their limit when resources are available. They are evicted before BestEffort pods but after Guaranteed pods.

BestEffort

No requests or limits set at all:

resources: {}

BestEffort pods are evicted first when the node is under pressure. Never use this in production.

Check a pod’s QoS class:

kubectl get pod api -o jsonpath='{.status.qosClass}'

Eviction Order

When a node runs low on memory, the kubelet evicts pods in this order:

1. BestEffort pods using the most memory relative to their request (which is zero)
2. Burstable pods using the most memory relative to their request
3. Guaranteed pods (only if the node is critically low)

This is why setting requests correctly matters even if you set limits. A pod with a 128Mi request using 500Mi of memory is evicted before a pod with a 256Mi request using 300Mi.

LimitRange: Namespace Defaults

Use LimitRange to set default requests and limits for a namespace. This prevents developers from deploying pods with no resource constraints:

apiVersion: v1
kind: LimitRange
metadata:
  name: default-limits
  namespace: production
spec:
  limits:
    - default:
        cpu: "500m"
        memory: "256Mi"
      defaultRequest:
        cpu: "100m"
        memory: "128Mi"
      max:
        cpu: "2000m"
        memory: "2Gi"
      min:
        cpu: "50m"
        memory: "64Mi"
      type: Container

• default sets the limit if none is specified
• defaultRequest sets the request if none is specified
• max and min reject pods outside these bounds

ResourceQuota: Namespace Totals

ResourceQuota limits the total resources a namespace can consume:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: namespace-quota
  namespace: production
spec:
  hard:
    requests.cpu: "8"
    requests.memory: "16Gi"
    limits.cpu: "16"
    limits.memory: "32Gi"
    pods: "50"

When a ResourceQuota is active, every pod must specify requests and limits. Without them, the pod is rejected. Use ResourceQuota together with LimitRange so developers get sensible defaults.

Practical Guidelines for Setting Values

Step 1: Observe Actual Usage

Deploy without limits first (in staging) and monitor actual usage:

# Current usage
kubectl top pods -n production

# Historical data with metrics-server
kubectl top pods -n production --sort-by=memory

For better data, use Prometheus and Grafana to track usage over days or weeks.

Step 2: Set Requests Based on Typical Usage

Set requests to the amount your container typically uses. If your API usually uses 200m CPU and 180Mi memory:

resources:
  requests:
    cpu: "250m"
    memory: "256Mi"

Add some headroom — about 20-30% above typical usage. This ensures the scheduler places your pod on a node with enough capacity for normal operation.

Step 3: Set Limits Based on Peak Usage

Set limits to handle spikes without letting a container consume the entire node:

resources:
  requests:
    cpu: "250m"
    memory: "256Mi"
  limits:
    cpu: "1000m"
    memory: "512Mi"

For memory, set the limit to handle peak usage. If your app peaks at 400Mi, set the limit to 512Mi. OOMKilled is disruptive — it is better to give a bit too much memory than too little.

For CPU, consider whether to set a limit at all. CPU throttling degrades performance silently. Some teams set CPU requests but not CPU limits, letting containers burst freely on CPU while the scheduler still reserves capacity.

Step 4: Iterate

Monitor, adjust, repeat. Resource usage changes as your application evolves. Review settings quarterly or after significant feature changes.

Common Mistakes

Setting requests equal to limits for everything. This prevents bursting and wastes resources. Only do this for Guaranteed QoS on critical pods.

Setting very low requests and very high limits. The scheduler thinks there is room on the node, but when all pods spike simultaneously, the node runs out of memory and evicts pods randomly.

Not setting any limits. A memory leak in one container can take down the entire node.

Ignoring CPU throttling. If your API response times are high, check if the container is being CPU throttled before adding more replicas.

Summary

Resource requests tell the scheduler where to place pods. Resource limits prevent containers from consuming too much. QoS classes determine eviction priority. Set requests based on typical usage with some headroom, set memory limits based on peak usage, and consider whether CPU limits are worth the throttling trade-off. Use LimitRange and ResourceQuota to enforce policies at the namespace level.