SRE Error Budgets and SLO/SLI Practices

The Core Idea

Every service has a reliability target. Users expect the checkout page to load, but they do not expect 100 percent uptime. They tolerate a small amount of failure. The question is: how much failure is acceptable?

SRE formalizes this with three concepts. An SLI (Service Level Indicator) is a metric that measures user-facing reliability. An SLO (Service Level Objective) is the target value for that metric. An error budget is the difference between 100 percent and the SLO: the amount of unreliability you are allowed.

If your SLO is 99.9 percent availability, your error budget is 0.1 percent. In a 30-day month, that is roughly 43 minutes of downtime. You can “spend” those 43 minutes on deployments, experiments, infrastructure migrations, or unexpected outages. When the budget runs out, you slow down and focus on reliability.

SLIs: What to Measure

SLIs must reflect user experience. Internal metrics like CPU usage or pod restart counts are useful for debugging but terrible as SLIs because they do not directly correlate with whether users can complete their tasks.

Good SLIs

Availability: The proportion of requests that succeed.

SLI = (total requests - 5xx errors) / total requests

Latency: The proportion of requests faster than a threshold.

SLI = requests with latency < 300ms / total requests

Correctness: The proportion of requests that return the right answer. Harder to measure, but critical for data-processing pipelines.

SLI = requests with valid response body / total requests

Freshness: For data pipelines, the proportion of time that data is no more than N minutes old.

SLI = time where data age < 5 minutes / total time

Choosing SLIs for Your Service

For an API service, start with availability and latency. For a data pipeline, use freshness and correctness. For a frontend, use availability and the proportion of page loads completing within a Core Web Vitals threshold.

A common mistake is having too many SLIs. Three to five per service is the sweet spot. More than that and the signal gets lost in noise.

SLOs: Setting Targets

An SLO puts a number on the SLI. Common targets:

Service Type	SLO	Error Budget (30 days)
Internal tool	99.0%	7.3 hours
B2B API	99.9%	43 minutes
Consumer checkout	99.95%	21.6 minutes
Payment processing	99.99%	4.3 minutes

How to Pick the Right Number

Start with what users actually experience today. If your service has been running at 99.7 percent availability over the last quarter, setting an SLO of 99.99 percent is not aspirational, it is delusional. You will burn through your error budget in the first week and the SLO becomes meaningless.

Set the SLO slightly above your current reliability. If you are at 99.7 percent, set it at 99.9 percent. This gives you a target to work toward without being unachievable.

Also consider your dependencies. If your database has 99.95 percent availability and your payment provider has 99.9 percent, your service mathematically cannot exceed 99.85 percent availability without significant redundancy. Your SLO must account for the reliability of your dependencies.

Error Budgets: The Math

The error budget is simple arithmetic:

Error budget = 1 - SLO

For a 99.9% SLO over 30 days:
  Budget = 0.1% of requests can fail
  Budget = 0.1% of 30 days = 43.2 minutes of total downtime

For request-based SLIs, calculate it in terms of requests:

If you serve 1,000,000 requests per day:
  Monthly requests = 30,000,000
  Error budget = 0.1% = 30,000 failed requests per month

You track the budget as a burn rate. If you are burning through the budget faster than the linear rate (roughly 1/30th per day), something is wrong.

Burn Rate Alerts

Instead of alerting when the error rate exceeds a fixed threshold, alert based on how fast you are consuming the error budget.

A burn rate of 1 means you will exactly exhaust the budget by the end of the window. A burn rate of 10 means you will exhaust the budget in 1/10th of the window.

# Prometheus alerting rules for multi-window burn rates
groups:
  - name: slo-burn-rate
    rules:
      # Fast burn: 2% of budget consumed in 1 hour
      - alert: ErrorBudgetFastBurn
        expr: |
          (
            sum(rate(http_requests_total{status=~"5.."}[1h])) / sum(rate(http_requests_total[1h]))
          ) > (14.4 * 0.001)
          and
          (
            sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m]))
          ) > (14.4 * 0.001)
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "Error budget fast burn detected"

      # Slow burn: 5% of budget consumed in 6 hours
      - alert: ErrorBudgetSlowBurn
        expr: |
          (
            sum(rate(http_requests_total{status=~"5.."}[6h])) / sum(rate(http_requests_total[6h]))
          ) > (6 * 0.001)
          and
          (
            sum(rate(http_requests_total{status=~"5.."}[30m])) / sum(rate(http_requests_total[30m]))
          ) > (6 * 0.001)
        for: 15m
        labels:
          severity: warning
        annotations:
          summary: "Error budget slow burn detected"

The multi-window approach uses a long window to detect sustained issues and a short window to confirm the issue is still happening. This reduces false positives compared to simple threshold-based alerts.

Error Budget Policies

An error budget is only useful if there are consequences for exhausting it. An error budget policy defines what happens:

When budget is healthy (more than 50 percent remaining):

• Teams ship features at normal velocity.
• Risky deployments and migrations are allowed.
• Experimentation is encouraged.

When budget is shrinking (25 to 50 percent remaining):

• Risky changes require additional review.
• Teams prioritize reliability-related work alongside features.
• Postmortems for budget-consuming incidents are required.

When budget is nearly exhausted (below 25 percent):

• Feature work is paused or significantly reduced.
• The team focuses on reliability improvements.
• All deployments require SRE approval.
• Only critical bug fixes and security patches ship.

When budget is exhausted (0 percent):

• Feature freeze until budget regenerates.
• All engineering effort goes to reliability.
• Escalation to engineering leadership.

This policy is negotiated between product, engineering, and SRE leadership. It makes the tradeoff between features and reliability explicit and data-driven instead of political.

Tracking Error Budgets

Build a dashboard that shows:

1. Current error budget remaining (percentage and absolute time or requests).
2. Budget burn rate over the last 1 hour, 1 day, and 7 days.
3. Projected budget exhaustion date at the current burn rate.
4. Historical budget trend over the last 30 and 90 days.

# Error budget remaining (request-based)
1 - (
  sum(increase(http_requests_total{status=~"5.."}[30d]))
  /
  (sum(increase(http_requests_total[30d])) * (1 - 0.999))
)

Make this dashboard visible to everyone: product managers, engineers, and leadership. When product asks “can we ship this risky feature?” the answer is on the dashboard. If there is 60 percent budget remaining, yes. If there is 5 percent, no.

Common Mistakes

Setting SLOs without buy-in. An SLO that only SRE cares about is just a number on a dashboard. Product and engineering leadership must agree to the error budget policy, including the feature freeze consequences.

Treating SLOs as SLAs. An SLO is an internal target. An SLA is a contractual commitment with financial penalties. Your SLO should be stricter than your SLA so you catch problems before they become contract violations.

Measuring the wrong thing. Server-side availability is not the same as user-perceived availability. A 200 response that returns an error page is not a success. Measure what the user experiences, not what the server thinks happened.

Never exhausting the budget. If your error budget is always at 99 percent, your SLO is too loose. You are over-investing in reliability at the expense of feature velocity. Tighten the SLO until the budget is occasionally consumed.

Start with one critical service. Define two SLIs (availability and latency), set an SLO based on current performance, build the burn-rate alerts, and propose an error budget policy to your team. The conversation that follows will be more valuable than any technology you deploy.