Performance Testing: Load, Stress, and Spike Testing

Performance testing answers a simple question: what happens to your application under realistic or extreme traffic? Without it, you find out in production when your site goes down during a product launch or marketing campaign. Performance testing is not just “hit the server with lots of requests.” Different test types answer different questions.

Types of performance tests

Load testing simulates expected traffic to verify your system meets performance requirements under normal conditions. “Can we handle 1,000 concurrent users with response times under 500ms?”

Stress testing pushes beyond normal capacity to find the breaking point. “What happens at 5,000 concurrent users? At 10,000?”

Spike testing simulates a sudden burst of traffic. “Can we handle 10x normal traffic for 2 minutes without crashing?”

Soak testing runs at moderate load for an extended period (hours or days) to find memory leaks, connection pool exhaustion, and gradual degradation.

Load testing with k6

k6 is an open-source tool built for developer productivity. Tests are written in JavaScript and run from the command line.

Installation

# macOS
brew install k6

# Linux
sudo apt install k6

# Docker
docker run --rm -i grafana/k6 run - <script.js

Basic load test

// load-test.js
import http from "k6/http";
import { check, sleep } from "k6";

export const options = {
  stages: [
    { duration: "1m", target: 50 },   // Ramp up to 50 users over 1 minute
    { duration: "3m", target: 50 },   // Stay at 50 users for 3 minutes
    { duration: "1m", target: 0 },    // Ramp down to 0 over 1 minute
  ],
  thresholds: {
    http_req_duration: ["p(95)<500"],  // 95% of requests must complete in <500ms
    http_req_failed: ["rate<0.01"],    // Less than 1% failure rate
  },
};

export default function () {
  // Simulate a user browsing the site
  const homeResponse = http.get("http://localhost:3000/");
  check(homeResponse, {
    "home status is 200": (r) => r.status === 200,
    "home loads in <200ms": (r) => r.timings.duration < 200,
  });

  sleep(1); // Think time: user reads the page

  const productsResponse = http.get("http://localhost:3000/api/products");
  check(productsResponse, {
    "products status is 200": (r) => r.status === 200,
    "products returns array": (r) => JSON.parse(r.body).length > 0,
  });

  sleep(2);

  // Simulate viewing a product
  const productResponse = http.get("http://localhost:3000/api/products/1");
  check(productResponse, {
    "product status is 200": (r) => r.status === 200,
  });

  sleep(1);
}

k6 run load-test.js

Stress test

Push past normal limits to find the breaking point.

// stress-test.js
import http from "k6/http";
import { check, sleep } from "k6";

export const options = {
  stages: [
    { duration: "2m", target: 100 },   // Normal load
    { duration: "5m", target: 100 },
    { duration: "2m", target: 200 },   // Beyond normal
    { duration: "5m", target: 200 },
    { duration: "2m", target: 500 },   // Stress level
    { duration: "5m", target: 500 },
    { duration: "2m", target: 1000 },  // Breaking point?
    { duration: "5m", target: 1000 },
    { duration: "5m", target: 0 },     // Recovery
  ],
  thresholds: {
    http_req_duration: ["p(99)<2000"],  // Even under stress, 99th percentile <2s
  },
};

export default function () {
  const response = http.get("http://localhost:3000/api/products");
  check(response, {
    "status is 200": (r) => r.status === 200,
  });
  sleep(1);
}

Spike test

Sudden surge of traffic, then back to normal.

// spike-test.js
import http from "k6/http";
import { sleep } from "k6";

export const options = {
  stages: [
    { duration: "1m", target: 50 },    // Normal traffic
    { duration: "10s", target: 500 },  // Spike! 10x in 10 seconds
    { duration: "2m", target: 500 },   // Sustained spike
    { duration: "10s", target: 50 },   // Back to normal
    { duration: "3m", target: 50 },    // Recovery period
    { duration: "30s", target: 0 },
  ],
};

export default function () {
  http.get("http://localhost:3000/api/products");
  sleep(0.5);
}

Testing authenticated endpoints

// authenticated-test.js
import http from "k6/http";
import { check, sleep } from "k6";

export function setup() {
  // Login once and share the token across virtual users
  const loginResponse = http.post(
    "http://localhost:3000/api/auth/login",
    JSON.stringify({
      email: "loadtest@example.com",
      password: "test-password",
    }),
    { headers: { "Content-Type": "application/json" } }
  );

  const token = JSON.parse(loginResponse.body).access_token;
  return { token };
}

export default function (data) {
  const params = {
    headers: {
      Authorization: `Bearer ${data.token}`,
      "Content-Type": "application/json",
    },
  };

  // Authenticated request
  const response = http.get("http://localhost:3000/api/orders", params);
  check(response, {
    "authenticated request succeeds": (r) => r.status === 200,
  });

  sleep(1);

  // Create an order
  const orderResponse = http.post(
    "http://localhost:3000/api/orders",
    JSON.stringify({
      items: [{ product_id: 1, quantity: 1 }],
    }),
    params
  );

  check(orderResponse, {
    "order created": (r) => r.status === 201,
  });

  sleep(2);
}

Load testing with Locust (Python)

Locust is a Python-based alternative that lets you define user behavior as Python classes.

# locustfile.py
from locust import HttpUser, task, between

class WebsiteUser(HttpUser):
    wait_time = between(1, 3)  # Random wait between 1-3 seconds

    def on_start(self):
        """Called when a simulated user starts."""
        response = self.client.post("/api/auth/login", json={
            "email": "loadtest@example.com",
            "password": "test-password",
        })
        self.token = response.json()["access_token"]
        self.headers = {"Authorization": f"Bearer {self.token}"}

    @task(3)  # Weight: 3x more likely than other tasks
    def browse_products(self):
        self.client.get("/api/products")

    @task(2)
    def view_product(self):
        product_id = 1  # In reality, pick randomly
        self.client.get(f"/api/products/{product_id}")

    @task(1)
    def place_order(self):
        self.client.post(
            "/api/orders",
            json={"items": [{"product_id": 1, "quantity": 1}]},
            headers=self.headers,
        )

# Run Locust with web UI
locust -f locustfile.py --host http://localhost:3000

# Run headless
locust -f locustfile.py --host http://localhost:3000 \
  --headless -u 100 -r 10 --run-time 5m

Interpreting results

A k6 output looks like this.

     scenarios: (100.00%) 1 scenario, 50 max VUs, 5m30s max duration
                default: Up to 50 looping VUs for 5m0s

     ✓ status is 200
     ✓ response time OK

     checks.........................: 100.00% ✓ 14523  ✗ 0
     data_received..................: 45 MB   150 kB/s
     data_sent......................: 1.2 MB  4.0 kB/s
     http_req_blocked...............: avg=1.2ms  min=0s     med=0s     max=245ms  p(90)=0s     p(95)=0s
     http_req_duration..............: avg=45ms   min=12ms   med=38ms   max=892ms  p(90)=78ms   p(95)=123ms
       { expected_response:true }...: avg=45ms   min=12ms   med=38ms   max=892ms  p(90)=78ms   p(95)=123ms
     http_reqs......................: 14523  48.41/s
     iteration_duration.............: avg=3.04s  min=3.01s  med=3.03s  max=4.89s  p(90)=3.08s  p(95)=3.12s
     vus............................: 50     min=1      max=50

Key metrics to watch

http_req_duration (p95): 95th percentile response time. If this is 123ms, 95% of requests completed in under 123ms. This is more meaningful than the average because averages hide outliers.

http_req_failed: Error rate. Anything above 1% is usually a problem.

http_reqs: Throughput (requests per second). This tells you the actual capacity.

Trend over time: A gradually increasing response time suggests resource exhaustion (connection pool, memory, CPU).

What the numbers tell you

p95 Response Time	Assessment
< 100ms	Excellent
100ms - 500ms	Good for most apps
500ms - 1000ms	Needs investigation
> 1000ms	Users will notice
> 3000ms	Unacceptable for interactive use

Finding bottlenecks

When performance tests reveal problems, here is how to track down the cause.

Database bottlenecks

The most common performance bottleneck. Signs: response time increases linearly with load, database CPU is high, or slow query logs show up.

-- Check for slow queries in PostgreSQL
SELECT
    query,
    calls,
    round(mean_exec_time::numeric, 2) AS avg_ms,
    round(total_exec_time::numeric, 2) AS total_ms
FROM pg_stat_statements
ORDER BY total_exec_time DESC
LIMIT 10;

Connection pool exhaustion

Signs: requests start queuing, response times spike suddenly (not gradually), and you see “connection pool exhausted” errors.

# Check your connection pool settings
# Most frameworks default to a small pool size
engine = create_engine(
    DATABASE_URL,
    pool_size=20,          # Default is often 5
    max_overflow=10,       # Extra connections when pool is full
    pool_timeout=30,       # Seconds to wait for a connection
    pool_recycle=1800,     # Recycle connections every 30 minutes
)

Memory leaks (soak testing)

Run a moderate load for several hours and monitor memory usage.

// soak-test.js
export const options = {
  stages: [
    { duration: "5m", target: 100 },   // Ramp up
    { duration: "4h", target: 100 },   // Hold for 4 hours
    { duration: "5m", target: 0 },     // Ramp down
  ],
};

If memory usage grows continuously without stabilizing, you have a leak.

CPU-bound bottlenecks

Signs: CPU usage hits 100% on one or more cores, response times increase proportionally with load.

Common causes: unoptimized image processing, JSON serialization of large payloads, regex operations on large strings, or compression of large responses.

Setting performance budgets

Define pass/fail criteria before you run tests.

// k6 thresholds act as automated pass/fail gates
export const options = {
  thresholds: {
    // Response time
    http_req_duration: [
      "p(95)<500",    // 95th percentile under 500ms
      "p(99)<1000",   // 99th percentile under 1000ms
    ],

    // Error rate
    http_req_failed: ["rate<0.01"],  // Less than 1% errors

    // Throughput
    http_reqs: ["rate>100"],  // At least 100 requests per second

    // Custom metrics for specific endpoints
    "http_req_duration{name:create_order}": ["p(95)<1000"],
    "http_req_duration{name:list_products}": ["p(95)<200"],
  },
};

CI/CD integration

Run performance tests in your deployment pipeline.

# GitHub Actions
performance-test:
  runs-on: ubuntu-latest
  services:
    app:
      image: myapp:${{ github.sha }}
      ports: ["3000:3000"]

  steps:
    - uses: actions/checkout@v4

    - name: Install k6
      run: |
        sudo gpg -k
        sudo gpg --no-default-keyring --keyring /usr/share/keyrings/k6-archive-keyring.gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D68
        echo "deb [signed-by=/usr/share/keyrings/k6-archive-keyring.gpg] https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list
        sudo apt-get update && sudo apt-get install k6

    - name: Run load test
      run: k6 run --out json=results.json tests/load-test.js

    - name: Upload results
      uses: actions/upload-artifact@v4
      with:
        name: k6-results
        path: results.json

Wrapping Up

Performance testing is not a one-time activity. Run load tests before every major release, stress tests quarterly to understand your current limits, and soak tests when you suspect memory leaks. Use k6 or Locust to write tests that simulate real user behavior. Set thresholds as automated pass/fail gates. When tests reveal problems, check the database first (it is almost always the database), then connection pools, then memory and CPU. The goal is not to optimize everything but to know your system’s limits before your users discover them.