Property-Based Testing: An Introduction

Most tests look like this: pick three inputs, write the expected outputs, and call it a day. That works until reality finds an input you did not pick. Property-based testing (PBT) flips the script: you describe a property the code should always satisfy, the library generates hundreds of inputs, and it tries to break you. The first time you watch it find a bug you never would have written, you stop writing single-example tests for anything tricky.

What a property is

A property is a statement that must hold for all inputs in some domain. Examples:

• Reversing a list twice gives the original list.
• Sorting a list produces a permutation of the input.
• Encoding then decoding a value returns the original.
• A discount is always at most the original amount.

These are not specific numbers. They are claims about behavior. The library does the work of finding inputs that violate the claim.

Mental model

Example-based:
input: [3,1,2]   expected: [1,2,3]
one assertion, one execution

Property-based:
forall xs : sort(sort(xs)) == sort(xs)
generator -> [random list] -> check
repeat 100x with shrinking on failure

When PBT fails, it does not just report the giant random input it found. It “shrinks” the input down to the smallest one that still fails. You get a tiny, reproducible counter-example instead of a 10,000-element list.

Hands-on: a property test in TypeScript

Using fast-check:

import fc from 'fast-check';
import { sortDesc } from './sort';

test('sortDesc is idempotent', () => {
  fc.assert(
    fc.property(fc.array(fc.integer()), (xs) => {
      const once = sortDesc(xs);
      const twice = sortDesc(once);
      expect(twice).toEqual(once);
    })
  );
});

test('sortDesc preserves length', () => {
  fc.assert(
    fc.property(fc.array(fc.integer()), (xs) => {
      expect(sortDesc(xs)).toHaveLength(xs.length);
    })
  );
});

test('sortDesc produces non-increasing order', () => {
  fc.assert(
    fc.property(fc.array(fc.integer()), (xs) => {
      const ys = sortDesc(xs);
      for (let i = 1; i < ys.length; i++) {
        expect(ys[i - 1]).toBeGreaterThanOrEqual(ys[i]);
      }
    })
  );
});

These three properties together pin down what sorting means. You did not write a single input.

Useful patterns

Oracle. Compare against a simple, slow implementation that is obviously correct. Your real implementation must produce the same output.

fc.assert(fc.property(fc.array(fc.integer()), (xs) =>
  fastSort(xs).join(',') === xs.slice().sort((a,b) => a-b).join(',')
));

Round-trip. Encoders and decoders, serializers and deserializers, compressors. decode(encode(x)) === x is one of the highest-value properties in any system.

Invariants. A bank transfer never changes total balance. A merge of two sorted lists is sorted. Stating invariants makes refactoring safer because the property describes intent, not implementation.

Model-based. Maintain a tiny model state in the test (a Map representing a cache) and check that your real implementation stays in sync after random operations.

Shrinking, the killer feature

If your test discovers that sortDesc([5, -3, 7, 0, 12, -100, 4]) fails, shrinking will simplify the input, often down to something like [0, -1]. The minimal case usually points at the bug directly. Spend less time bisecting; let the tool do it.

Hands-on: a real bug

Suppose someone wrote:

function isPalindrome(s: string) {
  return s === s.split('').reverse().join('');
}

Example tests for “abba” and “abc” pass. A property test:

fc.assert(fc.property(fc.string(), (s) => {
  const lower = s.toLowerCase();
  if (lower === lower.split('').reverse().join('')) {
    expect(isPalindrome(s)).toBe(true);
  }
}));

Will quickly fail on a string like "Aa". Shrinking confirms the casing bug.

Common pitfalls

• Writing tautologies. expect(x).toBe(x) is always true. The property has to express the function’s intent.
• Generators that are too narrow. Restricting input to positive integers may hide bugs that only appear with negatives, zero, or floats.
• Generators that are too broad. If half the inputs are invalid for your function, filter them with fc.pre instead of asserting only when valid; otherwise stats lie about coverage.
• Slow properties. PBT runs hundreds of iterations; a property that takes 100 ms each will take ten seconds. Keep them fast.
• Hidden state. If the function under test mutates global state, properties become flaky. Pure code is where PBT shines.
• Replacing example tests entirely. Examples still document intent and pin specific cases. Use both.

Practical tips

• Start with the easiest properties: idempotence, length preservation, round-trip. Even those find bugs.
• When PBT finds a failure, copy the minimal counter-example into a regular unit test before fixing. You now have a regression test forever.
• Use fc.context() to log shrunk inputs in CI; it makes debugging trivial.
• Seed the runner with a fixed value in CI for reproducibility, and rotate it occasionally so you do not over-fit.
• Run PBT with more iterations nightly than per-commit; the failure mode is “this would have failed at 10x iterations.”

Wrap-up

Property-based testing pays off the moment your code has any interesting structure: parsers, encoders, sorters, validators, business invariants. Write the properties once, get hundreds of cases for free, and let shrinking hand you the bug on a plate. It is one of the cheapest upgrades you can make to a test suite.