System Design: Design a Feature Flag Service

Feature flags let you decouple deployment from release. You ship code behind a flag, enable it for 1% of users, observe metrics, and gradually roll out to 100% — or kill it instantly if something breaks. Services like LaunchDarkly, Unleash, and Flagsmith provide this capability, and building one reveals important lessons about distributed configuration management.

Functional Requirements

• Create, update, and delete feature flags via an API and dashboard.
• Evaluate flags for a given user context (user ID, country, plan, device) and return boolean or multivariate values.
• Support targeting rules: enable a flag for specific users, segments, or a percentage of traffic.
• Support flag variations: boolean (on/off), string, number, or JSON values.
• Provide an SDK for server-side and client-side evaluation.
• Log every flag evaluation for analytics and debugging.

Non-Functional Requirements

• Evaluation latency: under 1ms per flag evaluation. Flags are checked in the hot path of every request.
• Availability: flag evaluation must work even if the central service is down.
• Consistency: flag changes should propagate to all evaluation points within 10 seconds.
• Scale: support tens of thousands of flags evaluated billions of times per day.

High-Level Architecture

The system has three components:

1. Control plane — a web service and dashboard for managing flags, targeting rules, and segments.
2. Flag store — a database holding all flag configurations.
3. Evaluation SDKs — libraries embedded in application code that evaluate flags locally using a cached copy of the flag configuration.

The critical insight is that flag evaluation happens locally in the SDK, not via an API call per evaluation. The SDK periodically syncs the full flag configuration from the control plane and evaluates flags in memory. This keeps evaluation latency under a millisecond and makes the system resilient to control plane outages.

Flag Data Model

A feature flag configuration includes:

{
  "key": "new-checkout-flow",
  "type": "boolean",
  "default_variation": false,
  "targeting_enabled": true,
  "rules": [
    {
      "description": "Beta users",
      "conditions": [
        { "attribute": "plan", "operator": "in", "values": ["beta"] }
      ],
      "variation": true
    },
    {
      "description": "10% rollout",
      "conditions": [],
      "percentage_rollout": {
        "bucket_by": "user_id",
        "variations": [
          { "value": true, "weight": 10 },
          { "value": false, "weight": 90 }
        ]
      }
    }
  ],
  "fallthrough_variation": false,
  "version": 42,
  "updated_at": "2026-07-02T10:00:00Z"
}

Rules are evaluated in order. The first matching rule determines the result. If no rules match, the fallthrough variation is returned. If targeting is disabled, the default variation is returned for all users.

Flag Evaluation Algorithm

Given a user context and a flag key, the SDK evaluates as follows:

1. Look up the flag configuration by key from the in-memory cache.
2. If the flag does not exist or is archived, return the application-defined default.
3. If targeting is disabled, return the default variation.
4. Iterate through rules in order. For each rule, evaluate all conditions against the user context. If all conditions match, return the rule’s variation.
5. For percentage rollouts, hash the user’s bucket attribute (typically user ID) to produce a deterministic number between 0 and 99. Map that number to a variation based on the weight distribution. This ensures the same user always gets the same variation without storing any per-user state.
6. If no rules match, return the fallthrough variation.

The hash-based bucketing is essential: it makes rollouts deterministic and sticky per user without requiring a database lookup.

Real-Time Flag Propagation

When an operator toggles a flag, the change must reach all evaluation points quickly. Three propagation mechanisms, in order of preference:

Server-Sent Events (SSE). SDKs maintain a persistent SSE connection to the control plane. When a flag changes, the control plane pushes an event containing the updated flag configuration. This delivers changes within seconds.

Polling. As a fallback, SDKs poll the control plane every 30 seconds. The request includes the current configuration version, and the server returns only flags that have changed since that version (delta updates).

Webhook push. For server-side SDKs behind firewalls that cannot maintain outbound connections, the control plane pushes changes to a webhook endpoint on the application’s network.

The SDK maintains a local persistent cache (file or embedded database) so that if the control plane is unreachable at startup, the SDK can load the last known configuration and continue operating.

Segments and Reusable Targeting

Rather than duplicating targeting conditions across flags, define segments — named groups of users with shared conditions. For example, a segment “enterprise-customers” with the condition plan in [enterprise, enterprise-plus] can be referenced by any flag.

Segments are stored alongside flags and synced to SDKs. When a segment’s conditions change, all flags referencing it automatically pick up the new targeting.

Analytics and Experimentation

Log every flag evaluation: (flag_key, user_id, variation, timestamp). Ship these events to an analytics pipeline (Kafka -> data warehouse).

This data enables:

• Rollout monitoring: track the percentage of users receiving each variation over time. Detect if a rollout is not progressing as expected.
• Impact analysis: join flag evaluation data with business metrics. Did enabling the new checkout flow increase conversion rate?
• A/B testing: when a flag has two or more variations, use statistical methods (chi-squared test, Bayesian analysis) to determine which variation performs better.

Keep the evaluation logger asynchronous and batched to avoid adding latency to the evaluation hot path.

Audit Trail

Every flag change (creation, rule update, toggle, deletion) must be recorded in an immutable audit log. Store the full before-and-after state of the flag, the actor who made the change, the timestamp, and an optional comment.

The audit trail serves both operational and compliance purposes. When an incident occurs, engineers can trace exactly when a flag was changed and by whom. For regulated industries, the audit log provides evidence of change control processes.

Scaling Considerations

• SDK-side evaluation: because evaluation is local, the control plane does not need to handle per-request traffic. It only handles SDK sync requests, which are orders of magnitude fewer.
• Edge caching: place a CDN in front of the flag configuration endpoint. SDKs fetch from the CDN, and the control plane only needs to invalidate the CDN cache on flag changes.
• Sharding the flag store: at very large scale (hundreds of thousands of flags), partition the flag store by project or environment.
• Multi-environment support: maintain separate flag configurations for development, staging, and production. Changes can be promoted through environments with approval workflows.

Failure Modes

• Control plane down: SDKs continue evaluating with their cached configuration. No impact on flag evaluation.
• SDK cache corruption: fall back to application-defined defaults for each flag. Ensure every evaluate() call includes a sensible default.
• Network partition: SDKs on one side of the partition keep their cached configuration. When the partition heals, they catch up via delta sync.

The system is designed so that the control plane is not in the critical path of any user-facing request.

Summary

A feature flag service separates the control plane (flag management, rules, segments) from the evaluation plane (local SDK evaluation using cached configuration). The critical design decisions are the evaluation algorithm (deterministic hash-based bucketing for percentage rollouts), the propagation mechanism (SSE for near-real-time updates with polling as fallback), and the caching strategy (local persistent cache for resilience). This architecture makes flag evaluation sub-millisecond and independent of control plane availability.