Saga Pattern for Distributed Transactions

In a monolithic application, placing an order is a single database transaction: deduct inventory, charge the payment, and create the order record. If any step fails, the database rolls everything back. In a microservices architecture, inventory lives in one service, payments in another, and orders in a third. Each has its own database. There is no single transaction that spans all three. The saga pattern solves this problem by breaking a distributed transaction into a sequence of local transactions, each with a compensating action for rollback.

Why Not Distributed Transactions?

Two-phase commit (2PC) is the traditional answer to distributed transactions. A coordinator asks all participants to prepare, then tells them all to commit. But 2PC has serious problems in microservices:

• Availability: if the coordinator crashes between prepare and commit, all participants are locked waiting.
• Latency: all participants must hold locks until the coordinator decides. This serializes work across services.
• Coupling: every service must support the 2PC protocol, which most modern databases and message brokers do not.

Sagas trade strong consistency for availability and partition tolerance, aligning with real-world microservices constraints.

The Saga Concept

A saga is a sequence of local transactions T1, T2, …, Tn, where each Ti has a compensating transaction Ci that semantically undoes Ti’s effect.

Forward flow (success):
  T1 (Create Order) -> T2 (Reserve Inventory) -> T3 (Process Payment) -> T4 (Confirm Order)

Compensating flow (T3 fails):
  T3 fails -> C2 (Release Inventory) -> C1 (Cancel Order)

Each step commits independently. If step 3 fails, the saga runs compensating transactions for steps 2 and 1 in reverse order. The system reaches a consistent state, just not through an atomic rollback.

Choreography-Based Sagas

In choreography, there is no central coordinator. Each service listens for events and decides what to do next.

┌──────────┐   OrderCreated   ┌───────────┐  InventoryReserved  ┌──────────┐
│  Order   │ ───────────────> │ Inventory │ ──────────────────> │ Payment  │
│ Service  │                  │  Service  │                     │ Service  │
└──────────┘                  └───────────┘                     └──────────┘
     ^                              ^                                │
     │                              │                                │
     │    OrderCancelled            │   InventoryReleased            │
     │ <───────────────────────     │ <────────────────────          │
     │         (on failure)         │       (on failure)             │
     │                              │                                │
     │                              │          PaymentProcessed      │
     └──────────────────────────────┴────────────────────────────────┘

Each service publishes an event after completing its local transaction. The next service in the chain reacts to that event.

# Inventory Service - event handler
async def handle_order_created(event):
    order_id = event["order_id"]
    items = event["items"]

    try:
        reserve_inventory(items)
        publish_event("InventoryReserved", {
            "order_id": order_id,
            "items": items
        })
    except InsufficientStockError:
        publish_event("InventoryReservationFailed", {
            "order_id": order_id,
            "reason": "insufficient_stock"
        })

# Payment Service - event handler
async def handle_inventory_reserved(event):
    order_id = event["order_id"]

    try:
        charge_payment(order_id)
        publish_event("PaymentProcessed", {"order_id": order_id})
    except PaymentDeclinedError:
        publish_event("PaymentFailed", {"order_id": order_id})

Choreography trade-offs

Advantages: no single point of failure, services are loosely coupled, easy to add new steps.

Disadvantages: the flow is implicit and scattered across services. Debugging “why did this order fail?” requires correlating events across multiple service logs. As the number of steps grows, the web of event dependencies becomes difficult to reason about.

Orchestration-Based Sagas

In orchestration, a central saga orchestrator defines the sequence and tells each service what to do.

                    ┌─────────────────┐
                    │ Saga            │
                    │ Orchestrator    │
                    └───────┬─────────┘
                            │
           ┌────────────────┼────────────────┐
           │                │                │
     ┌─────┴─────┐   ┌─────┴─────┐   ┌─────┴─────┐
     │  Order    │   │ Inventory │   │ Payment   │
     │  Service  │   │  Service  │   │  Service  │
     └───────────┘   └───────────┘   └───────────┘

The orchestrator maintains a state machine for each saga instance:

class OrderSagaOrchestrator:
    def __init__(self, order_id):
        self.order_id = order_id
        self.state = "STARTED"

    async def execute(self):
        # Step 1: Create order
        self.state = "ORDER_PENDING"
        await order_service.create_order(self.order_id)
        self.state = "ORDER_CREATED"

        # Step 2: Reserve inventory
        self.state = "INVENTORY_PENDING"
        try:
            await inventory_service.reserve(self.order_id)
            self.state = "INVENTORY_RESERVED"
        except Exception:
            await self.compensate_from("ORDER_CREATED")
            return

        # Step 3: Process payment
        self.state = "PAYMENT_PENDING"
        try:
            await payment_service.charge(self.order_id)
            self.state = "PAYMENT_PROCESSED"
        except Exception:
            await self.compensate_from("INVENTORY_RESERVED")
            return

        # Step 4: Confirm
        self.state = "COMPLETED"
        await order_service.confirm(self.order_id)

    async def compensate_from(self, state):
        if state >= "INVENTORY_RESERVED":
            await inventory_service.release(self.order_id)
        if state >= "ORDER_CREATED":
            await order_service.cancel(self.order_id)
        self.state = "COMPENSATED"

Orchestration trade-offs

Advantages: the entire flow is visible in one place, easier to debug, straightforward to add timeout and retry logic.

Disadvantages: the orchestrator is a single point of coordination (though not necessarily a single point of failure if it is itself resilient). It also creates tighter coupling between the orchestrator and each service.

Designing Compensating Transactions

Compensating transactions are the hardest part of the saga pattern. They do not “undo” in the database sense; they apply a semantic reversal.

Key principles:

Compensations must be idempotent. A compensation might be retried if the first attempt fails or if the system is unsure whether it succeeded.

Compensations cannot fail permanently. If a compensation fails, you have an inconsistent system. Design compensations to always eventually succeed, using retries with exponential backoff.

Some actions cannot be compensated. Sending an email, charging a credit card, or shipping a package cannot be undone. These are called “pivot transactions” and should be placed as late as possible in the saga.

Saga step ordering strategy:

  Compensatable steps first:
    T1: Create order (cancel order)
    T2: Reserve inventory (release inventory)

  Pivot transaction (point of no return):
    T3: Charge payment

  Non-compensatable steps last:
    T4: Send confirmation email
    T5: Notify warehouse

Once T3 succeeds, the saga is committed and the remaining steps must succeed (with retries if necessary).

Handling Failures in the Orchestrator

The orchestrator itself can crash. To handle this, persist the saga state:

CREATE TABLE saga_instances (
    saga_id      UUID PRIMARY KEY,
    saga_type    VARCHAR(100),
    current_step VARCHAR(100),
    state        VARCHAR(50),
    payload      JSONB,
    created_at   TIMESTAMPTZ,
    updated_at   TIMESTAMPTZ
);

On startup, the orchestrator queries for incomplete sagas and resumes them. Each step transition is written to the database before the step executes, making the orchestrator crash-recoverable.

Saga vs Two-Phase Commit

Aspect	Saga	2PC
Consistency	Eventual	Strong
Availability	High	Low (coordinator blocks)
Lock duration	None (local commits)	Long (until all commit)
Complexity	Compensating logic	Protocol complexity
Failure modes	Partial completion possible	All-or-nothing
Scalability	Scales well	Poor across many participants

Choreography vs Orchestration Decision Guide

Factor	Choreography	Orchestration
Number of steps	2-4 steps	4+ steps
Team structure	Separate teams per service	Single team owns the flow
Debugging	Distributed tracing required	Centralized saga log
Adding new steps	Easy (new subscriber)	Requires orchestrator change
Timeout handling	Each service manages its own	Centralized timeout logic

Key Takeaways

The saga pattern replaces distributed transactions with a sequence of local transactions and compensating actions. Choose choreography when your saga is simple and you want loose coupling. Choose orchestration when the flow is complex and you need centralized visibility. In both cases, make your compensations idempotent, order your steps to put irrevocable actions last, and persist saga state so you can recover from crashes. Sagas do not give you ACID, but they give you eventual consistency that works at microservices scale.