Introduction

Modern applications often need to store data and at the same time publish events.
Example:

• An e-commerce system creates an order in the database.
• At the same moment, it needs to notify a shipping service and a billing service.

If the app writes to the database and Kafka separately, things can go wrong:

• The database write succeeds but the Kafka message fails: event is lost.
• The Kafka message succeeds but the database write fails: inconsistent state.

This problem is called the dual-write problem.
The Outbox Pattern solves it.

What is the Outbox Pattern?

The idea:

1. The application writes its business data and an outbox record in the same database transaction.
2. Debezium monitors the outbox table using Change Data Capture (CDC).
3. Debezium publishes the outbox record to Kafka as an event.
4. Consumers (other services) read the event from Kafka.

Because both business data and outbox record are written atomically, there is no inconsistency.

Outbox Table Example

CREATE TABLE order_outbox (
  id UUID PRIMARY KEY,
  order_id UUID NOT NULL,
  event_type VARCHAR(50) NOT NULL,
  payload JSONB NOT NULL,
  created_at TIMESTAMP DEFAULT now()
);

• order_id: reference to the business entity.
• event_type: e.g., ORDER_CREATED, ORDER_PAID.
• payload: JSON with event details.
• created_at: timestamp for ordering.

Application Flow

sequenceDiagram
  autonumber
  participant App as Order Service
  participant DB as PostgreSQL
  participant DBZ as Debezium Connector
  participant K as Kafka
  participant S as Shipping Service
  participant B as Billing Service

  App->>DB: Insert into orders + order_outbox (same TX)
  DBZ->>DB: Read WAL (change in order_outbox)
  DBZ-->>K: Publish event to Kafka topic
  K-->>S: Deliver event to Shipping Service
  K-->>B: Deliver event to Billing Service

Debezium Connector Config

{
  "name": "ecommerce-order-outbox-connector",
  "config": {
    "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
    "database.hostname": "ecom-prod-db.example.com",
    "database.port": "5432",
    "database.user": "cdc_user",
    "database.password": "******",
    "database.dbname": "ecommerce",
    "database.server.name": "ecommerce",
    "table.include.list": "public.order_outbox",
    "transforms": "unwrap",
    "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState",
    "key.converter": "org.apache.kafka.connect.json.JsonConverter",
    "key.converter.schemas.enable": false,
    "value.converter": "org.apache.kafka.connect.json.JsonConverter",
    "value.converter.schemas.enable": false
  }
}

See this article for production grade connector.