Skip to content

Widhian Bramantya

coding is an art form

Menu
  • About Me
Menu
debezium

What is Debezium? – An Introduction to Change Data Capture

Posted on September 22, 2025October 8, 2025 by admin

Outbox Pattern (Sequence Diagram)

sequenceDiagram
  autonumber
  participant App as Application Service
  participant DB as Database (with Outbox Table)
  participant DBZ as Debezium Connector
  participant K as Kafka
  participant C as Consumer Service

  App->>DB: Write business data (orders)
  App->>DB: Insert outbox event (status=PENDING)
  DBZ->>DB: Read transaction log (binlog/WAL)
  DBZ-->>K: Publish event to topic (outbox.events)
  K-->>C: Deliver event
  C->>C: Process event (e.g., send email, update cache)
  C-->>DB: (optional) Ack/Update status via API
  1. Application Service (App)
    • The application performs a normal business action, for example:
      • “Create a new order”.
    • At the same time, it also writes an outbox event into a special table called the outbox table inside the same database.
    • The outbox event usually has a status like PENDING, which means it is waiting to be processed.
  1. Database (DB with Outbox Table)
    • The database now contains two things:
      1. The business data (like order details).
      2. The outbox event (which describes what happened, e.g., “OrderCreated”).
    • Both are stored in the same transaction, so they are always consistent.
  1. Debezium Connector (DBZ)
    • Debezium monitors the database transaction log (binlog for MySQL, WAL for PostgreSQL, etc.).
    • When it sees the new outbox event row, Debezium captures it.
  1. Apache Kafka (K)
    • Debezium publishes the outbox event into a Kafka topic, for example outbox.events.
    • Kafka acts as a broker, keeping the event safe and allowing many consumers to subscribe to it.
  1. Consumer Service (C)
    • A consumer (another microservice, for example an email service or inventory system) reads the event from Kafka.
    • It processes the event — for example:
      • Send a confirmation email to the customer.
      • Update the product stock.
      • Refresh a cache.
  1. (Optional) Acknowledgment
    • After processing, the consumer can send back an acknowledgment to the application or database.
    • For example, it might update the outbox row from PENDING to SENT or DONE.
    • This step is optional but can be useful for monitoring and reliability.
See also  RabbitMQ vs Kafka: Choosing the Right Messaging System for Your Project

Why Use the Outbox Pattern?

  • Ensures consistency between business data and events (no missing messages).
  • Avoids “dual-write problem” (where DB and Kafka writes could get out of sync).
  • Makes microservices more reliable and event-driven.

Related posts:

Implementing the Outbox Pattern with Debezium

Debezium Architecture – How It Works and Core Components

Connecting Debezium with Kafka for Real-Time Streaming

Pages: 1 2 3
Category: Debezium

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Linkedin

Widhian Bramantya

Recent Posts

  • Smart Automation in PostgreSQL: Managing Time-Based Data with pg_partman and pg_cron
  • Understanding PostgreSQL WAL, Slot, Publication, LSN, and Replication Lag
  • PostgreSQL Write-Ahead Log (WAL): Durability, Performance Tuning, and Recovery Explained
  • PostgreSQL Replication Deep Dive: From High Availability to Multi-Master Clusters
  • Finding Nearby Merchants in a Ride-Hailing App Using Elasticsearch Polygon Search
  • Advanced Text Search in Elasticsearch: N-Gram, Reverse, Fuzzy, and Search-as-you-type
  • Understanding and Customizing Analyzers in Elasticsearch
  • Log Management at Scale: Integrating Elasticsearch with Beats, Logstash, and Kibana
  • Index Lifecycle Management (ILM) in Elasticsearch: Automatic Data Control Made Simple
  • Blue-Green Deployment in Elasticsearch: Safe Reindexing and Zero-Downtime Upgrades
  • Maintaining Super Large Datasets in Elasticsearch
  • Elasticsearch Best Practices for Beginners
  • Implementing the Outbox Pattern with Debezium
  • Production-Grade Debezium Connector with Kafka (Postgres Outbox Example – E-Commerce Orders)
  • Connecting Debezium with Kafka for Real-Time Streaming
  • Debezium Architecture – How It Works and Core Components
  • What is Debezium? – An Introduction to Change Data Capture
  • Offset Management and Consumer Groups in Kafka
  • Partitions, Replication, and Fault Tolerance in Kafka
  • Delivery Semantics in Kafka: At Most Once, At Least Once, Exactly Once

Recent Comments

No comments to show.

Archives

  • October 2025
  • September 2025
  • August 2025
  • November 2021
  • October 2021
  • August 2021
  • July 2021
  • June 2021
  • March 2021
  • January 2021

Categories

  • Debezium
  • Devops
  • ElasticSearch
  • Golang
  • Kafka
  • Lua
  • NATS
  • PostgreSQL
  • Programming
  • RabbitMQ
  • Redis
  • VPC
© 2026 Widhian Bramantya | Powered by Minimalist Blog WordPress Theme