Skip to content

Widhian Bramantya

coding is an art form

Menu
  • About Me
Menu
postgresql

PostgreSQL Write-Ahead Log (WAL): Durability, Performance Tuning, and Recovery Explained

Posted on October 8, 2025October 8, 2025 by admin

PostgreSQL is known for its reliability, it keeps your data safe even during crashes or power failures.
The secret behind this reliability is a core feature called WAL (Write-Ahead Log).

In this article, we’ll explore how WAL works, how to tune it for performance, how it supports recovery, and how to monitor it effectively.

What is WAL (Write-Ahead Log)?

WAL stands for Write-Ahead Log, a mechanism that ensures all data changes are recorded before they are applied to the main database files.

Whenever PostgreSQL executes a transaction:

  1. It first writes the change into the WAL log file on disk.
  2. Only after the WAL is safely stored, it updates the actual data pages.

This rule “write ahead before applying” guarantees that if PostgreSQL crashes, it can replay the WAL to restore the database to a consistent state.

flowchart LR
    A[Client Transaction] --> B[Change in Memory Buffer]
    B --> C[Write Change to WAL on Disk]
    C --> D[Commit Transaction]
    D --> E[Apply Change to Data Files]
    E --> F[Checkpoint Confirms Data Safe]

Explanation

  • Client Transaction: A user sends a write query (INSERT, UPDATE, DELETE).
  • Change in Memory Buffer: PostgreSQL modifies the page in memory (shared buffer).
  • Write Change to WAL on Disk: Before touching the actual data file, it writes the change to the WAL log.
  • Commit Transaction: The transaction is marked committed only after WAL is safely stored.
  • Apply Change to Data Files: Later, background processes (checkpoint or bgwriter) write the actual data to the table files.
  • Checkpoint Confirms Data Safe: When a checkpoint occurs, PostgreSQL ensures all recent changes are safely persisted.
See also  Understanding PostgreSQL WAL, Slot, Publication, LSN, and Replication Lag

Why WAL is Important

FeatureDescription
DurabilityNo committed transaction is lost, WAL ensures every change is logged first.
Crash RecoveryAfter a crash, PostgreSQL replays WAL entries to recover unflushed data.
ReplicationWAL is the foundation for physical and logical replication.
Backup and PITRWAL files can be archived and used to restore to any specific time.

How WAL Works Step-by-Step

Let’s look at what happens during a simple transaction:

  1. A user runs a query: UPDATE users SET name = 'Alice' WHERE id = 1;
  2. PostgreSQL changes the data in memory (shared buffers).
  3. It then writes the change to WAL, before writing the actual table file.
  4. WAL is flushed to disk using fsync() to make sure it’s durable.
  5. Later, a background process called the checkpoint writes all modified pages to the main data files.

So even if the system crashes before the checkpoint, PostgreSQL can replay WAL to recover the missing changes.

WAL File Structure

WAL files are stored in the directory:

$PGDATA/pg_wal/

Each WAL segment file has a fixed size (default 16 MB). When one file fills up, PostgreSQL creates or reuses the next segment in sequence.

You can list them like this:

ls $PGDATA/pg_wal
000000010000000000000001
000000010000000000000002
000000010000000000000003

Each file name encodes:

  • Timeline ID (for backups and recovery)
  • Log segment number

Key WAL Parameters and How to Tune Them

WAL behavior can be tuned for performance and reliability using several key settings in postgresql.conf.

wal_level

Defines how much information is written to WAL.

ValueDescriptionUse Case
minimalBasic logging, no replication supportLocal-only workloads
replicaSupports physical replicationHA setups
logicalSupports logical replication and decodingBDR, data integration

Recommendation: Use replica for HA, or logical if you use logical replication.

See also  PostgreSQL Replication Deep Dive: From High Availability to Multi-Master Clusters

fsync

Controls whether PostgreSQL forces the OS to write data to disk immediately.

SettingEffect
onSafe — PostgreSQL ensures WAL and data are on disk before commit
offFaster, but risky — a crash may cause data loss

Recommendation: Always keep fsync = on in production. Turn it off only for testing or benchmarking.

synchronous_commit

Determines when a transaction is considered “committed.”

ValueBehavior
onWaits until WAL is written to disk (safest)
offReturns success before WAL is flushed (faster but risky)
remote_applyWaits until replica also applies WAL (strongest consistency)

Tip: For balance, use on for important data and off for background logging.

checkpoint_timeout and max_wal_size

A checkpoint is when PostgreSQL flushes all dirty pages (modified data) from memory to disk and marks the point in WAL that’s safe to discard.

ParameterDescription
checkpoint_timeoutMaximum time between checkpoints (default 5 min)
max_wal_sizeMax total WAL before triggering a new checkpoint

Tuning Tips:

  • Increasing max_wal_size reduces how often checkpoints happen, improving performance.
  • But too large means longer recovery time after crash.

Example:

checkpoint_timeout = 10min
max_wal_size = 2GB

Checkpoints and Performance

Each checkpoint writes dirty data pages to disk. Too many checkpoints can slow down the system because of heavy I/O.

Best practices:

  • Avoid small max_wal_size.
  • Use fast disks (SSD or NVMe).
  • Monitor with: SELECT * FROM pg_stat_bgwriter;

Related posts:

Smart Automation in PostgreSQL: Managing Time-Based Data with pg_partman and pg_cron

PostgreSQL Replication Deep Dive: From High Availability to Multi-Master Clusters

Understanding PostgreSQL WAL, Slot, Publication, LSN, and Replication Lag

Pages: 1 2 3
Category: PostgreSQL

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Linkedin

Widhian Bramantya

Recent Posts

  • Smart Automation in PostgreSQL: Managing Time-Based Data with pg_partman and pg_cron
  • Understanding PostgreSQL WAL, Slot, Publication, LSN, and Replication Lag
  • PostgreSQL Write-Ahead Log (WAL): Durability, Performance Tuning, and Recovery Explained
  • PostgreSQL Replication Deep Dive: From High Availability to Multi-Master Clusters
  • Finding Nearby Merchants in a Ride-Hailing App Using Elasticsearch Polygon Search
  • Advanced Text Search in Elasticsearch: N-Gram, Reverse, Fuzzy, and Search-as-you-type
  • Understanding and Customizing Analyzers in Elasticsearch
  • Log Management at Scale: Integrating Elasticsearch with Beats, Logstash, and Kibana
  • Index Lifecycle Management (ILM) in Elasticsearch: Automatic Data Control Made Simple
  • Blue-Green Deployment in Elasticsearch: Safe Reindexing and Zero-Downtime Upgrades
  • Maintaining Super Large Datasets in Elasticsearch
  • Elasticsearch Best Practices for Beginners
  • Implementing the Outbox Pattern with Debezium
  • Production-Grade Debezium Connector with Kafka (Postgres Outbox Example – E-Commerce Orders)
  • Connecting Debezium with Kafka for Real-Time Streaming
  • Debezium Architecture – How It Works and Core Components
  • What is Debezium? – An Introduction to Change Data Capture
  • Offset Management and Consumer Groups in Kafka
  • Partitions, Replication, and Fault Tolerance in Kafka
  • Delivery Semantics in Kafka: At Most Once, At Least Once, Exactly Once

Recent Comments

No comments to show.

Archives

  • October 2025
  • September 2025
  • August 2025
  • November 2021
  • October 2021
  • August 2021
  • July 2021
  • June 2021
  • March 2021
  • January 2021

Categories

  • Debezium
  • Devops
  • ElasticSearch
  • Golang
  • Kafka
  • Lua
  • NATS
  • PostgreSQL
  • Programming
  • RabbitMQ
  • Redis
  • VPC
© 2026 Widhian Bramantya | Powered by Minimalist Blog WordPress Theme