When your Elasticsearch grows very large, managing all indices by hand becomes impossible.
Old data takes space, slows down queries, and increases cost. Index Lifecycle Management (ILM) helps you automate this, deciding when to roll over, move, merge, freeze, or delete indices automatically.
This article explains ILM in simple English, including the frozen phase and how it can even store data in S3 for searchable archiving.
What Is ILM?
ILM (Index Lifecycle Management) is an automatic system that controls the lifecycle of an index.
It defines phases that represent the age of your data — from hot to frozen or deleted.
With ILM you can:
- Create new indices when the old one grows too large (rollover)
- Move older data to cheaper storage nodes
- Merge and compress old indices
- Keep old data searchable on S3 (frozen)
- Delete data automatically after it’s no longer needed
ILM = automated data aging system for Elasticsearch.
Why ILM Is Important
Without ILM:
- Indices grow forever
- Searches get slow
- Cluster uses too much memory and disk
- You must delete old data manually
With ILM:
– Better performance
– Lower cost
– Automatic cleanup
– Easier scaling
ILM Phases Explained
ILM divides an index’s lifetime into five phases:
Phase | Description | Typical Actions | License |
---|---|---|---|
Hot | Active data (frequent writes & queries) | Rollover, set replicas | Basic |
Warm | Read-only data, still queried | Move to warm nodes, shrink, forcemerge | Basic |
Cold | Rarely queried data | Move to cold tier, compress | Basic |
Frozen | Archived data, searchable via S3 or snapshots | Searchable snapshot, minimal cost | Platinum / Enterprise |
Delete | Data no longer needed | Delete index | Basic |
Hot Phase
This is where Elasticsearch actively writes and searches new data.
The index is read-write, and ILM will roll over when it grows large.
Example triggers:
"max_size": "30gb"
"max_age": "7d"
Warm Phase
Data becomes read-only but still useful.
You can move it to warm nodes (slower, cheaper machines).
Actions in this phase:
allocate
: move index to warm nodeforcemerge
: merge segments for faster readsshrink
: reduce shard countset_priority
: lower query priority
Cold Phase
Data is old and rarely accessed.
You can move it to cold nodes with slower disks or cheaper storage.
It’s still searchable but slower.
Actions:
allocate
to cold nodefreeze
(optional): keeps index searchable with very low memory usage- Reduce replicas
Frozen Phase
This is the final searchable archive phase.
The data is moved out of the cluster to an object storage (like Amazon S3, Azure Blob, or Google Cloud Storage) and accessed via a searchable snapshot.
- Data no longer takes local disk space.
- Queries are slower because they read directly from S3.
- Ideal for compliance or historical archives, data that must remain searchable but rarely used.
Example actions:
"frozen": { "min_age": "180d", "actions": { "searchable_snapshot": { "snapshot_repository": "s3_repository" } } }
Note:
- This feature is part of Elastic Stack Platinum or Enterprise license.
- It requires configuring a snapshot repository connected to S3 (or similar).
- Not available in the Basic (free) license.
Delete Phase
Data has no value anymore.
ILM can automatically delete it after a given time to save space.
Example:
"delete": { "min_age": "365d", "actions": { "delete": {} } }