Skip to content

Instantly share code, notes, and snippets.

@Lokey92
Created June 15, 2022 19:31
Show Gist options
  • Save Lokey92/11423cb0af57cb029abb24117dd2d64c to your computer and use it in GitHub Desktop.
Save Lokey92/11423cb0af57cb029abb24117dd2d64c to your computer and use it in GitHub Desktop.
Elasticsearch ILM - Reindexing warm tier data without cycling through the hot phase.
## AUTHOR: Logan Tooley ##
## DATE: 20220615 ##
## SCENARIO ################################################################################################################
# Imagine there's several indices that are on the warm data tier, all managed by an ILM policy.
# An admin wants to roll them into a backup but doesn't want to lose the ILM policy nor run it through the hot phase again.
# In this particular instance we want to roll old filebeat-7.16.2-* indices.
############################################################################################################################
## STEP 1: Create the backup index with the tier preference. This will still start in the hot phase.
## NOTE: It won't be a problem for the write index and it completes rollover automatically (using lifecycle.indexing_complete)
PUT filebeat-7.16.2-backup
{
"settings": {
"index": {
"number_of_shards": 1,
"number_of_replicas": 0,
"lifecycle.name": "filebeat",
"routing.allocation.include._tier_preference": "data_warm",
"lifecycle.rollover_alias": "filebeat-7.16.2",
"lifecycle.indexing_complete": true
}
},
"aliases": {
"filebeat-7.16.2": {
"is_write_index": "false"
}
}
}
# STEP 2: Move the backup to warm using a phase execution.
POST _ilm/move/filebeat-7.16.2-backup
{
"current_step": {
"phase": "hot",
"action": "rollover",
"name": "rollover"
},
"next_step": {
"phase": "warm",
"action": "forcemerge",
"name": "forcemerge"
}
}
# STEP 3: Reindex warm indices to the backup, stays on warm tier and doesn't need to start at hot.
POST _reindex
{
"source": {
"index": ["filebeat-7.16.2-2022.05.11-000002", "filebeat-7.16.2-2022.04.11-000001"]
},
"dest": {
"index": "filebeat-7.16.2-backup"
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment