ElasticSearch Curator Short Guide

Elasticsearch Curate Features

Curate, or manage your Elasticsearch indices
  • Alias Management – add, remove
  • Shard routing allocation
  • Indices Management – Close , Delete indices, Open closed indices, Optimize indices and modify number of replicas
  • Snapshot(backups management) – Show, Backup, Restore.
  • Change the number of replicas per shard for indices
  • Pattern Matching for statements (e.g delete all indices matching .marvel*)

 

Example Elasticsearch Curate Use Cases

Automatically/manually

  • Snapshots Index creation
    • All indices except Marvels regex support
  • Restore from snapshots (using the Elasticsearch end point via curl ….. Not with curator currently)
    • All indices in the back up
    • Specific index from the backup
    • Keep cluster state as is
  • Delete indices
    • Older than a date range
    • By a regex matching pattern

Background

  • Started off as clearESindices.py for a simple goal to delete indices.
  • It then became logstash_index_cleaner.py
  • Then moved under logstash repository as “expire_logs”.
  • After Jordan Sissel was hired by Elastic it then became Elasticsearch Curator and is now hosted at https://github.com/elastic/curator
  • Today : Curator now performs many operations on your Elasticsearch indices, from delete to snapshot to shard allocation routing.

 

Curator all-in Command Line

  • Curator 2.0 underlying feature was the first attempt to detach the popular Elasticsearch API from the CLI.
  • These allows scripting to carry out any of the tasks featured above meaning you can for instance:
    • Automate your back up and restore procedures.
  • Also with version 2.0, Curator ships in an API along side the wrapper scripts/ entry points. This API allows you to roll out your own scripts to perform similar or totally different tasks to Curator with the same underlying code that curator uses…
  • Documentation can be found here

 

Curator installation for Mac


# you may ignore the python installations if you already have this
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py

wget https://bootstrap.pypa.io/get-pip.py
sudo python get-pip.py

wget https://pypi.python.org/packages/source/u/urllib3/urllib3-1.8.3.tar.gz
pip install urllib3-1.8.3.tar.gz

wget https://pypi.python.org/packages/source/c/click/click-3.3.tar.gz -O click-3.3.tar.gz
sudo pip install click-3.3.tar.gz

# install elasticsearch-py
wget https://github.com/elastic/elasticsearch-py/archive/1.6.0.tar.gz -O elasticsearch-py.tar.gz
sudo pip install elasticsearch-py.tar.gz

# install elasticsearch-curator
wget https://github.com/elastic/curator/archive/v3.3.0.tar.gz -O elasticsearch-curator.tar.gz
sudo pip install elasticsearch-curator.tar.gz

# To test : Verify version
curator --version
# should echo: "curator, version 3.3.0"

# To example Commands try the help menu
curator --help
# should echo : help menu options

For more installation guides please see the official Elasticsearch Guides

 

Curator Command Line Flags

 

Index and Snapshot Selection

–newer-than

–older-than

–prefix

–suffix

–time-unit

–timestring

–regex

–exclude

Index selection only

–index

–all-indices

Snapshot selection only

–snapshot

–all-snapshots

–repository

 

Leave a Reply