Elasticsearch is a very powerful system, with a rich set of easy-to-understand REST APIs, all accessible via HTTP calls, from cURL, Kibana, or other tools. The details and options can get a little complex so this blog is a list of what we've found most useful over the years.
REST URL & Options Overview
Calling REST APIs are easy, and have four parts in addition to the actual hostname/IP and port:
- Verb - the GET, PUT, POST, and DELETE part of REST calls. In Kibana you can specify these directly, and for cURL, use the -X option.
- Path - The API and path, such as /_cluster/health or /logstash-cron-2020.07.03/_mapping - The first part of the path is usually an index name, unless it starts with an _.
- Arguments - Various options after the ? such as ?h or ?v
- Body - Some calls need a JSON body, such as setting options, and will be enclosed in { }
Common Options
There are some common options that work on many, but certainly not all URLs. These are:
- ?help - The help option will show you the fields available from an API, in a nice list with both short and long names, description, etc. Each API call has a set of defaults, but this will show you all the other things you can use, which can be very helpful in getting specific items.
- ?h= - The 'h' specifies which fields to include in the result, using the short or long names from the 'help' display, above. These are comma-separated, no spaces.
- ?v - The 'v' includes field names at the top of the reply; generally very helpful for anything you'll read with your own eyes, but not helpful when your code is calling the API.
- ?s - The 's' is for sorting, using the fields you list as sort keys.
Also, Kibana will save your Dev Tools queries in a cookie, so you can load up all the useful ones there to use from time to time, including with comments starting with # (so they are not marked as errors).
Status & Stats
Often the most useful API calls are about the cluster health, status, and statistics, such as:
- GET / - The most basic, shows you version, and simple in cURL, to show the node is up, get the version, and cluster name.
- GET /_cluster/health - The next easiest to use and most important as tells you about your cluster, including the name and all-important status, plus some stats on nodes, shards, etc.
- GET /_cat/nodes?v&h=heap.percent,diskUsedPercent,cpu,load_1m,master,name&s=name - Very useful node list, showing HEAP and disk use, CPU/load, and master role - I use this to keep an eye on load and disk use, plus who the master is.
- GET /_cluster/stats - Deeper overview of the cluster's statistics, with indices, docs, caches, segments, nodes, etc. Helpful in basic troubleshooting.
- GET /_nodes/stats - More stats, as a node level, including heap use, OS, and lots more counters.
- GET /_cat/thread_pool?v&h=node_name,name,type,active,size,queue,queue_size,rejected,largest,completed,min,max&s=node_name,name - Long, but useful view of the thread queues
- GET /_nodes/hot_threads/ - Shows hot threads if you are looking at high CPU or slow systems
- GET /_cat/count/logstash*?v - A fast way to get total doc counts for an index pattern, such as all this logstash* indices.
- GET /_cat/indices/logstash*?v - Nice wildcard index list with sizes, doc counts, status, etc.
- GET /_cat/indices/logstash-*?v&h=index,ss&s=ss:desc - Find your largest indexes by size
- GET /_cat/indices?v&health=yellow - Get the yellow indices; can use for red & green, too.
Settings
There are many settings in Elasticsearch, but the most common are at the Cluster and Index levels:
Cluster Settings
- GET /_cluster/settings - The most basic view, showing non-default persistent and transient settings.
- GET /_cluster/settings?include_defaults=true&flat_settings=true - Much bigger list including all defaults, and using the flat view to make it easier to read.
Index Settings
- GET /logstash-cron-2020.08.03/_settings - Basic settings list for a given index.
- GET /logstash-cron-2020.08.03 - Get the basic info and mappings (same as /index/_mapping)
Closing & Deleting Indices
- POST /logstash-*-2020.03.*/_close - Closing indices, with wildcards
- DELETE /logstash-*-2020.04.* - Deleting indices, being careful, of course.
Troubleshooting
- GET /_cat/shards?v&h=n,index,shard,prirep,state,sto,sc,unassigned.reason,unassigned.details&s=sto,index - Allocated and unallocated shards
GET _cat/shards?v&h=index,shard,prirep,state,unassigned.reason&s=state - See unassigned and why
GET /_cluster/allocation/explain - Get explain for latest/last unallocated
GET /_cluster/allocation/explain { "index": "logstash-cloudtrail-2019.10.16", "shard": 0, "primary": true} - Get specific shard allocation details
Templates
- GET /_cat/templates?v&s=order,name - List your templates
- GET /_template/logstash - Get template by name
Snapshot / Backups
- GET /_snapshot - Lists snapshot repositories configured in the system. You need these names for most other API calls.
- GET /_snapshot/s3_repository/_all?verbose=false - Lists every snapshot and the indices in each. Usually a very long list. s3_repository is the repo name.
- GET /_snapshot/_status - Get status and stats of any running snapshot
- GET /_cat/snapshots/s3_repository?v&h=id,status,start_time,duration,indicies,successful_shards,failed_shards,total_shards - Useful list of snapshots with helpful information.
Feel free to send more suggestions and useful URLs.
More about ELKman at www.ELKman.io - the only commercial ELK Stack Management Tool