Useful Elasticsearch API URLs

Elasticsearch is a very powerful system, with a rich set of easy-to-understand REST APIs, all accessible via HTTP calls, from cURL, Kibana, or other tools. The details and options can get a little complex so this blog is a list of what we've found most useful over the years.

Calling REST APIs are easy, and have four parts in addition to the actual hostname/IP and port:

• Verb - the GET, PUT, POST, and DELETE part of REST calls.  In Kibana you can specify these directly, and for cURL, use the -X option.
• Path - The API and path, such as /_cluster/health or /logstash-cron-2020.07.03/_mapping - The first part of the path is usually an index name, unless it starts with an _.
• Arguments - Various options after the ? such as ?h or ?v
• Body - Some calls need a JSON body, such as setting options, and will be enclosed in { }

Common Options

There are some common options that work on many, but certainly not all URLs. These are:

• ?help - The help option will show you the fields available from an API, in a nice list with both short and long names, description, etc.  Each API call has a set of defaults, but this will show you all the other things you can use, which can be very helpful in getting specific items. 
• ?h= - The 'h' specifies which fields to include in the result, using the short or long names from the 'help' display, above.  These are comma-separated, no spaces.
• ?v - The 'v' includes field names at the top of the reply; generally very helpful for anything you'll read with your own eyes, but not helpful when your code is calling the API.
• ?s - The 's' is for sorting, using the fields you list as sort keys.

Also, Kibana will save your Dev Tools queries in a cookie, so you can load up all the useful ones there to use from time to time, including with comments starting with # (so they are not marked as errors).

Often the most useful API calls are about the cluster health, status, and statistics, such as:

• GET / - The most basic, shows you version, and simple in cURL, to show the node is up, get the version, and cluster name.  
• GET /_cluster/health - The next easiest to use and most important as tells you about your cluster, including the name and all-important status, plus some stats on nodes, shards, etc.
• GET /_cat/nodes?v&h=heap.percent,diskUsedPercent,cpu,load_1m,master,name&s=name - Very useful node list, showing HEAP and disk use, CPU/load, and master role - I use this to keep an eye on load and disk use, plus who the master is.
• GET /_cluster/stats - Deeper overview of the cluster's statistics, with indices, docs, caches, segments, nodes, etc.  Helpful in basic troubleshooting.
• GET /_nodes/stats - More stats, as a node level, including heap use, OS, and lots more counters.

• GET /_cat/thread_pool?v&h=node_name,name,type,active,size,queue,queue_size,rejected,largest,completed,min,max&s=node_name,name - Long, but useful view of the thread queues
• GET /_nodes/hot_threads/ - Shows hot threads if you are looking at high CPU or slow systems
• GET /_cat/count/logstash*?v - A fast way to get total doc counts for an index pattern, such as all this logstash* indices.

• GET /_cat/indices/logstash*?v - Nice wildcard index list with sizes, doc counts, status, etc.
• GET /_cat/indices/logstash-*?v&h=index,ss&s=ss:desc - Find your largest indexes by size
• GET /_cat/indices?v&health=yellow - Get the yellow indices; can use for red & green, too.

There are many settings in Elasticsearch, but the most common are at the Cluster and Index levels:

Cluster Settings

• GET /_cluster/settings - The most basic view, showing non-default persistent and transient settings.
• GET /_cluster/settings?include_defaults=true&flat_settings=true - Much bigger list including all defaults, and using the flat view to make it easier to read.

Index Settings

• GET /logstash-cron-2020.08.03/_settings - Basic settings list for a given index.
• GET /logstash-cron-2020.08.03 - Get the basic info and mappings (same as /index/_mapping)

Closing & Deleting Indices

• POST /logstash-*-2020.03.*/_close - Closing indices, with wildcards
• DELETE /logstash-*-2020.04.* - Deleting indices, being careful, of course.

Troubleshooting

• GET /_cat/shards?v&h=n,index,shard,prirep,state,sto,sc,unassigned.reason,unassigned.details&s=sto,index - Allocated and unallocated shards

• GET _cat/shards?v&h=index,shard,prirep,state,unassigned.reason&s=state - See unassigned and why

• GET /_cluster/allocation/explain - Get explain for latest/last unallocated

• GET /_cluster/allocation/explain { "index": "logstash-cloudtrail-2019.10.16", "shard": 0, "primary": true} - Get specific shard allocation details

Templates

• GET /_cat/templates?v&s=order,name - List your templates
• GET /_template/logstash - Get template by name

• GET /_snapshot - Lists snapshot repositories configured in the system.  You need these names for most other API calls.
• GET /_snapshot/s3_repository/_all?verbose=false - Lists every snapshot and the indices in each. Usually a very long list.  s3_repository is the repo name.
• GET /_snapshot/_status - Get status and stats of any running snapshot
• GET /_cat/snapshots/s3_repository?v&h=id,status,start_time,duration,indicies,successful_shards,failed_shards,total_shards - Useful list of snapshots with helpful information.

Feel free to send more suggestions and useful URLs.

More about ELKman at www.ELKman.io - the only commercial ELK Stack Management Tool