Return to site

Useful Elasticsearch API URLs

ELK Stack URL Cheat Sheet

Elasticsearch is a very powerful system, with a rich set of easy-to-understand REST APIs, all accessible via HTTP calls, from cURL, Kibana, or other tools. The details and options can get a little complex so this blog is a list of what we've found most useful over the years.

REST URL & Options Overview

Calling REST APIs are easy, and have four parts in addition to the actual hostname/IP and port:

  • Verb - the GET, PUT, POST, and DELETE part of REST calls.  In Kibana you can specify these directly, and for cURL, use the -X option.
  • Path - The API and path, such as /_cluster/health or /logstash-cron-2020.07.03/_mapping - The first part of the path is usually an index name, unless it starts with an _.
  • Arguments - Various options after the ? such as ?h or ?v
  • Body - Some calls need a JSON body, such as setting options, and will be enclosed in { }

Common Options

There are some common options that work on many, but certainly not all URLs. These are:

  • ?help - The help option will show you the fields available from an API, in a nice list with both short and long names, description, etc.  Each API call has a set of defaults, but this will show you all the other things you can use, which can be very helpful in getting specific items. 
  • ?h= - The 'h' specifies which fields to include in the result, using the short or long names from the 'help' display, above.  These are comma-separated, no spaces.
  • ?v - The 'v' includes field names at the top of the reply; generally very helpful for anything you'll read with your own eyes, but not helpful when your code is calling the API.
  • ?s - The 's' is for sorting, using the fields you list as sort keys.
So for example, we may see this node list, with headings, a list of fields, and sorted by name:
GET /_cat/nodes?v&h=heap.percent,diskUsedPercent,cpu,master,name&s=name

Also, Kibana will save your Dev Tools queries in a cookie, so you can load up all the useful ones there to use from time to time, including with comments starting with # (so they are not marked as errors).

Status & Stats

Often the most useful API calls are about the cluster health, status, and statistics, such as:

  • GET / - The most basic, shows you version, and simple in cURL, to show the node is up, get the version, and cluster name.  
  • GET /_cluster/health - The next easiest to use and most important as tells you about your cluster, including the name and all-important status, plus some stats on nodes, shards, etc.
  • GET /_cat/nodes?v&h=heap.percent,diskUsedPercent,cpu,load_1m,master,name&s=name - Very useful node list, showing HEAP and disk use, CPU/load, and master role - I use this to keep an eye on load and disk use, plus who the master is.
  • GET /_cluster/stats - Deeper overview of the cluster's statistics, with indices, docs, caches, segments, nodes, etc.  Helpful in basic troubleshooting.
  • GET /_nodes/stats - More stats, as a node level, including heap use, OS, and lots more counters.
Threads
  • GET /_cat/thread_pool?v&h=node_name,name,type,active,size,queue,queue_size,rejected,largest,completed,min,max&s=node_name,name - Long, but useful view of the thread queues
  • GET /_nodes/hot_threads/ - Shows hot threads if you are looking at high CPU or slow systems
  • GET /_cat/count/logstash*?v - A fast way to get total doc counts for an index pattern, such as all this logstash* indices.
Indices
  • GET /_cat/indices/logstash*?v - Nice wildcard index list with sizes, doc counts, status, etc.
  • GET /_cat/indices/logstash-*?v&h=index,ss&s=ss:desc - Find your largest indexes by size
  • GET /_cat/indices?v&health=yellow - Get the yellow indices; can use for red & green, too.

Settings

There are many settings in Elasticsearch, but the most common are at the Cluster and Index levels:

Cluster Settings

  • GET /_cluster/settings - The most basic view, showing non-default persistent and transient settings.
  • GET /_cluster/settings?include_defaults=true&flat_settings=true - Much bigger list including all defaults, and using the flat view to make it easier to read.

Index Settings

  • GET /logstash-cron-2020.08.03/_settings - Basic settings list for a given index.
  • GET /logstash-cron-2020.08.03 - Get the basic info and mappings (same as /index/_mapping)

Closing & Deleting Indices

  • POST /logstash-*-2020.03.*/_close - Closing indices, with wildcards
  • DELETE /logstash-*-2020.04.* - Deleting indices, being careful, of course.

Troubleshooting

  • GET /_cat/shards?v&h=n,index,shard,prirep,state,sto,sc,unassigned.reason,unassigned.details&s=sto,index - Allocated and unallocated shards
  • GET _cat/shards?v&h=index,shard,prirep,state,unassigned.reason&s=state - See unassigned and why

  • GET /_cluster/allocation/explain - Get explain for latest/last unallocated

  • GET /_cluster/allocation/explain { "index": "logstash-cloudtrail-2019.10.16", "shard": 0, "primary": true} - Get specific shard allocation details

Templates

  • GET /_cat/templates?v&s=order,name - List your templates
  • GET /_template/logstash - Get template by name

Snapshot / Backups

  • GET /_snapshot - Lists snapshot repositories configured in the system.  You need these names for most other API calls.
  • GET /_snapshot/s3_repository/_all?verbose=false - Lists every snapshot and the indices in each. Usually a very long list.  s3_repository is the repo name.
  • GET /_snapshot/_status - Get status and stats of any running snapshot
  • GET /_cat/snapshots/s3_repository?v&h=id,status,start_time,duration,indicies,successful_shards,failed_shards,total_shards - Useful list of snapshots with helpful information.

Feel free to send more suggestions and useful URLs.

More about ELKman at www.ELKman.io - the only commercial ELK Stack Management Tool

All Posts
×

Almost done…

We just sent you an email. Please click the link in the email to confirm your subscription!

OK

Copyright 2020 by Wapiti Systems, Inc.
ELK Manager is not affiliated in any way with ElasticSearch BV.
ElasticSearch®, Logstash®, Kibana® and Beats® are registered trademarks of Elasticsearch BV.