===============
 elasticsearch
===============
.. contents::
   :local:

Elasticsearch documentation
===========================

https://amsterdam.luminis.eu/2016/10/18/elasticsearch-5-is-coming-what-is-new-and-improved/
  New features of ES 5.

REST syntax conventions
=======================

To get data in table form use ``/_cat`` endpoint::

  GET /_cat/nodes

To pretty print output append query::

  GET /_cat/nodes?pretty=1

Get base information
====================

Cluster health::

  GET /_cat/health?v
  GET /_cluster/health?pretty
  GET /_cluster/health?pretty&level=cluster

List of nodes in cluster (ip, RAM, CPU)::

  GET /_cat/nodes?v
  GET /_cat/master?v

  watch -d curl -s 'localhost:9200/_cat/nodes?v'

List cluster state::

  GET /_cluster/state?pretty

List of tasks executed in cluster::

  GET /_cat/tasks?v
  GET /_cat/tasks?detailed
  GET _tasks

List of indexes (status, health, size)::

  GET /_cat/indices
  GET /_cat/indices?v
  GET /_cat/indices?v&s=index
  GET /_cluster/health?pretty&level=indices

List of mappings across all indexes::

  GET /_mapping
  GET /_all/_mapping

List of shards::

  GET /_cluster/health?pretty&level=shards

https://www.elastic.co/guide/en/elasticsearch/reference/current/_cluster_health.html
  Cluster Health.
https://www.elastic.co/guide/en/elasticsearch/reference/current/_list_all_indices.html
  List All Indices.
https://www.elastic.co/guide/en/elasticsearch/reference/current/cat.html
  cat APIs.
https://www.elastic.co/guide/en/elasticsearch/reference/current/tasks.html
  Task Management API.

Managing indexes
================

Assign number of shards and replicas::

  curl -XPUT -d '{settings: {index: "number_of_shards": 3, "number_of_replicas": 1}}'

https://www.elastic.co/guide/en/elasticsearch/reference/current/_delete_an_index.html
  Delete an Index.
https://github.com/elastic/curator
  Manage/archive indices.
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/about.html
  Elasticsearch Curator helps you curate, or manage, your Elasticsearch indices and snapshots.
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/about-features.html
  Curator allows for many different operations to be performed to both indices and snapshots.

https://www.elastic.co/guide/en/elasticsearch/guide/current/retiring-data.html
  Retiring Data.

Lucene search syntax
====================

``TERM1 TERM2`` is same as ``TERM1 OR TERM2``.

``"TERM1 TERM2"`` is for phrase.

``"TERM1 TERM2"~5`` is for proximity.

``TERM~0.8`` is for fuzzy.

``*`` is for wildcard.

Boosting is done with ``^N`` syntax, like ``TERM1^10 TERM2``.

Range with ``[2017-01-01 TO 2017-02-29]``.

Logical ``AND``, ``OR``, ``NOT`` and parentheses for grouping.

Fields are set before colon, like ``FIELD:TERM``.

https://www.elastic.co/guide/en/elasticsearch/reference/master/query-dsl-query-string-query.html
  Query String Query
https://www.elastic.co/guide/en/elasticsearch/reference/master/query-dsl-simple-query-string-query.html
  Simple Query String Query

ES Query DSL
============

``_exists_:<field>`` for testing field existence.

.. note:: ``_missing_:<field>`` was removed from Kibana 5.x, use ``NOT _exists_:<field>``.

``-<field>:<val>`` or ``-<field>:"<val>"`` for excluding field value.

``+<field>:<val>`` or ``+<field>:"<val>"`` for including field value.

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html
  Exists Query


Performance
===========

https://www.elastic.co/guide/en/elasticsearch/reference/5.5/search-profile.html
  Profile API.
https://www.elastic.co/guide/en/elasticsearch/reference/current/_explain_analyze.html
  Explain Analyze.
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-explain.html
  Explain API.

https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-disk-usage.html
  Tune for disk usage.
https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-indexing-speed.html
  Tune for indexing speed.
https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-search-speed.html
  Tune for search speed.
https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster
  How many shards should I have in my Elasticsearch cluster?
https://www.elastic.co/blog/filebeat-modiles-access-logs-and-elasticsearch-storage-requirements
  Filebeat modules, access logs and Elasticsearch storage requirements.
https://www.elastic.co/guide/en/elasticsearch/reference/master/heap-size.html
  Setting the heap size.

JSON search syntax
==================

https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-from-size.html
  Search results pagination.
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html
  Query String Query.
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html
  Aggregation.
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html
  Sort.

Alerting
========

https://github.com/Yelp/elastalert
  Easy & Flexible Alerting With Elasticsearch.
http://elastalert.readthedocs.io/en/latest/elastalert.html
  Easy & Flexible Alerting With Elasticsearch.
https://github.com/sirensolutions/sentinl/wiki/SENTINL-Alerts-in-Dashboard
  SENTINL Alerts in Dashboard.
https://github.com/sirensolutions/sentinl/wiki/SENTINL-Config-Example
  SENTINL Config Example
https://github.com/sirensolutions/sentinl/wiki/SENTINL-Tutorial
  SENTINL Tutorial
https://github.com/sirensolutions/sentinl/wiki/SENTINL-Watcher-Anatomy
  SENTINL Watcher Anatomy
https://github.com/sirensolutions/sentinl/wiki/SENTINL-Watcher-Examples
  SENTINL Watcher Examples