Icon providers.
===============
elasticsearch
===============
.. contents::
:local:
Elasticsearch documentation
===========================
https://amsterdam.luminis.eu/2016/10/18/elasticsearch-5-is-coming-what-is-new-and-improved/
New features of ES 5.
Releases
========
https://github.com/elastic/elasticsearch/releases
Git releases & tags.
https://www.elastic.co/support/eol
Elastic Product End of Life Dates.
Installing & configuring
========================
https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html
Install Elasticsearch with Debian Package.
Basic config
============
Common cluster name inside given intranet::
cluster.name: mycluster
Unique node name within claster::
node.name: "node1"
Node types::
node.master: true
node.ingest: true
node.data: true
Network interfaces to bind to::
network.host: [_local_, node1.example.com]
Port definitions::
http.port : 9200
tcp.port : 9300
Override default locations::
path.data: /path/to/data1,/path/to/data2
path.logs: /path/to/logs
path.plugins: /path/to/plugins
Explicit list of seed nodes in cluster::
discovery.zen.ping.unicast.hosts: ["master1.example.com", "master2.example.com:9300"]
Dump applied configs::
GET /_cluster/settings?include_defaults=true
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html
Node types.
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-zen.html
Discovery settings.
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-transport.html
Transport definitions.
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html
Configuring Elasticsearch » HTTP
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html
Configuring Elasticsearch » Network settings
REST syntax conventions
=======================
To get data in table form use ``/_cat`` endpoint::
GET /_cat/nodes
To pretty print output append query::
GET /_cat/nodes?pretty=1
Get base information
====================
Cluster health::
GET /_cat/health?v
GET /_cluster/health?pretty
GET /_cluster/health?pretty&level=cluster
List of nodes in cluster (ip, RAM, CPU)::
GET /_cat/nodes?v
GET /_cat/master?v
watch -d curl -s 'localhost:9200/_cat/nodes?v'
List cluster state::
GET /_cluster/state?pretty
GET /_cluster/allocation/explain
List of tasks executed in cluster::
GET /_cat/tasks?v
GET /_cat/tasks?detailed
GET _tasks
List of indexes (status, health, size)::
GET /_cat/indices
GET /_cat/indices?v
GET /_cat/indices?v&s=index
GET /_cluster/health?pretty&level=indices
List of shards::
GET /_cat/shards?v
List of mappings across all indexes::
GET /_mapping
GET /_all/_mapping
GET /twitter,kimchy/_mapping
https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-mapping.html
Get Mapping.
List of shards::
GET /_cluster/health?pretty&level=shards
https://www.elastic.co/guide/en/elasticsearch/reference/current/_cluster_health.html
Cluster Health.
https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-allocation-explain.html
Cluster Allocation Explain API
https://www.elastic.co/guide/en/elasticsearch/reference/current/_list_all_indices.html
List All Indices.
https://www.elastic.co/guide/en/elasticsearch/reference/current/cat.html
cat APIs.
https://www.elastic.co/guide/en/elasticsearch/reference/current/tasks.html
Task Management API.
Managing indexes
================
Assign number of shards and replicas::
curl -XPUT -d '{settings: {index: "number_of_shards": 3, "number_of_replicas": 1}}'
https://www.elastic.co/guide/en/elasticsearch/reference/current/_delete_an_index.html
Delete an Index.
https://github.com/elastic/curator
Manage/archive indices.
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/about.html
Elasticsearch Curator helps you curate, or manage, your Elasticsearch indices and snapshots.
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/about-features.html
Curator allows for many different operations to be performed to both indices and snapshots.
https://www.elastic.co/guide/en/elasticsearch/guide/current/retiring-data.html
Retiring Data.
Lucene search syntax
====================
``TERM1 TERM2`` is same as ``TERM1 OR TERM2``.
``"TERM1 TERM2"`` is for phrase.
``"TERM1 TERM2"~5`` is for proximity.
``TERM~0.8`` is for fuzzy.
``*`` is for wildcard.
Boosting is done with ``^N`` syntax, like ``TERM1^10 TERM2``.
Range with ``[2017-01-01 TO 2017-02-29]``.
Logical ``AND``, ``OR``, ``NOT`` and parentheses for grouping.
Fields are set before colon, like ``FIELD:TERM``.
https://www.elastic.co/guide/en/elasticsearch/reference/master/query-dsl-query-string-query.html
Query String Query
https://www.elastic.co/guide/en/elasticsearch/reference/master/query-dsl-simple-query-string-query.html
Simple Query String Query
ES Query DSL
============
``_exists_:<field>`` for testing field existence.
.. note:: ``_missing_:<field>`` was removed from Kibana 5.x, use ``NOT _exists_:<field>``.
``-<field>:<val>`` or ``-<field>:"<val>"`` for excluding field value.
``+<field>:<val>`` or ``+<field>:"<val>"`` for including field value.
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html
Exists Query
Performance
===========
https://www.elastic.co/guide/en/elasticsearch/reference/5.5/search-profile.html
Profile API.
https://www.elastic.co/guide/en/elasticsearch/reference/current/_explain_analyze.html
Explain Analyze.
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-explain.html
Explain API.
https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-disk-usage.html
Tune for disk usage.
https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-indexing-speed.html
Tune for indexing speed.
https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-search-speed.html
Tune for search speed.
https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster
How many shards should I have in my Elasticsearch cluster?
https://www.elastic.co/guide/en/elasticsearch/reference/master/heap-size.html
Setting the heap size.
Storage requirements
====================
https://www.elastic.co/blog/elasticsearch-storage-the-true-story
The true story behind Elasticsearch storage requirements (2015).
https://www.elastic.co/blog/elasticsearch-storage-the-true-story-2.0
Part 2.0: The true story behind Elasticsearch storage requirements (2015).
https://www.elastic.co/blog/minimize-index-storage-size-elasticsearch-6-0
Space Saving Improvements in Elasticsearch 6.0 (2017).
https://www.elastic.co/blog/filebeat-modiles-access-logs-and-elasticsearch-storage-requirements
Filebeat modules, access logs and Elasticsearch storage requirements.
JSON search syntax
==================
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-from-size.html
Search results pagination.
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html
Query String Query.
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html
Aggregation.
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html
Sort.
Alerting
========
https://github.com/Yelp/elastalert
Easy & Flexible Alerting With Elasticsearch.
http://elastalert.readthedocs.io/en/latest/elastalert.html
Easy & Flexible Alerting With Elasticsearch.
https://github.com/sirensolutions/sentinl/wiki/SENTINL-Alerts-in-Dashboard
SENTINL Alerts in Dashboard.
https://github.com/sirensolutions/sentinl/wiki/SENTINL-Config-Example
SENTINL Config Example
https://github.com/sirensolutions/sentinl/wiki/SENTINL-Tutorial
SENTINL Tutorial
https://github.com/sirensolutions/sentinl/wiki/SENTINL-Watcher-Anatomy
SENTINL Watcher Anatomy
https://github.com/sirensolutions/sentinl/wiki/SENTINL-Watcher-Examples
SENTINL Watcher Examples