This section will introduce the Elasticsearch-Logstash-Kibana (ELK) stack and explain how to set it up on Mesos while also discussing the problems commonly encountered during the setup process.
The ELK stack, a combination of Elasticsearch, Logstash, and Kibana, is an end-to-end solution for log analytics. Elasticsearch provides search capabilities, Logstash is a log management software, while Kibana serves as the visualization layer. The stack is commercially backed by a company called Elastic.
Elasticsearch is a Lucene-based open source distributed search engine designed for high scalability and fast search query response time. It simplifies the usage of Lucene, a highly performant search engine library, by providing a powerful REST API on top. Some of the important concepts in Elasticsearch are highlighted as follows: