The ELK stack involves different components that work together to address the challenges cited previously. The core parts of the stack can be described in a nutshell as the following:
- ElasticSearch: Is a scalable and distributed document store. It allows to index data in real time by achieving fast search response. ElasticSearch is designed to scale horizontally and provides high availability capabilities.
- LogStash: This enables collecting and processing logs by defining a data pipeline. LogStash is able to parse different sets of structured and unstructured data. The hallmark of this component is the ability to centralize several input data source types and convert them to a standard format.
LogStash supports a large number of plugins for input and output. Custom plugins can be also developed for specific data format. Examples of existing input plugins can be found here: https://www.elastic...