Logstash is a lightweight, open source data processing pipeline. It allows collecting data from a wide variety of sources, transforming it on the fly, and sending it to any desired destination.
It is most often used as a data pipeline for Elasticsearch, a popular analytics and search engine. Logstash is a popular choice for loading data into Elasticsearch because of its tight integration, powerful log processing capabilities, and over 200 prebuilt open source plugins that can help you get your data indexed the way you want it.
The following is a structure of Logstash.conf:
input {
...
}
filter {
...
}
output {
..
}
Installation of Logstash:
$ wget https://artifacts.elastic.co/downloads/logstash/logstash-6.1.2.rpm
$ sudo rpm --install logstash-6.1.2.rpm
$ sudo /bin/systemctl daemon-reload
$ sudo systemctl start logstash.service