First, we will cover the configuration of different Beats in order to get the metric data into Elasticsearch directly or using Logstash, based on our requirements. The following flowchart shows us the way through which Beats ship the data to Logstash or Elasticsearch and then we can plot into Kibana using that data:
We will configure Filebeat, Metricbeat, and Packetbeat so that these Beats can send the data to the central Elasticseach cluster or Logstash. Once this process is started, we can use the Elasticseach data to create different visualizations and can then integrate those visualizations to create the dashboard. So, let's start the Beats configuration.
Filebeat is a lightweight data shipper that can be used to forward the logs and files data. Filebeat reads and forwards the log lines and in case of any downtime, it remembers the locations and resumes from that location once everything is back online:
Filebeat has internal modules for Apache, NGINX, MySQL...