The following table shows comparisons between Logstash, Fluentd, Apache Flume, Apache NiFi, and Apache Kafka:
Logstash | Fluentd | Apache Flume | Apache NiFi | Apache Kafka | |
Concerns | No UI and hard to write filters | Windows installation is still in the experiment stage. | Hard to manage multiple connections. | Not mature enough when compared to other tools available on the market. | No UI and hard to maintain offsets. |
Main Features | Flexibility and Interoperability | Simplicity and robustness | Provides best integration with HDFS, reliable and scalable. | Flow management, ease of use, security, flexible scaling model. | Fast, provides pub/sub based data streams, easy to integrate and use, partitioned. |
Plugins | 90+ plugins | 125+ plugins | 50+ plugins and custom components | An ample amount of processors are available. Also you can write your own easily. | No plugin, you can write your own code. |
Scalability | Yes | Yes | Yes | Yes | Yes |
Reliability | At least once using Filebeat | At most once or at least once | At... |