The following subsections we will cover important external system integrations for Apex applications and the corresponding connectors provided by the Apex library. The section will start with the streaming data connectors, used for continuous processing and low latency use cases. Next, we will look at the file connectors, which are frequently used, especially for batch use cases, where massive amounts of data need to be processed and the ability to read or write with high throughput in a scalable manner is important. Finally, we will look at a few database connectors.
In its own words, Apache Kafka (http://kafka.apache.org/) "is used for building real-time data pipelines and streaming apps. It is horizontally scalable, fault-tolerant, wicked fast, and runs in production in thousands of companies".
Apache Kafka is a distributed, horizontally scalable, fault tolerant and high-throughput pub-sub messaging system. In contrast to similar messaging systems, Kafka was not...