Samza architecture
Samza[1] is an open source stream/event processing system that was developed at LinkedIn.
You may know LinkedIn as a social network for professionals, but just like with every successful social network, the core and hidden part of such successful organizations is their engineering department, which churns out some next-gen technologies to solve their present-day issues. Samza was born out of one of those needs.
As per LinkedIn Engineering[3] , at the start of 2016, a staggering 1.3 trillion events (pertaining to application and system monitoring, member-behavior tracking, and inter-application communication) were being published every day into Kafka (their primary messaging system in the event processing framework) with peaks of 4.5 million messages/sec per cluster. And this is just part of the overall events that get generated at LinkedIn. LinkedIn uses Samza to process this deluge of events in real time. And LinkedIn is not the only one that uses Apache Samza. Companies...