So what is stream processing and why is it important? In traditional data processing, data is typically processed in batch mode. The data will be dealt with on a regular schedule. One fundamental challenge with conventional data processing is it's inherently reactive because it focuses on ageing information. Stream processing, on the other hand, processes data as it flows through in real time.
The following are some of the highlights of why stream processing is critical:
- Response time is critical:
- Reducing decision latency can unlock business value
- Need to ask questions about data in motion
- Can't wait for data to get to rest before running computation
- Actions by human actors:
- See and seize insights
- Live visualization
- Alerts and alarms
- Dynamic aggregation
- Machine-to-machine interactions:
- Data movement with enrichment
- Kick-off workflows for automation
Before one goes into stream analytics, it is essential to understand the core basics around events and different models of publishing and consuming events. Let's get more familiar with queues, Pub/Sub, and events, which will surely help you understand the later chapters better. In the following sections, we will explore queues, Pub/Sub, and events.