In this section, we will talk about the various features and components of Lambda Architecture. Let's move forward and first look at the need for Lambda Architecture, then, we will dive deep into the various other aspects of it.
The driving force behind Lambda was the latency introduced by the MapReduce paradigm. Hadoop or MapReduce solved the purpose of a distributed and scalable batch processing system, but, at the same time, it was also true that batch views were created on stale and outdated data (by at least 3-4 hours). Though it was acceptable in a few cases where data arrived once or twice in a day, but it was not acceptable to use cases where real-time updates could make a significant difference to the overall computations.
The next evident question is as to why we can't compute and recompute everything on the fly.
We can only do this if our systems have unlimited CPU, memory, and network speed, which obviously is not the case...