The Hadoop ecosystem and HDInsight platform are constantly evolving and new components are being added with every release that enable new use cases and improved experience for data consumers. In this chapter, we reviewed HBase, Storm, and Tez. HBase provides a low latency database that currently powers applications such as Facebook messaging. Storm provides real-time data processing capabilities and complements the batch processing with MapReduce. Tez is the next generation MapReduce-like framework built on top of YARN projects such as Hive and Pig can be leveraged for improved performance.

In the next chapter, we will review the tips and architectural considerations for starting a new Data Lake initiative.