In this chapter, the main objective was to learn about various Hadoop design alternatives. We've learned a lot when it comes to the Hadoop cluster and its best practices for deployment in a typical production environment. We started with a basic understanding about Hadoop and we proceeded to Hadoop configuration, installation, and HDFS architecture. We also learned about various techniques for achieving HDFS high availability. We also looked into YARN architecture. Finally, we looked at various file formats and how to choose one based on your use case.
In the next chapter, we will see how to ingest data into a newly created Hadoop cluster.