The Data Lake vision and promise is to transform the vast amount of big data into real insights that help drive fact-based decisions. We are now witnessing this dream being realized in several enterprises across multiple industries such as fashion, food, retail, banks, and others. While there are a few success stories, there are a significant number of organizations that have the new platform stuck in development or proof of concept stage and are unable to move to production.
In this chapter, I will share the approaches to ensure a successful big data project. The topics covered in this chapter are as follows:
Challenges on building a production Data Lake
The success path for a sustainable production Data Lake
Architectural considerations
Online resources