Prologue

The Data and Analytics Journey So Far

“We are surrounded by data but starved for insights”

– Jay Baer

We have been surrounded by digital data for almost a century now and every decade has had its unique challenges regarding how to get the best value out of that data. But these challenges were narrow in scope and manageable since the data itself was manageable. Even though data was rapidly growing in the 20th century, its volume, velocity, and variety were still limited in nature. And then we hit the 21st century and the world of data drastically changed. Data started to exponentially grow due to multiple reasons:

• The adoption of the internet picked up speed and data grew into big data
• Smartphone devices became a common household entity and these devices all generated tons of data
• Social media took off and added to the deluge of information
• Robotics, smart edge devices, industrial devices, drones, gaming, VR, and other artificial intelligence-driven gadgets took the growth of data to a whole new level.

However, across all this, the common theme that exists even today is that data gets produced, processed, stored, and consumed.

Now, even though the history of data and analytics goes back many decades, I don’t want to dig everything up. Since this book revolves around cloud computing technologies, it is important to understand how we got here, what systems were in place in the on-premises data center world, and why those same systems and the architectural patterns surrounding them struggle to cater to the business and technology needs of today.

In this prologue, we will cover the following main topics:

• Introduction to the data and analytics journey
• Traditional data platforms
• Challenges with on-premises data systems
• What this book is all about

If you are already well versed with the traditional data platforms and their challenges, you can skip this introduction and directly jump to Chapter 1.