Data is being produced at an accelerated pace with advancements in technology. The widespread usage and adoption of the Internet of Things (IoT) is a great example of this. These specifically purposed IoT devices are tens of billions in number and are growing rapidly. Many of these devices, using their sensors, continually produce observations as data. Even though the data might be small as a unit, combined together it becomes humongous. IoT is just one example of how much and how fast the data is being created.
This kind of data is sometimes referred to as big data that is too big to fit on a single machine for storage and computing purposes. Big data has three important properties:
- Variety: Data in different formats and structures
- Velocity: New data arriving at a fast rate
- Volume: Huge overall data size
In the prior chapters, we learned how to deal...