Book Image

Architecting Data-Intensive Applications

By : Anuj Kumar
Book Image

Architecting Data-Intensive Applications

By: Anuj Kumar

Overview of this book

<p>Are you an architect or a developer who looks at your own applications gingerly while browsing through Facebook and applauding it silently for its data-intensive, yet ?uent and efficient, behaviour? This book is your gateway to build smart data-intensive systems by incorporating the core data-intensive architectural principles, patterns, and techniques directly into your application architecture.</p> <p>This book starts by taking you through the primary design challenges involved with architecting data-intensive applications. You will learn how to implement data curation and data dissemination, depending on the volume of your data. You will then implement your application architecture one step at a time. You will get to grips with implementing the correct message delivery protocols and creating a data layer that doesn’t fail when running high traffic. This book will show you how you can divide your application into layers, each of which adheres to the single responsibility principle. By the end of this book, you will learn to streamline your thoughts and make the right choice in terms of technologies and architectural principles based on the problem at hand.</p>
Table of Contents (18 chapters)
Title Page
Packt Upsell
Contributors
Preface
Index

The data explosion problem


In order to understand the data explosion problem we are talking about, let’s take a realistic use case to bring home the point.

We all know Instagram. It is, in the most simplest of terms, a photo-sharing website. One statistic https://blog.hootsuite.com/instagram-statistics/ claims number of monthly active users on Instagram is 800 million. Now, let's assume that on average, each active user posts a new photo every day. There would be some users who wouldn’t post a new pic every day but there would be users who would post more than one picture every day. So we will even it out to 1 picture per day per user.

 

Next, if we assume the average size of the picture to be 1 MB, then, on average Instagram generates: 1MB X 800 Million = 800 tera bytes of images alone.

Now this data excludes all the meta information related to images, such as the number of likes, who liked it when, and filters applied on the image. This also excludes any replication of these images for high...