Book Image

Architecting Data-Intensive Applications

By : Anuj Kumar
Book Image

Architecting Data-Intensive Applications

By: Anuj Kumar

Overview of this book

<p>Are you an architect or a developer who looks at your own applications gingerly while browsing through Facebook and applauding it silently for its data-intensive, yet ?uent and efficient, behaviour? This book is your gateway to build smart data-intensive systems by incorporating the core data-intensive architectural principles, patterns, and techniques directly into your application architecture.</p> <p>This book starts by taking you through the primary design challenges involved with architecting data-intensive applications. You will learn how to implement data curation and data dissemination, depending on the volume of your data. You will then implement your application architecture one step at a time. You will get to grips with implementing the correct message delivery protocols and creating a data layer that doesn’t fail when running high traffic. This book will show you how you can divide your application into layers, each of which adheres to the single responsibility principle. By the end of this book, you will learn to streamline your thoughts and make the right choice in terms of technologies and architectural principles based on the problem at hand.</p>
Table of Contents (18 chapters)
Title Page
Packt Upsell
Contributors
Preface
Index

Data quality


You can collect data from anywhere and from any source. Whether the source of your data is reliable and is providing you with quality data is something you should work toward. If the source of your data is garbage, then the output of your processing system will also be garbage. Garbage in, garbage out. For example, if you are collecting data by scrapping different websites, you need to understand how to make sense out of it as you can't always assume that the data will be good quality. This may be useful in certain application use cases, but not all. Stringent data quality rules are usually applied to data that is used in compliance use cases. These quality rules can be applied at the collection level or at the processing level. Generally, these rules are spread across the collection and processing level. Collection Systems usually handle the first level of checking on the data quality, and then an underlying processing system does a more thorough quality check.