Book Image

Architecting Data-Intensive Applications

By : Anuj Kumar
Book Image

Architecting Data-Intensive Applications

By: Anuj Kumar

Overview of this book

<p>Are you an architect or a developer who looks at your own applications gingerly while browsing through Facebook and applauding it silently for its data-intensive, yet ?uent and efficient, behaviour? This book is your gateway to build smart data-intensive systems by incorporating the core data-intensive architectural principles, patterns, and techniques directly into your application architecture.</p> <p>This book starts by taking you through the primary design challenges involved with architecting data-intensive applications. You will learn how to implement data curation and data dissemination, depending on the volume of your data. You will then implement your application architecture one step at a time. You will get to grips with implementing the correct message delivery protocols and creating a data layer that doesn’t fail when running high traffic. This book will show you how you can divide your application into layers, each of which adheres to the single responsibility principle. By the end of this book, you will learn to streamline your thoughts and make the right choice in terms of technologies and architectural principles based on the problem at hand.</p>
Table of Contents (18 chapters)
Title Page
Packt Upsell
Contributors
Preface
Index

Types of data sources


In a data-intensive application, knowing the source of your data is one of the most important things. Understanding the relevance of the data you are collecting for processing and analysis will help you get insights into your data quicker.

Data for your processing needs can, and usually will, originate from a variety of sources. All different sources will usually provide different types of information that you can augment or relate to each other in a manner to suit your business needs.

Taking our example on intrusion and vulnerability detection, you can collect data from system and application logs. You also need to collect data from services that map IP Address to Domain names so that you can enrich your information and check the domains against the blacklisted domains, for example. You need to collect vulnerability information about different products from, let's say, national vulnerability database, so that you can try to plug the gaps. Thus data is originating from...