Types of data sources
In a data-intensive application, knowing the source of your data is one of the most important things. Understanding the relevance of the data you are collecting for processing and analysis will help you get insights into your data quicker.
Data for your processing needs can, and usually will, originate from a variety of sources. All different sources will usually provide different types of information that you can augment or relate to each other in a manner to suit your business needs.
Taking our example on intrusion and vulnerability detection, you can collect data from system and application logs. You also need to collect data from services that map IP Address to Domain names so that you can enrich your information and check the domains against the blacklisted domains, for example. You need to collect vulnerability information about different products from, let's say, national vulnerability database, so that you can try to plug the gaps. Thus data is originating from...