Due to the capability of processing variety of data and volume of data, data sources for Hadoop has increased and along with that the complexity has increased enormously. We now see huge amount of batch and streaming and real-time analysis processed in Hadoop, for which data ingestion can become a bottleneck or can break a system, if not designed according to the requirement.
Let's look at some of the data sources, which can produce enormous volume of data or consistent data continuously:
Data sensors: These are thousands of sensors, producing data continuously.
Machine Data: Produces data which should be processed in near real time for avoiding huge loss.
Telco Data: CDR data and other telecom data generates high volume of data.
Healthcare system data: Genes, images, ECR records are unstructured and complex to process.
Social Media: Facebook, Twitter, Google Plus, YouTube, and others get a huge volume of data.
Geological Data: Semiconductors and other geological data produce...