This section covers the full working example using Apache Hadoop as the data storage in conjunction with our SCV use case.
We have already covered the pseudo-distributed deployment of Hadoop in the earlier chapters. We will continue with the same setup for understanding the examples in this chapter.
For Hadoop examples, we will expand the data generation utility of customer profile even further. For batch processes it is generally expected that the volume of data is quite substantial.
Hence we will try and generate about 2 million customer records and their related information for batch processing.
We shall use the same data generation utility as before for populating the data in the database, the only difference being that we will be generating 2 million customer records in the DB. Generating these many rows in relational database may take some time depending on the machine configuration.
If you would like to cleanup the PostgreSQL database...