Chapter 7. Big Data with Python

With the advent of cloud computing technologies, big data has become increasingly commonplace. What is big data exactly and how can you work with big data to gather useful information? How different is big data from the kind of data we come across everyday? This chapter will specifically answer these questions and introduce you to the use of big data in finance. Big data tools provide the scalability and reliability of analyzing large volumes of data coming from multiple sources. In meeting these big data needs, Apache Hadoop became the primary choice for financial institutions and enterprises. As such, it is crucial for financial engineers to be familiar with Hadoop for financial applications.

As we begin to process large datasets, we also need to find an avenue to store this data. The de facto standard for relational database management was Structured Query Language (SQL). The nature of digital data is varied, and other means of storing data became...