-
Book Overview & Buying
-
Table Of Contents
Python Data Analysis - Fourth Edition
By :
Parallel computing is most valuable when it supports the entire workflow of our data analysis, not just the final result. We often need to read data from many files, clean and validate it, handle missing values, and prepare features for modeling. These steps are often the primary cost drivers of a pipeline. Dask DataFrame allows us to use the same preprocessing logic we would write in pandas, while executing it across partitions in parallel. In this section, we focus on parallel data loading and preprocessing, which will become important when we perform machine learning modeling.
Let us start by understanding parallel data processing with Dask.
In many data analysis processes, the analysis begins by reading the dataset from its sources. In many businesses, this means reading from raw data files such as CSV files. However, many real datasets are stored as multiple files. For example, web...