About the Book

Businesses today operate online and generate data almost continuously. While not all data in its raw form may seem useful, if processed and analyzed correctly, it can provide you with valuable hidden insights. The Data Analysis Workshop will help you learn how to discover these hidden patterns in your data, to analyze them and leverage the results to help transform your business.

The book begins by taking you through the use case of a bike rental shop. You'll be shown how to correlate data, plot histograms, and analyze temporal features. As you progress, you'll learn how to plot data for a hydraulic system using the Seaborn and Matplotlib libraries, and explore a variety of use cases that show you how to join and merge databases, prepare data for analysis, and handle imbalanced data.

By the end of the book, you'll have learned different data analysis techniques, including hypothesis testing, correlation, and null-value imputation, and will have become a confident data analyst.

Audience

The Data Analysis Workshop is for programmers who already know how to code in Python and want to use it to perform data analysis. If you are looking to gain practical experience in data science with Python, this book is for you.

About the Chapters

Chapter 1, Bike Sharing Analysis, teaches you how to analyze data from bike-sharing services and identify usage patterns, depending on time features and weather conditions.

Chapter 2, Absenteeism at Work, teaches you how to perform standard data analysis techniques, such as estimating conditional probabilities, Bayes' theorem, and the Kolmogorov-Smirnov test for distribution comparison.

Chapter 3, Analyzing Bank Marketing Campaign Data, teaches you how to analyze marketing campaign data related to new financial products and model the relationships between the different features in the data and their impact on the final outcome of the campaign.

Chapter 4, Tackling Company Bankruptcy, prepares you to perform exploratory data analysis using pandas profiling and how to handle imbalances in the data using the oversampling technique.

Chapter 5, Analyzing the Online Shopper's Purchasing Intention, teaches you how to perform univariate and bivariate analysis, implement clustering, and make recommendations based on the predictions.

Chapter 6, Analysis of Credit Card Defaulters, teaches you how to analyze the characteristics and build the persona of the customer who is most likely to default on their credit card payments.

Chapter 7, Analyzing the Heart Disease Dataset, prepares you to execute various data analysis techniques, such as searching for missing values and outliers, and plot visualizations to observe trends and patterns that exist in the data.

Chapter 8, Analyzing Online Retail II Dataset, prepares you to analyze data retrieved from an online retail company to observe patterns and correlations, and to evaluate the business more accurately and in more depth.

Chapter 9, Analysis of the Energy Consumed by Appliances, teaches you how to perform feature engineering by creating new features from existing ones, conducting exploratory data analysis, and designing informative visualizations.

Chapter 10, Analyzing Air Quality, instructs you how to implement data analysis techniques pertaining to a specific dataset—the Beijing Multi-Site Air-Quality Dataset.

Conventions

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "For this reason, use the np.issubdtype() function, which checks if the type of the first argument is equal to or a subtype of the second one."

Words that you see onscreen (for example, in menus or dialog boxes) also appear in the text like this: "Then, set the title as Distribution of Presence of Heart Disease by Number of Vessels Colored by Fluoroscopy."

A block of code is set as follows:

d.set_title('Distribution of Presence of Heart Disease by Number of Major Vessels Coloured by Fluoroscopy')
plt.show()

New terms and important words are shown like this: "An ARFF (Attribute-Relation File Format) file is an ASCII text file. It essentially explains a list of instances that commonly share an attribute set."

Code Presentation

Lines of code that span multiple lines are split using a backslash ( \ ). When the code is executed, Python will ignore the backslash, and treat the code on the next line as a direct continuation of the current line.

For example:

history = model.fit(X, y, epochs=100, batch_size=5, verbose=1, \
                    validation_split=0.2, shuffle=False)

Comments are added into code to help explain specific bits of logic. Single-line comments are denoted using the # symbol, as follows:

# Print the sizes of the dataset
print("Number of Examples in the Dataset = ", X.shape[0])
print("Number of Features for each example = ", X.shape[1])

Multi-line comments are enclosed by triple quotes, as shown below:

"""
Define a seed for the random number generator to ensure the 
result will be reproducible
"""
seed = 1
np.random.seed(seed)
random.set_seed(seed)

Setting up Your Environment

Before we explore the book in detail, we need to set up specific software and tools. In the following section, we shall see how to do that.

Installation and Setup

Jupyter notebooks are available once you install Anaconda on your system. Anaconda can be installed for Windows systems using the steps available at https://docs.anaconda.com/anaconda/install/windows/.

For other systems, navigate to the respective installation guide from https://docs.anaconda.com/anaconda/install/.

These installations will be executed in the C drive of your system. You can choose to change the destination.

Installing Libraries

pip comes pre-installed with Anaconda. Once Anaconda is installed on your machine, all the required libraries can be installed using pip, for example, pip install numpy. Alternatively, you can install all the required libraries using pip install –r requirements.txt. You can find the requirements.txt file at https://packt.live/2N6PPZv.

The exercises and activities will be executed in Jupyter Notebooks. Jupyter is a Python library and can be installed in the same way as the other Python libraries – that is, with pip install jupyter, but fortunately, it comes pre-installed with Anaconda. To open a notebook, simply run the command jupyter notebook in the Terminal or Command Prompt.

Accessing the Code Files

You can find the complete code files of this book at https://packt.live/34iiLFK. You can also run many activities and exercises directly in your web browser by using the interactive lab environment at https://packt.live/2UWecgN.

We've tried to support interactive versions of all activities and exercises, but we recommend a local installation as well for instances where this support isn't available.

Note

It is imperative that you use a single notebook for each chapter and follow all the practical elements in the order that they appear in the book to avoid errors.

If you have any issues or questions about installation, please email us at [email protected].