Hands-On Data Science and Python Machine Learning

By : Frank Kane

Hands-On Data Science and Python Machine Learning

By: Frank Kane

Overview of this book

Join Frank Kane, who worked on Amazon and IMDb’s machine learning algorithms, as he guides you on your first steps into the world of data science. Hands-On Data Science and Python Machine Learning gives you the tools that you need to understand and explore the core topics in the field, and the confidence and practice to build and analyze your own machine learning models. With the help of interesting and easy-to-follow practical examples, Frank Kane explains potentially complex topics such as Bayesian methods and K-means clustering in a way that anybody can understand them. Based on Frank’s successful data science course, Hands-On Data Science and Python Machine Learning empowers you to conduct data analysis and perform efficient machine learning using Python. Let Frank help you unearth the value in your data using the various data mining and data analysis techniques available in Python, and to develop efficient predictive models to predict future results. You will also learn how to perform large-scale machine learning on Big Data using Apache Spark. The book covers preparing your data for analysis, training machine learning models, and visualizing the final data analysis.

Preface

Free Chapter

Getting Started

Installing Enthought Canopy

Using and understanding IPython (Jupyter) Notebooks

Python basics - Part 1

Understanding Python code

Importing modules

Python basics - Part 2

Running Python scripts

Summary

Statistics and Probability Refresher, and Python Practice

Types of data

Mean, median, and mode

Using mean, median, and mode in Python

Standard deviation and variance

Probability density function and probability mass function

Types of data distributions

Percentiles and moments

Summary

Matplotlib and Advanced Probability Concepts

A crash course in Matplotlib

Covariance and correlation

Conditional probability

Bayes' theorem

Summary

Predictive Models

Linear regression

Polynomial regression

Multivariate regression and predicting car prices

Multi-level models

Summary

Machine Learning with Python

Machine learning and train/test

Using train/test to prevent overfitting of a polynomial regression

Bayesian methods - Concepts

Implementing a spam classifier with Naïve Bayes

K-Means clustering

Clustering people based on income and age

Measuring entropy

Decision trees - Concepts

Decision trees - Predicting hiring decisions using Python

Ensemble learning

Support vector machine overview

Using SVM to cluster people by using scikit-learn

Summary

Recommender Systems

What are recommender systems?

Item-based collaborative filtering

How item-based collaborative filtering works?

Finding movie similarities

Improving the results of movie similarities

Making movie recommendations to people

Improving the recommendation results

Summary

More Data Mining and Machine Learning Techniques

K-nearest neighbors - concepts

Using KNN to predict a rating for a movie

Dimensionality reduction and principal component analysis

A PCA example with the Iris dataset

Data warehousing overview

Reinforcement learning

Summary

Dealing with Real-World Data

Bias/variance trade-off

K-fold cross-validation to avoid overfitting

Data cleaning and normalisation

Cleaning web log data

Normalizing numerical data

Detecting outliers

Summary

Apache Spark - Machine Learning on Big Data

Installing Spark

Spark introduction

Spark and Resilient Distributed Datasets (RDD)

Introducing MLlib

Decision Trees in Spark with MLlib

K-Means Clustering in Spark

TF-IDF

Searching wikipedia with Spark MLlib

Using the Spark 2.0 DataFrame API for MLlib

Summary

Testing and Experimental Design

A/B testing concepts

T-test and p-value

Measuring t-statistics and p-values using Python

Determining how long to run an experiment for

A/B test gotchas

Summary

Customer Reviews

5 star

4 star

3 star

2 star

1 star

K-Means Clustering in Spark

Alright, let's look at another example of using Spark in MLlib, and this time we're going to look at k-means clustering, and just like we did with decision trees, we're going to take the same example that we did using scikit-learn and we're going to do it in Spark instead, so it can actually scale up to a massive Dataset. So, again, I've made sure to close out of everything else, and I'm going to go into my book materials and open up the SparkKMeans Python script, and let's study what's going on in.

Alright, so again, we begin with some boilerplate stuff.

from pyspark.mllib.clustering import KMeans 
from numpy import array, random 
from math import sqrt 
from pyspark import SparkConf, SparkContext 
from sklearn.preprocessing import scale

We're going to import the KMeans package from the clustering MLlib...

Hands-On Data Science and Python Machine Learning

By : Frank Kane

Hands-On Data Science and Python Machine Learning

By: Frank Kane

Overview of this book

Related Content you might be interested in

Current Title:

Hands-On Data Science and Python Machine Learning

Frank Kane's Taming Big Data with Apache Spark and Python

Hands-On Recommendation Systems with Python

Getting Started with Haskell Data Analysis