Book Image

Large Scale Machine Learning with Python

By : Luca Massaron, Bastiaan Sjardin, Alberto Boschetti
Book Image

Large Scale Machine Learning with Python

By: Luca Massaron, Bastiaan Sjardin, Alberto Boschetti

Overview of this book

Large Python machine learning projects involve new problems associated with specialized machine learning architectures and designs that many data scientists have yet to tackle. But finding algorithms and designing and building platforms that deal with large sets of data is a growing need. Data scientists have to manage and maintain increasingly complex data projects, and with the rise of big data comes an increasing demand for computational and algorithmic efficiency. Large Scale Machine Learning with Python uncovers a new wave of machine learning algorithms that meet scalability demands together with a high predictive accuracy. Dive into scalable machine learning and the three forms of scalability. Speed up algorithms that can be used on a desktop computer with tips on parallelization and memory allocation. Get to grips with new algorithms that are specifically designed for large projects and can handle bigger files, and learn about machine learning in big data environments. We will also cover the most effective machine learning techniques on a map reduce framework in Hadoop and Spark in Python.

Related Content you might be interested in

Current Title:

Small book image
Large Scale Machine Learning with Python
Luca Massaron | Bastiaan Sjardin | Alberto Boschetti
Aug, 2016 | 420 pages
No titles found
Table of Contents (17 chapters)
Large Scale Machine Learning with Python
Large Scale Machine Learning with Python
Credits
Credits
About the Authors
About the Authors
About the Reviewer
About the Reviewer
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
First Steps to Scalability
First Steps to Scalability
Explaining scalability in detail
Python for large scale machine learning
Python packages
Summary
2
Scalable Learning in Scikit-learn
Scalable Learning in Scikit-learn
Out-of-core learning
Streaming data from sources
Stochastic learning
Feature management with data streams
Summary
3
Fast SVM Implementations
Fast SVM Implementations
Datasets to experiment with on your own
Support Vector Machines
Feature selection by regularization
Including non-linearity in SGD
Hyperparameter tuning
Summary
4
Neural Networks and Deep Learning
Neural Networks and Deep Learning
The neural network architecture
Neural networks and regularization
Neural networks and hyperparameter optimization
Neural networks and decision boundaries
Deep learning at scale with H2O
Deep learning and unsupervised pretraining
Deep learning with theanets
Autoencoders and unsupervised learning
Summary
5
Deep Learning with TensorFlow
Deep Learning with TensorFlow
TensorFlow installation
Machine learning on TensorFlow with SkFlow
Keras and TensorFlow installation
Convolutional Neural Networks in TensorFlow through Keras
CNN's with an incremental approach
GPU Computing
Summary
6
Classification and Regression Trees at Scale
Classification and Regression Trees at Scale
Bootstrap aggregation
Random forest and extremely randomized forest
Fast parameter optimization with randomized search
CART and boosting
XGBoost
Out-of-core CART with H2O
Summary
7
Unsupervised Learning at Scale
Unsupervised Learning at Scale
Unsupervised methods
Feature decomposition – PCA
PCA with H2O
Clustering – K-means
K-means with H2O
LDA
Summary
8
Distributed Environments – Hadoop and Spark
Distributed Environments – Hadoop and Spark
From a standalone machine to a bunch of nodes
Setting up the VM
The Hadoop ecosystem
Spark
Summary
9
Practical Machine Learning with Spark
Practical Machine Learning with Spark
Setting up the VM for this chapter
Sharing variables across cluster nodes
Data preprocessing in Spark
Machine learning with Spark
Summary
Introduction to GPUs and Theano
Introduction to GPUs and Theano
GPU computing
Theano – parallel computing on the GPU
Installing Theano
Index
Index

Theano – parallel computing on the GPU

Theano is a Python library originally developed by James Bergstra at the University of Montreal. It aims at providing more expressive ways to write mathematical functions with symbolic representations (F. Bastien, P. Lamblin, R. Pascanu, J. Bergstra, I. Goodfellow, A. Bergeron, N. Bouchard, D. Warde-Farley and Y. Bengio. Theano: new features and speed improvements. NIPS 2012 Deep Learning Workshop). Interestingly, Theano is named after the Greek mathematician, who may have been Pythagoras' wife. It's strongest points are fast c-compiled computations, symbolic expressions, and GPU computation, and Theano is under active development. Improvements are made regularly with new features. Theano's implementations are much wider than scalable machine learning so I will narrow down and use Theano for deep learning. Visit the Theano website for more information—http://deeplearning.net/software/theano/.

When we want to perform more complex computations on multidimensional matrices, basic NumPy will resort to costly loops and iterations driving up the CPU load as we have seen earlier. Theano aims to optimize these computations by compiling them into highly optimized C-code and, if possible, utilizing the GPU. For neural networks and deep learning, Theano has the useful capability to automatically differentiate mathematical functions, which is very convenient for calculation of the partial derivatives when using algorithms such as backpropagation.

Currently, Theano is used in all sorts of deep learning projects and has become the most used platform in this field. Lately, new packages have been built on top of Theano in order to make utilizing deep learning functionalities easier for us. Considering the steep learning curve of Theano, we will use packages built on Theano, such as theanets, pylearn2, and Lasagne.

Previous Section
End of Section
Next Section