Ongoing advances in computational power (per Moore's Law) have begun to make machine learning, once mostly a research discipline, more viable in commercial contexts. This has caused an explosion of new applications and new or rediscovered techniques, catapulting the obscure concepts of data science, AI, and machine learning into the public consciousness and strategic planning of companies internationally.

The rapid development of machine learning applications is fueled by an ongoing struggle to continually innovate, playing out at an array of research labs. The techniques developed by these pioneers are seeding new application areas and experiencing growing public awareness. While some of the innovations sought in AI and applied machine learning are still elusively far from readiness, others are a reality. Self-driving cars, sophisticated image recognition and altering capability, ever-greater strides in genetics research, and perhaps most pervasively of all, increasingly tailored content in our digital stores, e-mail inboxes, and online lives.

With all of these possibilities and more at the fingertips of the committed data scientist, the profession is seeing a meteoric, if clumsy, growth. Not only are there far more data scientists and AI practitioners now than there were even two years ago (in early 2014), but the accessibility and openness around solutions at the high end of machine learning research has increased.

Research teams at Google and Facebook began to share more and more of their architecture, languages, models, and tools in the hope of seeing them applied and improved on by the growing data scientist population.

The machine learning community matured enough to begin seeing trends as popular algorithms were defined or rediscovered. To put this more accurately, pre-existing trends from a mainly research community began to receive great attention from industry, with one product being a group of machine learning experts straddling industry and academia. Another product, the subject of this section, is a growing awareness of advanced algorithms that can be used to crack the frontier problems of the current day. From month to month, we see new advances made, scores rise, and the frontier moves ever further out.

What all of this means is that there may never have been a better time to move into the field of data science and develop your machine learning skillset. The introductory algorithms (including clustering, regression models, and neural network architectures) and tools are widely covered in web courses and blog content. While the techniques at the cutting edge of data science (including deep learning, semi-supervised algorithms, and ensembles) remain less accessible, the techniques themselves are now available through software libraries in multiple languages. All that's needed is the combination of theoretical knowledge and practical guidance to implement models correctly. That is the requirement that this book was written to address.

You've begun to read a book that focuses on teaching some of the advanced modeling techniques that've emerged in recent years. This book is aimed at anyone who wants to learn about those algorithms, whether you're an experienced data scientist or developer looking to parlay existing skills into a new environment.

I aimed first and foremost at making sure that you understand the algorithms in question. Some of them are fairly tricky and tie into other concepts in statistics and machine learning.

For neophyte readers, I definitely recommend gathering an initial understanding of key concepts, including the following:

At times, this book won't be able to give a subject the attention that it deserves. We cover a lot of ground in this book and the pace is fairly brisk as a result! At the end of each chapter, I refer you to further reading, in a book or online article, so that you can build a broader base of relevant knowledge. I'd suggest that it's worth doing additional reading around any unfamiliar concept that comes up as you work through this book, as machine learning knowledge tends to tie together synergistically; the more you have, the more readily you'll understand new concepts as you expand your toolkit.

This concept of expanding a toolkit of skills is fundamental to what I've tried to achieve with this book. Each chapter introduces one or multiple algorithms and looks to achieve several goals:

Beyond the transfer of knowledge and practical skills, this book looks to achieve a more important goal; specifically, to discuss and convey some of the qualities that are common to skilled machine learning practitioners. These include creativity, demonstrated both in the definition of sophisticated architectures and problem-specific cleaning techniques. Rigor is another key quality, emphasized throughout this book by a focus on measuring performance against meaningful targets and critically assessing early efforts.

Finally, this book makes no effort to obscure the realities of working on solving data challenges: the mixed results of early trials, large iteration counts, and frequent impasses. Yet at the same time, using a mixture of toy examples, dissection of expert approaches and, toward the end of the book, more real-world challenges, we show how a creative, tenacious, and rigorous approach can break down these barriers and deliver meaningful results.

As we proceed, I wish you the best of luck and encourage you to enjoy yourself as you go, tackling the content prepared for you and applying what you've learned to new domains or data.

Let's get started!

Chapter 1, Unsupervised Machine Learning, shows you how to apply unsupervised learning techniques to identify patterns and structure within datasets.

Chapter 2, Deep Belief Networks, explains how the RBM and DBN algorithms work; you'll know how to use them and will feel confident in your ability to improve the quality of the results that you get out of them.

Chapter 3, Stacked Denoising Autoencoders, continues to build our skill with deep architectures by applying stacked denoising autoencoders to learn feature representations for high-dimensional input data.

Chapter 4, Convolutional Neural Networks, shows you how to apply the convolutional neural network (or Convnet).

Chapter 5, Semi-Supervised Learning, explains how to apply several semi-supervised learning techniques, including CPLE, self-learning, and S3VM.

Chapter 6, Text Feature Engineering, discusses data preparation skills that significantly increase the effectiveness of all the models that we've previously discussed.

Chapter 7, Feature Engineering Part II, shows you how to interrogate the data to weed out or mitigate quality issues, transform it into forms that are conducive to machine learning, and creatively enhance that data.

Chapter 8, Ensemble Methods, looks at building more sophisticated model ensembles and methods of building robustness into your model solutions.

Chapter 9, Additional Python Machine Learning Tools, reviews some of the best in recent tools available to data scientists, identifies the benefits that they offer, and discusses how to apply them alongside tools and techniques discussed earlier in this book, within a consistent working process.

Appendix A, Chapter Code Requirements, discusses tool requirements for the book, identifying required libraries for each chapter.

The entirety of this book's content leverages openly available data and code, including open source Python libraries and frameworks. While each chapter's example code is accompanied by a README file documenting all the libraries required to run the code provided in that chapter's accompanying scripts, the content of these files is collated here for your convenience.

It is recommended that some libraries required for earlier chapters be available when working with code from any later chapter. These requirements are identified using bold text. Particularly, it is important to set up the first chapter's required libraries for any content later in the book.

This title is for Python developers and analysts or data scientists who are looking to add to their existing skills by accessing some of the most powerful recent trends in data science. If you've ever considered building your own image or text-tagging solution or entering a Kaggle contest, for instance, this book is for you!

Prior experience of Python and grounding in some of the core concepts of machine learning would be helpful.

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "We will begin applying PCA to the handwritten digits dataset with the following code."

A block of code is set as follows:

import numpy as np
from sklearn.datasets import load_digits
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
from sklearn.preprocessing import scale
from sklearn.lda import LDA
import matplotlib.cm as cm

digits = load_digits()
data = digits.data

n_samples, n_features = data.shape
n_digits = len(np.unique(digits.target))
labels = digits.target

Any command-line input or output is written as follows:

[ 0.39276606  0.49571292  0.43933243  0.53573558  0.42459285           
  0.55686854  0.4573401   0.49876358  0.50281585  0.4689295 ]

0.4772857426

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail <[email protected]>, and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.