Book Image

IPython Interactive Computing and Visualization Cookbook

By : Cyrille Rossant
Book Image

IPython Interactive Computing and Visualization Cookbook

By: Cyrille Rossant

Overview of this book

Related Content you might be interested in

Current Title:

Small book image
IPython Interactive Computing and Visualization Cookbook
Cyrille Rossant
Sep, 2014 | 512 pages
No titles found
Table of Contents (22 chapters)
IPython Interactive Computing and Visualization Cookbook
IPython Interactive Computing and Visualization Cookbook
Credits
Credits
About the Author
About the Author
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
A Tour of Interactive Computing with IPython
A Tour of Interactive Computing with IPython
Introduction
Introducing the IPython notebook
Getting started with exploratory data analysis in IPython
Introducing the multidimensional array in NumPy for fast array computations
Creating an IPython extension with custom magic commands
Mastering IPython's configuration system
Creating a simple kernel for IPython
2
Best Practices in Interactive Computing
Best Practices in Interactive Computing
Introduction
Choosing (or not) between Python 2 and Python 3
Efficient interactive computing workflows with IPython
Learning the basics of the distributed version control system Git
A typical workflow with Git branching
Ten tips for conducting reproducible interactive computing experiments
Writing high-quality Python code
Writing unit tests with nose
Debugging your code with IPython
3
Mastering the Notebook
Mastering the Notebook
Introduction
Teaching programming in the notebook with IPython blocks
Converting an IPython notebook to other formats with nbconvert
Adding custom controls in the notebook toolbar
Customizing the CSS style in the notebook
Using interactive widgets – a piano in the notebook
Creating a custom JavaScript widget in the notebook – a spreadsheet editor for pandas
Processing webcam images in real time from the notebook
4
Profiling and Optimization
Profiling and Optimization
Introduction
Evaluating the time taken by a statement in IPython
Profiling your code easily with cProfile and IPython
Profiling your code line-by-line with line_profiler
Profiling the memory usage of your code with memory_profiler
Understanding the internals of NumPy to avoid unnecessary array copying
Using stride tricks with NumPy
Implementing an efficient rolling average algorithm with stride tricks
Making efficient array selections in NumPy
Processing huge NumPy arrays with memory mapping
Manipulating large arrays with HDF5 and PyTables
Manipulating large heterogeneous tables with HDF5 and PyTables
5
High-performance Computing
High-performance Computing
Introduction
Accelerating pure Python code with Numba and just-in-time compilation
Accelerating array computations with Numexpr
Wrapping a C library in Python with ctypes
Accelerating Python code with Cython
Optimizing Cython code by writing less Python and more C
Releasing the GIL to take advantage of multicore processors with Cython and OpenMP
Writing massively parallel code for NVIDIA graphics cards (GPUs) with CUDA
Writing massively parallel code for heterogeneous platforms with OpenCL
Distributing Python code across multiple cores with IPython
Interacting with asynchronous parallel tasks in IPython
Parallelizing code with MPI in IPython
Trying the Julia language in the notebook
6
Advanced Visualization
Advanced Visualization
Introduction
Making nicer matplotlib figures with prettyplotlib
Creating beautiful statistical plots with seaborn
Creating interactive web visualizations with Bokeh
Visualizing a NetworkX graph in the IPython notebook with D3.js
Converting matplotlib figures to D3.js visualizations with mpld3
Getting started with Vispy for high-performance interactive data visualizations
7
Statistical Data Analysis
Statistical Data Analysis
Introduction
Exploring a dataset with pandas and matplotlib
Getting started with statistical hypothesis testing – a simple z-test
Getting started with Bayesian methods
Estimating the correlation between two variables with a contingency table and a chi-squared test
Fitting a probability distribution to data with the maximum likelihood method
Estimating a probability distribution nonparametrically with a kernel density estimation
Fitting a Bayesian model by sampling from a posterior distribution with a Markov chain Monte Carlo method
Analyzing data with the R programming language in the IPython notebook
8
Machine Learning
Machine Learning
Introduction
Getting started with scikit-learn
Predicting who will survive on the Titanic with logistic regression
Learning to recognize handwritten digits with a K-nearest neighbors classifier
Learning from text – Naive Bayes for Natural Language Processing
Using support vector machines for classification tasks
Using a random forest to select important features for regression
Reducing the dimensionality of a dataset with a principal component analysis
Detecting hidden structures in a dataset with clustering
9
Numerical Optimization
Numerical Optimization
Introduction
Finding the root of a mathematical function
Minimizing a mathematical function
Fitting a function to data with nonlinear least squares
Finding the equilibrium state of a physical system by minimizing its potential energy
10
Signal Processing
Signal Processing
Introduction
Analyzing the frequency components of a signal with a Fast Fourier Transform
Applying a linear filter to a digital signal
Computing the autocorrelation of a time series
11
Image and Audio Processing
Image and Audio Processing
Introduction
Manipulating the exposure of an image
Applying filters on an image
Segmenting an image
Finding points of interest in an image
Detecting faces in an image with OpenCV
Applying digital filters to speech sounds
Creating a sound synthesizer in the notebook
12
Deterministic Dynamical Systems
Deterministic Dynamical Systems
Introduction
Plotting the bifurcation diagram of a chaotic dynamical system
Simulating an elementary cellular automaton
Simulating an ordinary differential equation with SciPy
Simulating a partial differential equation – reaction-diffusion systems and Turing patterns
13
Stochastic Dynamical Systems
Stochastic Dynamical Systems
Introduction
Simulating a discrete-time Markov chain
Simulating a Poisson process
Simulating a Brownian motion
Simulating a stochastic differential equation
14
Graphs, Geometry, and Geographic Information Systems
Graphs, Geometry, and Geographic Information Systems
Introduction
Manipulating and visualizing graphs with NetworkX
Analyzing a social network with NetworkX
Resolving dependencies in a directed acyclic graph with a topological sort
Computing connected components in an image
Computing the Voronoi diagram of a set of points
Manipulating geospatial data with Shapely and basemap
Creating a route planner for a road network
15
Symbolic and Numerical Mathematics
Symbolic and Numerical Mathematics
Introduction
Diving into symbolic computing with SymPy
Solving equations and inequalities
Analyzing real-valued functions
Computing exact probabilities and manipulating random variables
A bit of number theory with SymPy
Finding a Boolean propositional formula from a truth table
Analyzing a nonlinear differential system – Lotka-Volterra (predator-prey) equations
Getting started with Sage
Index
Index

Learning from text – Naive Bayes for Natural Language Processing

In this recipe, we show how to handle text data with scikit-learn. Working with text requires careful preprocessing and feature extraction. It is also quite common to deal with highly sparse matrices.

We will learn to recognize whether a comment posted during a public discussion is considered insulting to one of the participants. We will use a labeled dataset from Impermium, released during a Kaggle competition.

Getting ready

Download the Troll dataset from the book's GitHub repository at https://github.com/ipython-books/cookbook-data.

This dataset was obtained from Kaggle, at www.kaggle.com/c/detecting-insults-in-social-commentary.

How to do it...

  1. Let's import our libraries:

    In [1]: import numpy as np
        import pandas as pd
        import sklearn
        import sklearn.cross_validation as cv
        import sklearn.grid_search as gs
        import sklearn.feature_extraction.text as text
        import sklearn.naive_bayes as nb...
Previous Section
End of Section 6
Next Section