While Python is often said to have batteries included, there are a few key libraries that really take Python's ability to work with data to another level. In this recipe, we will install what is sometimes called the SciPy stack, which includes NumPy, SciPy, pandas, matplotlib, and Jupyter.
This recipe assumes that you have a standard Python installed.
Note
If, in the previous section, you decided to install the Anaconda distribution (or another distribution of Python with the needed libraries included), you can skip this recipe.
To check whether you have a particular Python package installed, start up your Python interpreter and try to import the package. If successful, the package is available on your machine. Also, you will probably need root access to your machine via the sudo command.
The following steps will allow you to install the Python data stack on Linux:
- When installing this stack on Linux, you must know which distribution of Linux you are using. The flavor of Linux usually determines the package management system that you will be using, and the options include
apt-get,yum, andrpm. - Open your browser and navigate to http://www.scipy.org/install.html , which contains detailed instructions for most platforms.
- These instructions may change and should supersede the instructions offered here, if different:
- Open up a shell.
- If you are using Ubuntu or Debian, type the following:
sudo apt-get install build-essential python-dev python- setuptools python-numpy python-scipy python-matplotlib ipython ipython-notebook python-pandas python-sympy python-nose
- If you are using Fedora, type the following:
sudo yum install numpy scipy python-matplotlib ipython python-pandas sympy python-nose
- You have several options to install the Python data stack on your Macintosh running OS X. These are:
- The first option is to download pre-built installers (
.dmg) for each tool, and install them as you would any other Mac application (this is recommended). - The second option is if you have MacPorts, a command line-based system to install software, available on your system. You will also probably need XCode with the command-line tools already installed. If so, you can enter:
- The first option is to download pre-built installers (
sudo port install py27-numpy py27-scipy py27-matplotlib py27- ipython +notebook py27-pandas py27-sympy py27-nose
- As the third option, Chris Fonnesbeck provides a bundled way to install the stack on the Mac that is tested and covers all the packages we will use here. Refer to http://fonnesbeck.github.io/ScipySuperpack .
All the preceding options will take time as a large number of files will be installed on your system.
Installing the SciPy stack has been challenging historically due to compilation dependencies, including the need for Fortran. Thus, we don't recommend that you compile and install from source code, unless you feel comfortable doing such things.
Now, the better question is, what did you just install? We installed the latest versions of NumPy, SciPy, matplotlib, IPython, IPython Notebook, pandas, SymPy, and nose. The following are their descriptions:
- SciPy: This is a Python-based ecosystem of open source software for mathematics, science, and engineering and includes a number of useful libraries for machine learning, scientific computing, and modeling.
- NumPy: This is the foundational Python package providing numerical computation in Python, which is C-like and incredibly fast, particularly when using multidimensional arrays and linear algebra operations. NumPy is the reason that Python can do efficient, large-scale numerical computation that other interpreted or scripting languages cannot do.
- matplotlib: This is a well-established and extensive 2D plotting library for Python that will be familiar to MATLAB users.
- IPython: This offers a rich and powerful interactive shell for Python. It is a replacement for the standard Python Read-Eval-Print Loop (REPL), among many other tools.
- Jupyter Notebook: This offers a browser-based tool to perform and record work done in Python with support for code, formatted text, markdown, graphs, images, sounds, movies, and mathematical expressions.
- pandas: This provides a robust data frame object and many additional tools to make traditional data and statistical analysis fast and easy.
- nose: This is a test harness that extends the unit testing framework in the Python standard library.
We will discuss the various packages in greater detail in the chapter in which they are introduced. However, we would be remiss if we did not at least mention the Python IDEs. In general, we recommend using your favorite programming text editor in place of a full-blown Python IDE. This can include the open source Atom from GitHub, the excellent Sublime Text editor, or TextMate, a favorite of the Ruby crowd. Vim and Emacs are both excellent choices not only because of their incredible power but also because they can easily be used to edit files on a remote server, a common task for the data scientist. Each of these editors is highly configurable with plugins that can handle code completion, highlighting, linting, and more. If you must have an IDE, take a look at PyCharm (the community edition is free) from the IDE wizards at JetBrains, Spyder, and Ninja-IDE. You will find that most Python IDEs are better suited for web development as opposed to data work.
You can also take a look at the following for reference:
- For more information on pandas, refer to the Python Data Analysis Library article at http://pandas.pydata.org/
- Visit the NumPy website at http://www.numpy.org/
- Visit the SciPy website at http://www.scipy.org/
- Visit the matplotlib website at http://matplotlib.org/
- Visit the IPython website at http://ipython.org/
- Refer the History of SciPy article at http://wiki.scipy.org/History_of_SciPy
- Visit the MacPorts home page at http://www.macports.org/
- Visit the XCode web page at https://developer.apple.com/xcode/features/
- Visit the XCode download page at https://developer.apple.com/xcode/downloads/