Book Image

Python Data Analysis - Second Edition

By : Ivan Idris
Book Image

Python Data Analysis - Second Edition

By: Ivan Idris

Overview of this book

Data analysis techniques generate useful insights from small and large volumes of data. Python, with its strong set of libraries, has become a popular platform to conduct various data analysis and predictive modeling tasks. With this book, you will learn how to process and manipulate data with Python for complex analysis and modeling. We learn data manipulations such as aggregating, concatenating, appending, cleaning, and handling missing values, with NumPy and Pandas. The book covers how to store and retrieve data from various data sources such as SQL and NoSQL, CSV fies, and HDF5. We learn how to visualize data using visualization libraries, along with advanced topics such as signal processing, time series, textual data analysis, machine learning, and social media analysis. The book covers a plethora of Python modules, such as matplotlib, statsmodels, scikit-learn, and NLTK. It also covers using Python with external environments such as R, Fortran, C/C++, and Boost libraries.
Table of Contents (22 chapters)
Python Data Analysis - Second Edition
Credits
About the Author
About the Reviewers
www.PacktPub.com
Customer Feedback
Preface
Key Concepts
Online Resources

Using IPython as a shell


Data analysts, data scientists, and engineers are used to experimenting. IPython was created by scientists with experimentation in mind. The interactive environment that IPython provides is comparable to an interactive computing environment provided by Matlab, Mathematica, and Maple.

The following is a list of features of the IPython shell:

  • Tab completion, which helps you find a command

  • History mechanism

  • Inline editing

  • Ability to call external Python scripts with %run

  • Access to system commands

  • Access to the Python debugger and profiler

The following list describes how to use the IPython shell:

  • Starting a session: To start a session with IPython,enter the following instruction on the command line:

    $ ipython3
    Python 3.5.2 (default, Sep 28 2016, 18:08:09) 
    Type "copyright", "credits" or "license" for more information.
            IPython 5.1.0 -- An enhanced Interactive Python.
    ?         -> Introduction and overview of IPython's features.
    %quickref -> Quick reference.
    help      -> Python's own help system.
    object?   -> Details about 'object', use 'object??' for extra 
                         details.
    In [1]: quit()
    

    Tip

    The quit() function or Ctrl + D quits the IPython shell.

  • Saving a session: We might want to be able to go back to our experiments. In IPython, it is easy to save a session for later use with the following command:

    In [1]: %logstart
    Activating auto-logging. Current session state plus future 
             input saved:
             Filename : ipython_log.py
             Mode : rotate
             Output logging : False
             Raw input log : False
             Timestamping : False
    State : active
    

    Logging can be switched off as follows:

    In [9]: %logoff
    Switching logging OFF
    
  • Executing a system shell command: Execute a system shell command in the default IPython profile by prefixing the command with the ! symbol. For instance, the following input will get the current date:

    In [1]: !date
    

    In fact, any line prefixed with ! is sent to the system shell. We can also store the command output, as shown here:

    In [2]: thedate = !date
    In [3]: thedate
    
  • Displaying history: We can show the history of our commands with the %hist command. For example:

    In [1]: a = 2 + 2
    In [2]: a
    Out[2]: 4
    In [3]: %hist
    a = 2 + 2
    a
    %hist
    

    This is a common feature in command line interface (CLI) environments. We can also search through the history with the -g switch as follows:

    In [5]: %hist -g a = 2
          1: a = 2 + 2
    

We saw a number of so-called magic functions in action. These functions start with the % character. If the magic function is used on a line by itself, the % prefix is optional.