Book Image

Pandas 1.x Cookbook - Second Edition

By : Matthew Harrison, Theodore Petrou
Book Image

Pandas 1.x Cookbook - Second Edition

By: Matthew Harrison, Theodore Petrou

Overview of this book

The pandas library is massive, and it's common for frequent users to be unaware of many of its more impressive features. The official pandas documentation, while thorough, does not contain many useful examples of how to piece together multiple commands as one would do during an actual analysis. This book guides you, as if you were looking over the shoulder of an expert, through situations that you are highly likely to encounter. This new updated and revised edition provides you with unique, idiomatic, and fun recipes for both fundamental and advanced data manipulation tasks with pandas. Some recipes focus on achieving a deeper understanding of basic principles, or comparing and contrasting two similar operations. Other recipes will dive deep into a particular dataset, uncovering new and unexpected insights along the way. Many advanced recipes combine several different features across the pandas library to generate results.
Table of Contents (17 chapters)
15
Other Books You May Enjoy
16
Index

Filtering columns with time data

The last section showed how to filter data that has a DatetimeIndex. Often, you will have columns with dates in them, and it does not make sense to have that column be the index. In this section, we will reproduce the slicing of the preceding section with columns. Sadly, the slicing constructs do not work on columns, so we will have to take a different tack.

How to do it…

  1. Read in the Denver crimes dataset from the hdf5 file crimes.h5 and inspect the column types:
    >>> crime = pd.read_hdf('data/crime.h5', 'crime')
    >>> crime.dtypes
    OFFENSE_TYPE_ID              category
    OFFENSE_CATEGORY_ID          category
    REPORTED_DATE          datetime64[ns]
    GEO_LON                       float64
    GEO_LAT                       float64
    NEIGHBORHOOD_ID              category
    IS_CRIME                        int64
    IS_TRAFFIC                      int64
    dtype: object
    
  2. Select all the rows where...