Book Image

Hands-On Data Science with the Command Line

By : Jason Morris, Chris McCubbin, Raymond Page
Book Image

Hands-On Data Science with the Command Line

By: Jason Morris, Chris McCubbin, Raymond Page

Overview of this book

The Command Line has been in existence on UNIX-based OSes in the form of Bash shell for over 3 decades. However, very little is known to developers as to how command-line tools can be OSEMN (pronounced as awesome and standing for Obtaining, Scrubbing, Exploring, Modeling, and iNterpreting data) for carrying out simple-to-advanced data science tasks at speed. This book will start with the requisite concepts and installation steps for carrying out data science tasks using the command line. You will learn to create a data pipeline to solve the problem of working with small-to medium-sized files on a single machine. You will understand the power of the command line, learn how to edit files using a text-based and an. You will not only learn how to automate jobs and scripts, but also learn how to visualize data using the command line. By the end of this book, you will learn how to speed up the process and perform automated tasks using command-line tools.
Table of Contents (8 chapters)

Introduction to cut

Let's break the command down before you run it. The cut command removes sections from each line of a file. The -d parameter tells cut we are working with a tsv (tab separated values), and the -f parameter tells cut what fields we are interested in. Since product_title is the sixth field in our file, we started with that:

cut -d$'\t' -f 6,8,13,14 reviews.tsv | more
Unlike most programs, cut starts at 1 instead of 0.

Let’s see the results:

Much better! Let's go ahead and save this as a new file:

cut -d$'\t' -f 6,8,13,14 reviews.tsv > stripped_reviews.tsv

The following is what you should see once you run the preceding command:

Let's see how many times the word Packt shows up in this dataset:

grep -i Packt stripped_reviews.tsv | wc -w

The following is what you should see once you run the preceding command:

Let&apos...