Book Image

Hands-On Data Science with the Command Line

By : Jason Morris, Chris McCubbin, Raymond Page
Book Image

Hands-On Data Science with the Command Line

By: Jason Morris, Chris McCubbin, Raymond Page

Overview of this book

The Command Line has been in existence on UNIX-based OSes in the form of Bash shell for over 3 decades. However, very little is known to developers as to how command-line tools can be OSEMN (pronounced as awesome and standing for Obtaining, Scrubbing, Exploring, Modeling, and iNterpreting data) for carrying out simple-to-advanced data science tasks at speed. This book will start with the requisite concepts and installation steps for carrying out data science tasks using the command line. You will learn to create a data pipeline to solve the problem of working with small-to medium-sized files on a single machine. You will understand the power of the command line, learn how to edit files using a text-based and an. You will not only learn how to automate jobs and scripts, but also learn how to visualize data using the command line. By the end of this book, you will learn how to speed up the process and perform automated tasks using command-line tools.

Related Content you might be interested in

Current Title:

Small book image
Hands-On Data Science with the Command Line
Jason Morris | Chris McCubbin | Raymond Page
Jan, 2019 | 124 pages
Small book image
Command Line Fundamentals
Vivek Nagarajan
Dec, 2018 | 314 pages
Small book image
Bash Quick Start Guide
Tom Ryder
Sep, 2018 | 186 pages
Small book image
Learning Linux Shell Scripting.
Ganesh Naik
May, 2018 | 332 pages
Table of Contents (8 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Data Science at the Command Line and Setting It Up
Data Science at the Command Line and Setting It Up
History of the command line
We don't want to BaSH other shells, but...
Language-focused shells
So, why the command line?
Getting set up with Windows 10
Getting set up on OS X
Getting set up on Ubuntu Linux
Summary
2
Essential Commands
Essential Commands
Essential commands
Navigating the command line
Summary
3
Shell Workflows, and Data Acquisition and Massaging
Shell Workflows, and Data Acquisition and Massaging
Download the data
Using the file command
Performing a word count
Introduction to cut
Detached processing
Summary
4
Bash Functions and Data Visualization
Bash Functions and Data Visualization
My first shell script
Advanced shell scripting magic
From dumb Terminal to glam Terminal
Summary
5
Loops, Functions, and String Processing
Loops, Functions, and String Processing
Once, twice, three times a lady loops
It's the end of the world as we know it while and until
The simple case
Pay no heed to the magician redirecting your attention
Regular expressions and grep
awk, sed, and tr
Summary
6
SQL, Math, and Wrapping it up
SQL, Math, and Wrapping it up
cut and viewing data as columnar
Simulating selects
Keys to the kingdom
Math in bash itself
Python (pandas, numpy, scikit-learn)
Analyzing weather data in bash
Summary
7
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Keys to the kingdom

Now that we can explore data with the command line and have mastered transforming text, we'll provide you with the keys to the kingdom. SQLite is a public domain library that implements a SQL engine and provides a sqlite command shell for interacting with database files. Unlike Oracle, MySQL, and other database engines that provide a network endpoint, sqlite is offline and locally driven by library calls to interact with a single file that is the entire database. This makes backups easy. Backups can be created by doing cp database.sq3 backups/`date +%F`-database.sq3. One can version control it, but that's unlikely to compress well with delta comparisons.

Using SQLite

Easy import of CSV files ...

Previous Section
End of Section 4
Next Section