Book Image

Learning Google BigQuery

By : Thirukkumaran Haridass, Mikhail Berlyant, Eric Brown
Book Image

Learning Google BigQuery

By: Thirukkumaran Haridass, Mikhail Berlyant, Eric Brown

Overview of this book

Google BigQuery is a popular cloud data warehouse for large-scale data analytics. This book will serve as a comprehensive guide to mastering BigQuery, and how you can utilize it to quickly and efficiently get useful insights from your Big Data. You will begin with getting a quick overview of the Google Cloud Platform and the various services it supports. Then, you will be introduced to the Google BigQuery API and how it fits within in the framework of GCP. The book covers useful techniques to migrate your existing data from your enterprise to Google BigQuery, as well as readying and optimizing it for analysis. You will perform basic as well as advanced data querying using BigQuery, and connect the results to various third party tools for reporting and visualization purposes such as R and Tableau. If you're looking to implement real-time reporting of your streaming data running in your enterprise, this book will also help you. This book also provides tips, best practices and mistakes to avoid while working with Google BigQuery and services that interact with it. By the time you're done with it, you will have set a solid foundation in working with BigQuery to solve even the trickiest of data problems.

Related Content you might be interested in

Current Title:

Small book image
Learning Google BigQuery
Thirukkumaran Haridass | Mikhail Berlyant | Eric Brown
Dec, 2017 | 264 pages
Small book image
Data Exploration and Preparation with BigQuery
Mike Kahn
Nov, 2023 | 264 pages
Small book image
Google Cloud Platform for Architects
Vitthal Srinivasan | Judy Raj | Janani Ravi
Jun, 2018 | 372 pages
Small book image
Google Cloud Platform for Developers
Steven Porter | Ted Hunter
Jul, 2018 | 506 pages
Table of Contents (9 chapters)
Preface
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Free Chapter
1
Google Cloud and Google BigQuery
Google Cloud and Google BigQuery
Getting started with Google Cloud
Summary
2
Google Cloud SDK
Google Cloud SDK
Installing Google Cloud SDK
gsutil for Google Cloud Storage
Using the bq utility for BigQuery
Using the gcloud utility
Connecting to Cloud SQL using gcloud
Deploying to Google App Engine
Summary
3
Google BigQuery Data Types
Google BigQuery Data Types
Supported data types
Data type considerations
Converting data
Sanitizing data
When to transform your data? Before or after loading to BigQuery?
Arithmetic Operators
Comparison Operators
Date Time Functions
String Functions
Regular Expression Functions
Functions for transformation
Mastering transformation with User-Defined Functions
Summary
Further Reading
4
BigQuery SQL Basic
BigQuery SQL Basic
The BigQuery interface
Summary
Further reading
5
BigQuery SQL Advanced
BigQuery SQL Advanced
Partition tables
Querying external data sources using BigQuery
Wildcard tables
User-defined functions
Views
Querying nested and repeated records
Summary
Further reading
6
Google BigQuery API
Google BigQuery API
Accessing Google BigQuery
Programming with BigQuery API in C# .NET
Programming with BigQuery API in Python
Roles and permissions
Summary
7
Visualizing BigQuery Data
Visualizing BigQuery Data
Why is data visualization important?
The danger of summary statistics
Making data visualization work for you
Three tools for visualizing BigQuery data
Summary
8
Google Cloud Pub/Sub
Google Cloud Pub/Sub
Introduction
Getting started with Cloud Pub/Sub
Cloud Pub/Sub via Google Cloud Console
Cloud Pub/Sub via Google Cloud SDK
Cloud Pub/Sub pricing
Message output formats
Importing message data into BigQuery
Google Cloud Dataprep
Summary
Further reading

Programming with BigQuery API in Python

The following section covers the interaction with BigQuery API using the Python programming language. The code samples cover how to create a new dataset and new table in BigQuery, load data from Google Cloud Storage to the table, execute a query, and return the results or copy the data to a new table.

To get started in Python, make sure the Google Cloud SDK is installed and configured as mentioned in Chapter 2, Google Cloud SDK. Python version 3.6.1 was used in coding these samples. To install the BigQuery API client library for Python, run the following command from the command prompt:

pip install --upgrade google-cloud-bigquery

If the project is going to have other services as well, then it is also better to install client libraries for other services by running the command given here:

pip install --upgrade google-cloud
...
Previous Section
End of Section 4
Next Section