Book Image

Learning Google BigQuery

By : Thirukkumaran Haridass, Mikhail Berlyant, Eric Brown

Book Image

Learning Google BigQuery

By: Thirukkumaran Haridass, Mikhail Berlyant, Eric Brown

Overview of this book

Google BigQuery is a popular cloud data warehouse for large-scale data analytics. This book will serve as a comprehensive guide to mastering BigQuery, and how you can utilize it to quickly and efficiently get useful insights from your Big Data. You will begin with getting a quick overview of the Google Cloud Platform and the various services it supports. Then, you will be introduced to the Google BigQuery API and how it fits within in the framework of GCP. The book covers useful techniques to migrate your existing data from your enterprise to Google BigQuery, as well as readying and optimizing it for analysis. You will perform basic as well as advanced data querying using BigQuery, and connect the results to various third party tools for reporting and visualization purposes such as R and Tableau. If you're looking to implement real-time reporting of your streaming data running in your enterprise, this book will also help you. This book also provides tips, best practices and mistakes to avoid while working with Google BigQuery and services that interact with it. By the time you're done with it, you will have set a solid foundation in working with BigQuery to solve even the trickiest of data problems.

Preface

What this book covers

What you need for this book

Who this book is for

Reader feedback

Customer support

Free Chapter

Google Cloud and Google BigQuery

Google Cloud and Google BigQuery

Getting started with Google Cloud

Google Cloud SDK

Google Cloud SDK

Installing Google Cloud SDK

gsutil for Google Cloud Storage

Using the bq utility for BigQuery

Using the gcloud utility

Connecting to Cloud SQL using gcloud

Deploying to Google App Engine

Google BigQuery Data Types

Google BigQuery Data Types

Supported data types

Data type considerations

Converting data

Sanitizing data

When to transform your data? Before or after loading to BigQuery?

Arithmetic Operators

Comparison Operators

Date Time Functions

String Functions

Regular Expression Functions

Functions for transformation

Mastering transformation with User-Defined Functions

Further Reading

BigQuery SQL Basic

BigQuery SQL Basic

The BigQuery interface

Further reading

BigQuery SQL Advanced

BigQuery SQL Advanced

Partition tables

Querying external data sources using BigQuery

Wildcard tables

User-defined functions

Querying nested and repeated records

Further reading

Google BigQuery API

Google BigQuery API

Accessing Google BigQuery

Programming with BigQuery API in C# .NET

Programming with BigQuery API in Python

Roles and permissions

Visualizing BigQuery Data

Visualizing BigQuery Data

Why is data visualization important?

The danger of summary statistics

Making data visualization work for you

Three tools for visualizing BigQuery data

Google Cloud Pub/Sub

Google Cloud Pub/Sub

Getting started with Cloud Pub/Sub

Cloud Pub/Sub via Google Cloud Console

Cloud Pub/Sub via Google Cloud SDK

Cloud Pub/Sub pricing

Message output formats

Importing message data into BigQuery

Google Cloud Dataprep

Further reading

Customer Reviews

5 star

0

4 star

0

3 star

0

2 star

0

1 star

0

Using the gcloud utility

The gcloud utility is used to interact with the rest of the services on the Google Cloud Platform, other than BigQuery and Google Cloud Storage. The commands in the gcloud utility are grouped for each service. The following are the service groups for some of the services on the Google Cloud Platform:

Service group	Google Cloud service
App	App Engine standard and flexible environment
Compute	Compute Engine to manage virtual machines
Container	Container Engine to manage containers and clusters
Dataflow	Manage Cloud Dataflow services for ETL and data processing
Dataproc	Manage Cloud Dataproc service which consists of Apache Hadoop, Spark, Pig, and Hive
Datastore	Manage Cloud Datastore service which creates entities on a NoSQL database
SQL	Manage Cloud SQL service which consists of MySQL or PostgreSQL databases...