Book Image

Building Google Cloud Platform Solutions

By : Ted Hunter, Steven Porter, Legorie Rajan PS
Book Image

Building Google Cloud Platform Solutions

By: Ted Hunter, Steven Porter, Legorie Rajan PS

Overview of this book

GCP is a cloud computing platform with a wide range of products and services that enable you to build and deploy cloud-hosted applications. This Learning Path will guide you in using GCP and designing, deploying, and managing applications on Google Cloud. You will get started by learning how to use App Engine to access Google's scalable hosting and build software that runs on this framework. With the help of Google Compute Engine, you’ll be able to host your workload on virtual machine instances. The later chapters will help you to explore ways to implement authentication and security, Cloud APIs, and command-line and deployment management. As you hone your skills, you’ll understand how to integrate your new applications with various data solutions on GCP, including Cloud SQL, Bigtable, and Cloud Storage. Following this, the book will teach you how to streamline your workflow with tools, including Source Repositories, Container Builder, and Stackdriver. You'll also understand how to deploy and debug services with IntelliJ, implement continuous delivery pipelines, and configure robust monitoring and alerts for your production systems. By the end of this Learning Path, you'll be well versed with GCP’s development tools and be able to develop, deploy, and manage highly scalable and reliable applications. This Learning Path includes content from the following Packt products: • Google Cloud Platform for Developers Ted Hunter and Steven Porter • Google Cloud Platform Cookbook by Legorie Rajan PS
Table of Contents (29 chapters)
Title Page
Copyright and Credits
About Packt
Contributors
Preface
Index

Creating a Dataflow pipeline to store streaming data


Google Dataflow is a service for stream and batch processing at scale. When there is a need for processing lots of streamed data like click stream or data from IoT devices, Dataflow will be the starting point for receiving all the stream data. The data can then be sent to storage (BigQuery, Bigtable, GCS) for further processing (ML):

For this recipe, let's consider a weather station (IoT device) that is sending temperature data to GCP. The data is emitted constantly by the IoT device and is stored on Google Storage for later analytics processing. Considering the intermittent nature of data connectivity between the device and GCP, we'll need a solution to receive the messages, process/handle them, and store them. For this solution, we'll create a Dataflow pipeline using a Google provided template—Cloud Pub/Sub to Cloud Storage text.

Getting ready

The following are the initial setup verification steps for the creation of the network before...