Book Image

Implementing Cloud Design Patterns for AWS - Second Edition

By : Sean Keery, Clive Harber, Marcus Young
Book Image

Implementing Cloud Design Patterns for AWS - Second Edition

By: Sean Keery, Clive Harber, Marcus Young

Overview of this book

Whether you're just getting your feet wet in cloud infrastructure or already creating complex systems, this book will guide you through using the patterns to fit your system needs. Starting with patterns that cover basic processes such as source control and infrastructure-as-code, the book goes on to introduce cloud security practices. You'll then cover patterns of availability and scalability and get acquainted with the ephemeral nature of cloud environments. You'll also explore advanced DevOps patterns in operations and maintenance, before focusing on virtualization patterns such as containerization and serverless computing. In the final leg of your journey, this book will delve into data persistence and visualization patterns. You'll get to grips with architectures for processing static and dynamic data, as well as practices for managing streaming data. By the end of this book, you will be able to design applications that are tolerant of underlying hardware failures, resilient against an unexpected influx of data, and easy to manage and replicate.
Table of Contents (20 chapters)
Title Page
Dedication
About Packt
Contributors
Preface
Free Chapter
1
Introduction to Amazon Web Services
Index

Batching


Sometimes, your data sources produce too much data to process, the operation that you want to use over the data uses is too intensive to process on collection, or you need to analyze data but it doesn't need to be in real time (whatever that means for your application). In order to process this data effectively, it needs to happen away from the action, so to speak, in a remote system or at an off-peak time. Batch processing happens periodically.

Batch processing is a useful tool in the arsenal of data scientists, developers, and engineers. Being able to process large amounts of data for further analysis or for presentation to business users without overloading your application services or databases allows you to schedule and execute your analysis patterns across a number of AWS services.

In order to set this service up, perform the following steps:

  1. Create an IAM role and security group:
resource "aws_iam_role" "ecs_instance_role" {
  name = "ecs_instance_role"

  assume_role_policy...