Book Image

Solutions Architect's Handbook - Second Edition

By : Saurabh Shrivastava, Neelanjali Srivastav
4 (2)
Book Image

Solutions Architect's Handbook - Second Edition

4 (2)
By: Saurabh Shrivastava, Neelanjali Srivastav

Overview of this book

Becoming a solutions architect requires a hands-on approach, and this edition of the Solutions Architect's Handbook brings exactly that. This handbook will teach you how to create robust, scalable, and fault-tolerant solutions and next-generation architecture designs in a cloud environment. It will also help you build effective product strategies for your business and implement them from start to finish. This new edition features additional chapters on disruptive technologies, such as Internet of Things (IoT), quantum computing, data engineering, and machine learning. It also includes updated discussions on cloud-native architecture, blockchain data storage, and mainframe modernization with public cloud. The Solutions Architect's Handbook provides an understanding of solution architecture and how it fits into an agile enterprise environment. It will take you through the journey of solution architecture design by providing detailed knowledge of design pillars, advanced design patterns, anti-patterns, and the cloud-native aspects of modern software design. By the end of this handbook, you'll have learned the techniques needed to create efficient architecture designs that meet your business requirements.
Table of Contents (22 chapters)
20
Other Books You May Enjoy
21
Index

Big data architecture best practices

In previous sections, you learned about various big data technology and architecture patterns. Let's look at the following reference architecture diagram with different layers of a data lake architecture to learn best practices.

Figure 13.12: Data lake reference architecture

The preceding diagram depicts an end-to-end data pipeline in a data lake architecture using the AWS cloud platform with the following components:

  • AWS Direct Connect to set up a high-speed network connection between the on-premises data center and AWS to migrate data. If you have large volumes of archive data, it's better to use the AWS Snow family to move it offline.
  • A data ingestion layer with various components to ingest streaming data using Amazon Kinesis, relational data using AWS Data Migration Service (DMS), secure file transfer using AWS Transfer for SFTP, and AWS DataSync to update data files between cloud and on-prem systems...