What this book covers
Each of the chapters in this book takes the approach of introducing important concepts and key AWS services and then providing a hands-on exercise related to the topic of the chapter:
Chapter 1, An Introduction to Data Engineering, reviews the challenges of ever-increasing datasets, and the role of the data engineer in working with data in the cloud.
Chapter 2, Data Management Architectures for Analytics, introduces foundational concepts and technologies related to big data processing.
Chapter 3, The AWS Data Engineer's Toolkit, provides an introduction to a wide range of AWS services that are used for ingesting, processing, and consuming data.
Chapter 4, Data Cataloging, Security, and Governance, covers the all-important topics of keeping data secure, ensuring good data governance, and the importance of cataloging your data.
Chapter 5, Architecting Data Engineering Pipelines, provides an approach for whiteboarding the high-level design of a data engineering pipeline.
Chapter 6, Ingesting Batch and Streaming Data, looks at the variety of data sources that we may need to ingest from and examines AWS services for ingesting both batch and streaming data.
Chapter 7, Transforming Data to Optimize for Analytics, covers common transformations for optimizing datasets and for applying business logic.
Chapter 8, Identifying and Enabling Data Consumers, is about better understanding the different types of data consumers that a data engineer may work to prepare data for.
Chapter 9, Loading Data into a Data Mart, focuses on the use of data warehouses as a data mart and looks at moving data between a data lake and data warehouse. This chapter also does a deep dive into Amazon Redshift, a cloud-based data warehouse.
Chapter 10, Orchestrating the Data Pipeline, looks at how various data engineering tasks and transformations can be put together in a data pipeline, and how these can be run and managed with pipeline orchestration tools such as AWS Step Functions.
Chapter 11, Ad Hoc Queries with Amazon Athena, does a deeper dive into the Amazon Athena service, which can be used for running SQL queries directly on data in the data lake, and beyond.
Chapter 12, Visualizing Data with Amazon QuickSight, discusses the importance of being able to craft visualizations of data, and how the Amazon QuickSight service enables this.
Chapter 13, Enabling Artificial Intelligence and Machine Learning, reviews how AI and ML are increasingly important for gaining new value from data, and introduces some of the AWS services for both ML and AI.
Chapter 14, Wrapping Up the First Part of Your Learning Journey, concludes the book by looking at the bigger picture of data analytics, including real-world examples of data pipelines and a review of emerging trends in the industry.