Modern Data Architectures with Python
By :
Modern Data Architectures with Python
By:
Overview of this book
Modern Data Architectures with Python will teach you how to seamlessly incorporate your machine learning and data science work streams into your open data platforms. You’ll learn how to take your data and create open lakehouses that work with any technology using tried-and-true techniques, including the medallion architecture and Delta Lake.
Starting with the fundamentals, this book will help you build pipelines on Databricks, an open data platform, using SQL and Python. You’ll gain an understanding of notebooks and applications written in Python using standard software engineering tools such as git, pre-commit, Jenkins, and Github. Next, you’ll delve into streaming and batch-based data processing using Apache Spark and Confluent Kafka. As you advance, you’ll learn how to deploy your resources using infrastructure as code and how to automate your workflows and code development. Since any data platform's ability to handle and work with AI and ML is a vital component, you’ll also explore the basics of ML and how to work with modern MLOps tooling. Finally, you’ll get hands-on experience with Apache Spark, one of the key data technologies in today’s market.
By the end of this book, you’ll have amassed a wealth of practical and theoretical knowledge to build, manage, orchestrate, and architect your data ecosystems.
Table of Contents (19 chapters)
Preface
Part 1:Fundamental Data Knowledge
Free Chapter
Chapter 1: Modern Data Processing Architecture
Chapter 2: Understanding Data Analytics
Part 2: Data Engineering Toolset
Chapter 3: Apache Spark Deep Dive
Chapter 4: Batch and Stream Data Processing Using PySpark
Chapter 5: Streaming Data with Kafka
Part 3:Modernizing the Data Platform
Chapter 6: MLOps
Chapter 7: Data and Information Visualization
Chapter 8: Integrating Continous Integration into Your Workflow
Chapter 9: Orchestrating Your Data Workflows
Part 4:Hands-on Project
Chapter 10: Data Governance
Chapter 11: Building out the Groundwork
Chapter 12: Completing Our Project
Index
Customer Reviews