Building ETL Pipelines with Python
By :
Building ETL Pipelines with Python
By:
Overview of this book
Modern extract, transform, and load (ETL) pipelines for data engineering have favored the Python language for its broad range of uses and a large assortment of tools, applications, and open source components. With its simplicity and extensive library support, Python has emerged as the undisputed choice for data processing.
In this book, you’ll walk through the end-to-end process of ETL data pipeline development, starting with an introduction to the fundamentals of data pipelines and establishing a Python development environment to create pipelines. Once you've explored the ETL pipeline design principles and ET development process, you'll be equipped to design custom ETL pipelines. Next, you'll get to grips with the steps in the ETL process, which involves extracting valuable data; performing transformations, through cleaning, manipulation, and ensuring data integrity; and ultimately loading the processed data into storage systems. You’ll also review several ETL modules in Python, comparing their pros and cons when building data pipelines and leveraging cloud tools, such as AWS, to create scalable data pipelines. Lastly, you’ll learn about the concept of test-driven development for ETL pipelines to ensure safe deployments.
By the end of this book, you’ll have worked on several hands-on examples to create high-performance ETL pipelines to develop robust, scalable, and resilient environments using Python.
Table of Contents (22 chapters)
Preface
Part 1:Introduction to ETL, Data Pipelines, and Design Principles
Free Chapter
Chapter 1: A Primer on Python and the Development Environment
Chapter 2: Understanding the ETL Process and Data Pipelines
Chapter 3: Design Principles for Creating Scalable and Resilient Pipelines
Part 2:Designing ETL Pipelines with Python
Chapter 4: Sourcing Insightful Data and Data Extraction Strategies
Chapter 5: Data Cleansing and Transformation
Chapter 6: Loading Transformed Data
Chapter 7: Tutorial – Building an End-to-End ETL Pipeline in Python
Chapter 8: Powerful ETL Libraries and Tools in Python
Part 3:Creating ETL Pipelines in AWS
Chapter 9: A Primer on AWS Tools for ETL Processes
Chapter 10: Tutorial – Creating an ETL Pipeline in AWS
Chapter 11: Building Robust Deployment Pipelines in AWS
Part 4:Automating and Scaling ETL Pipelines
Chapter 12: Orchestration and Scaling in ETL Pipelines
Chapter 13: Testing Strategies for ETL Pipelines
Chapter 14: Best Practices for ETL Pipelines
Chapter 15: Use Cases and Further Reading
Index
Customer Reviews