This book is for developers, data scientists, business analysts, or anyone who needs to reliably analyze large amounts of large-scale, real-world data. Whether you're tasked with creating your company's business intelligence function, or creating great data platforms for your machine learning models, or looking to use code to magnify the impact of your business, this book is for you.
Hands-On Big Data Analytics with PySpark
By :
Hands-On Big Data Analytics with PySpark
By:
Overview of this book
Apache Spark is an open source parallel-processing framework that has been around for quite some time now. One of the many uses of Apache Spark is for data analytics applications across clustered computers. In this book, you will not only learn how to use Spark and the Python API to create high-performance analytics with big data, but also discover techniques for testing, immunizing, and parallelizing Spark jobs.
You will learn how to source data from all popular data hosting platforms, including HDFS, Hive, JSON, and S3, and deal with large datasets with PySpark to gain practical big data experience. This book will help you work on prototypes on local machines and subsequently go on to handle messy data in production and at scale. This book covers installing and setting up PySpark, RDD operations, big data cleaning and wrangling, and aggregating and summarizing data into useful reports. You will also learn how to implement some practical and proven techniques to improve certain aspects of programming and administration in Apache Spark.
By the end of the book, you will be able to build big data analytical solutions using the various PySpark offerings and also optimize them effectively.
Table of Contents (15 chapters)
Preface
Free Chapter
Installing Pyspark and Setting up Your Development Environment
Getting Your Big Data into the Spark Environment Using RDDs
Big Data Cleaning and Wrangling with Spark Notebooks
Aggregating and Summarizing Data into Useful Reports
Powerful Exploratory Data Analysis with MLlib
Putting Structure on Your Big Data with SparkSQL
Transformations and Actions
Immutable Design
Avoiding Shuffle and Reducing Operational Expenses
Saving Data in the Correct Format
Working with the Spark Key/Value API
Testing Apache Spark Jobs
Leveraging the Spark GraphX API
Other Books You May Enjoy
Customer Reviews