Exploratory Data Analysis (EDA), or Initial Data Analysis (IDA), is an approach to data analysis that attempts to maximize insight into data. This includes assessing the quality and structure of the data, calculating summary or descriptive statistics, and plotting appropriate graphs. It can uncover underlying structures and suggest how the data should be modeled. Furthermore, EDA helps us detect outliers, errors, and anomalies in our data, and deciding what to do about such data is often more important than other, more sophisticated analysis. EDA enables us to test our underlying assumptions, discover clusters and other patterns in our data, and identify the possible relationships between various variables. A careful EDA process is vital to understanding the data and is sometimes sufficient to reveal such poor data quality that using...
-
Book Overview & Buying
-
Table Of Contents
Learning Spark SQL
By :
Learning Spark SQL
By:
Overview of this book
In the past year, Apache Spark has been increasingly adopted for the development of distributed applications. Spark SQL APIs provide an optimized interface that helps developers build such applications quickly and easily. However, designing web-scale production applications using Spark SQL APIs can be a complex task. Hence, understanding the design and implementation best practices before you start your project will help you avoid these problems.
This book gives an insight into the engineering practices used to design and build real-world, Spark-based applications. The book's hands-on examples will give you the required confidence to work on any future projects you encounter in Spark SQL.
It starts by familiarizing you with data exploration and data munging tasks using Spark SQL and Scala. Extensive code examples will help you understand the methods used to implement typical use-cases for various types of applications. You will get a walkthrough of the key concepts and terms that are common to streaming, machine learning, and graph applications. You will also learn key performance-tuning details including Cost Based Optimization (Spark 2.2) in Spark SQL applications. Finally, you will move on to learning how such systems are architected and deployed for a successful delivery of your project.
Table of Contents (13 chapters)
Preface
Free Chapter
Getting Started with Spark SQL
Using Spark SQL for Processing Structured and Semistructured Data
Using Spark SQL for Data Exploration
Using Spark SQL for Data Munging
Using Spark SQL in Streaming Applications
Using Spark SQL in Machine Learning Applications
Using Spark SQL in Graph Applications
Using Spark SQL with SparkR
Developing Applications with Spark SQL
Using Spark SQL in Deep Learning Applications
Tuning Spark SQL Components for Performance
Spark SQL in Large-Scale Application Architectures
