Book Overview & Buying
Table Of Contents

Apache Spark Quick Start Guide

By : Shrey Mehrotra, Grade

3 (1)

Buy this Book

Apache Spark Quick Start Guide

3 (1)

By: Shrey Mehrotra, Grade

Buy this Book

Overview of this book

Apache Spark is a ?exible framework that allows processing of batch and real-time data. Its unified engine has made it quite popular for big data use cases. This book will help you to get started with Apache Spark 2.0 and write big data applications for a variety of use cases. It will also introduce you to Apache Spark – one of the most popular Big Data processing frameworks. Although this book is intended to help you get started with Apache Spark, but it also focuses on explaining the core concepts. This practical guide provides a quick start to the Spark 2.0 architecture and its components. It teaches you how to set up Spark on your local machine. As we move ahead, you will be introduced to resilient distributed datasets (RDDs) and DataFrame APIs, and their corresponding transformations and actions. Then, we move on to the life cycle of a Spark application and learn about the techniques used to debug slow-running applications. You will also go through Spark’s built-in modules for SQL, streaming, machine learning, and graph analysis. Finally, the book will lay out the best practices and optimization techniques that are key for writing efficient Spark applications. By the end of this book, you will have a sound fundamental understanding of the Apache Spark framework and you will be able to write and optimize Spark applications.

Preface

Who this book is for

What this book covers

To get the most out of this book

Get in touch

Free Chapter

Introduction to Apache Spark

What is Spark?

Spark architecture overview

Spark language APIs

Spark components

Making the most of Hadoop and Spark

Summary

Apache Spark Installation

AWS elastic compute cloud (EC2)

Configuring Spark

Summary

Spark RDD

What is an RDD?

Programming using RDDs

Transformations and actions

Types of RDDs

Caching and checkpointing

Understanding partitions

Drawbacks of using RDDs

Summary

Spark DataFrame and Dataset

DataFrames

Datasets

Summary

Spark Architecture and Application Execution Flow

A sample application

Application execution modes

Application monitoring

Summary

Spark SQL

Summary

Spark Streaming, Machine Learning, and Graph Analysis

Spark Streaming

Machine learning

Graph processing

Summary

Spark Optimizations

Cluster-level optimizations

Application optimizations

Summary

Other Books You May Enjoy

Leave a review - let other readers know what you think

DataFrames

As we already mentioned, DataFrame APIs are abstractions of RDD APIs. DataFrames are distributed collections of data that are organized in the form of rows and columns. In other words, DataFrames provide APIs to efficiently process structured data that's available in different sources. The sources could be an RDD, different types of files in a filesystem, any RDBMS, or Hive tables.

The features of DataFrames are as follows:

DataFrames can process data that's available in different formats, such as CSV, AVRO, and JSON, or stored in any storage media, such as Hive, HDFS, and RDBMS
DataFrames can process data volumes from kilobytes to petabytes
Use the Spark-SQL query optimizer to process data in a distributed and optimized manner
Support for APIs in multiple languages, including Java, Scala, Python, and R

...

Tech Concepts

Programming languages

Tech Tools