Book Image

Practical Data Analysis - Second Edition

By : Hector Cuesta, Dr. Sampath Kumar
Book Image

Practical Data Analysis - Second Edition

By: Hector Cuesta, Dr. Sampath Kumar

Overview of this book

Beyond buzzwords like Big Data or Data Science, there are a great opportunities to innovate in many businesses using data analysis to get data-driven products. Data analysis involves asking many questions about data in order to discover insights and generate value for a product or a service. This book explains the basic data algorithms without the theoretical jargon, and you’ll get hands-on turning data into insights using machine learning techniques. We will perform data-driven innovation processing for several types of data such as text, Images, social network graphs, documents, and time series, showing you how to implement large data processing with MongoDB and Apache Spark.
Table of Contents (21 chapters)
Practical Data Analysis - Second Edition
Credits
About the Authors
About the Reviewers
www.PacktPub.com
Preface

An introduction to Apache Spark


Apache Spark is an open source cluster computer system with implicit data parallelism and fault tolerance. Spark was originally created at AMPlab from UC Berkeley; the main goal of Spark is to be fast to run and read and to apply in-memory processing. Spark allows you to manipulate distributed datasets, such as local collections. In this section, we will present the basic operations with Spark programming model and its ecosystem.

Tip

We can find more information about pandas from its website: http://spark.apache.org/

The Spark ecosystem

Spark comes with a lot of high-level libraries for SQL querying, machine learning, graph processing, and streaming data. These libraries provide all inclusive environment ready to use. The following figure shows the complete Spark ecosystem:

Take a look at the following:

  • Spark Core API:

    The characteristics of Spark Core API are as follows:

    • It is the execution engine that allows all the other functionalities built on top

    • It provides...