Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Apache Spark for Machine Learning
  • Table Of Contents Toc
Apache Spark for Machine Learning

Apache Spark for Machine Learning

By : Deepak Gowda
4.5 (2)
close
close
Apache Spark for Machine Learning

Apache Spark for Machine Learning

4.5 (2)
By: Deepak Gowda

Overview of this book

In the world of big data, efficiently processing and analyzing massive datasets for machine learning can be a daunting task. Written by Deepak Gowda, a data scientist with over a decade of experience and 30+ patents, this book provides a hands-on guide to mastering Spark’s capabilities for efficient data processing, model building, and optimization. With Deepak’s expertise across industries such as supply chain, cybersecurity, and data center infrastructure, he makes complex concepts easy to follow through detailed recipes. This book takes you through core machine learning concepts, highlighting the advantages of Spark for big data analytics. It covers practical data preprocessing techniques, including feature extraction and transformation, supervised learning methods with detailed chapters on regression and classification, and unsupervised learning through clustering and recommendation systems. You’ll also learn to identify frequent patterns in data and discover effective strategies to deploy and optimize your machine learning models. Each chapter features practical coding examples and real-world applications to equip you with the knowledge and skills needed to tackle complex machine learning tasks. By the end of this book, you’ll be ready to handle big data and create advanced machine learning models with Apache Spark.
Table of Contents (16 chapters)
close
close
Lock Free Chapter
1
Part 1: Introduction and Fundamentals
5
Part 2: Supervised Learning
8
Part 3: Unsupervised Learning
12
Part 4: Model Deployment

Ingesting data

Data ingestion is the process of importing and loading data into a system, such as a database, a data warehouse, or a data lake. Data ingestion can be done manually or automatically, using various tools and techniques. Data ingestion is the first step in data analysis and machine learning, as it prepares the data for further processing and usage.

Apache Spark is a powerful, distributed data processing system that can read from a wide variety of data sources. Its ability to integrate with many diverse types of data storage systems is one of the reasons for its popularity in big data processing and analytics. Here are some of the key data sources from which Apache Spark can ingest data.

Filesystems

Let’s explore an example of data ingestion from Hadoop Distributed File System (HDFS). Here is a sample code snippet to read data from HDFS:

from pyspark.sql import SparkSession
spark = SparkSession.builder \
    .appName("HDFS Read...
CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Apache Spark for Machine Learning
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon