Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Databricks Certified Associate Developer for Apache Spark Using Python
  • Table Of Contents Toc
Databricks Certified Associate Developer for Apache Spark Using Python

Databricks Certified Associate Developer for Apache Spark Using Python

By : Saba Shah
5 (3)
close
close
Databricks Certified Associate Developer for Apache Spark Using Python

Databricks Certified Associate Developer for Apache Spark Using Python

5 (3)
By: Saba Shah

Overview of this book

Spark has become a de facto standard for big data processing. Migrating data processing to Spark saves resources, streamlines your business focus, and modernizes workloads, creating new business opportunities through Spark’s advanced capabilities. Written by a senior solutions architect at Databricks, with experience in leading data science and data engineering teams in Fortune 500s as well as startups, this book is your exhaustive guide to achieving the Databricks Certified Associate Developer for Apache Spark certification on your first attempt. You’ll explore the core components of Apache Spark, its architecture, and its optimization, while familiarizing yourself with the Spark DataFrame API and its components needed for data manipulation. You’ll also find out what Spark streaming is and why it’s important for modern data stacks, before learning about machine learning in Spark and its different use cases. What’s more, you’ll discover sample questions at the end of each section along with two mock exams to help you prepare for the certification exam. By the end of this book, you’ll know what to expect in the exam and gain enough understanding of Spark and its tools to pass the exam. You’ll also be able to apply this knowledge in a real-world setting and take your skillset to the next level.
Table of Contents (18 chapters)
close
close
1
Part 1: Exam Overview
3
Part 2: Introducing Spark
6
Part 3: Spark Operations
10
Part 4: Spark Applications
13
Part 5: Mock Papers
14
Chapter 9: Mock Test 1
15
Chapter 10: Mock Test 2

Execution hierarchy

Let’s look at the execution flow of a Spark application with the help of the architecture depicted in Figure 3.1:

Figure 3.1: Spark architecture

Figure 3.1: Spark architecture

These steps outline the flow from submitting a Spark job to freeing up resources when the job is completed:

  1. Spark executions start with a user submitting a spark-submit request to the Spark engine. This will create a Spark application. Once an action is performed, it will result in a job being created.
  2. This request will initiate communication with the cluster manager. In turn, the cluster manager initializes the Spark driver to execute the main() method of the Spark application. To execute this method, SparkSession is created.
  3. The driver starts communicating with the cluster manager and asks for resources to start planning for execution.
  4. The cluster manager then starts the executors, which can communicate with the driver directly.
  5. The driver creates a logical...
CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Databricks Certified Associate Developer for Apache Spark Using Python
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon