This section will provide some important aspects of the Spark driver's UI. We will see the statistics of the jobs we executed using Spark shell on Spark UI.
As described in the Getting started with Apache Spark section, Spark driver's UI runs at http://localhost:4040/
(unless you make any changes to default settings).
When you start Spark shell, Spark driver's UI will look as follows:

SparkContext
is an entry point to every Spark application. Every Spark job is launched with a SparkContext
and can consist of only one SparkContext
.
Spark shell, being a Spark application starts with SparkContext
and every SparkContext
launches its own web UI. The default port is 4040
. Spark UI can be enabled/disabled or can be launched on a separate port using the following properties:
Property | Default value |
|
|
|
|
For example, Spark shell application with Spark UI running on 5050
port can be launched as:
spark-shell --confspark.ui.port=5050
If multiple Spark applications...