This section will provide some important aspects of the Spark driver's UI. We will see the statistics of the jobs we executed using Spark shell on Spark UI.
As described in the Getting started with Apache Spark section, Spark driver's UI runs at
http://localhost:4040/ (unless you make any changes to default settings).
When you start Spark shell, Spark driver's UI will look as follows:
SparkContext is an entry point to every Spark application. Every Spark job is launched with a
SparkContext and can consist of only one
Spark shell, being a Spark application starts with
SparkContext and every
SparkContext launches its own web UI. The default port is
4040. Spark UI can be enabled/disabled or can be launched on a separate port using the following properties:
For example, Spark shell application with Spark UI running on
5050 port can be launched as:
If multiple Spark applications...