Spark allows you to look at basic charts without much code. Using the magic symbol at the top of the notebook segment, you can change language easily from Python to Scala or SQL. One word of caution about using Databricks' built-in charting system is that it only looks at the first 10,000 records. For a large dataset, there are other charting libraries. The steps are as follows:
- Query the data in Databricks using the %sql magic, as shown:
%sql
select * from Telemetry
- Select the chart icon at the bottom of the returned data grid. It will bring up the chart builder UI, as shown in the following screenshot:
- Select the chart type that best represents the data. Some charts are better suited for variable comparison while others can help reveal trends.
The following section reviews when and why you would use different chart types.