Book Image

Business Intelligence with Databricks SQL

By : Vihag Gupta
Book Image

Business Intelligence with Databricks SQL

By: Vihag Gupta

Overview of this book

In this new era of data platform system design, data lakes and data warehouses are giving way to the lakehouse – a new type of data platform system that aims to unify all data analytics into a single platform. Databricks, with its Databricks SQL product suite, is the hottest lakehouse platform out there, harnessing the power of Apache Spark™, Delta Lake, and other innovations to enable data warehousing capabilities on the lakehouse with data lake economics. This book is a comprehensive hands-on guide that helps you explore all the advanced features, use cases, and technology components of Databricks SQL. You’ll start with the lakehouse architecture fundamentals and understand how Databricks SQL fits into it. The book then shows you how to use the platform, from exploring data, executing queries, building reports, and using dashboards through to learning the administrative aspects of the lakehouse – data security, governance, and management of the computational power of the lakehouse. You’ll also delve into the core technology enablers of Databricks SQL – Delta Lake and Photon. Finally, you’ll get hands-on with advanced SQL commands for ingesting data and maintaining the lakehouse. By the end of this book, you’ll have mastered Databricks SQL and be able to deploy and deliver fast, scalable business intelligence on the lakehouse.

Related Content you might be interested in

Current Title:

Small book image
Business Intelligence with Databricks SQL
Vihag Gupta
Sep, 2022 | 348 pages
Small book image
Optimizing Databricks Workloads
Sarthak Sarbahi | Anirudh Kala | Anshul Bhatnagar
Dec, 2021 | 230 pages
Small book image
Simplifying Data Engineering and Analytics with Delta
Anindita Mahapatra
Jul, 2022 | 334 pages
Small book image
Practical Machine Learning on Databricks
Debu Sinha
Nov, 2023 | 244 pages
Table of Contents (21 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Share Your Thoughts
1
Part 1: Databricks SQL on the Lakehouse
Part 1: Databricks SQL on the Lakehouse
Free Chapter
2
Chapter 1: Introduction to Databricks
Chapter 1: Introduction to Databricks
Technical requirements
An overview of Databricks, the company
An overview of the Lakehouse architecture
An overview of the Databricks Lakehouse platform
Summary
3
Chapter 2: The Databricks Product Suite – A Visual Tour
Chapter 2: The Databricks Product Suite – A Visual Tour
Technical requirements
Basic navigation with the sidebar
The SQL persona view
The Machine Learning persona view
The Data Science and Engineering persona view
Summary
4
Chapter 3: The Data Catalog
Chapter 3: The Data Catalog
Technical requirements
Understanding the data organization model in 
Databricks SQL
Exploring data visually with the Data Catalog
Exploring the data programmatically with SQL statements
Summary
5
Chapter 4: The Security Model
Chapter 4: The Security Model
Technical requirements
The Databricks SQL security model
User-facing table access control
The internals of cloud storage access
Summary
6
Chapter 5: The Workbench
Chapter 5: The Workbench
Technical requirements
Working with queries
Visualizing query results
Creating and publishing dashboards
Administering and governing artifacts
Summary
7
Chapter 6: The SQL Warehouses
Chapter 6: The SQL Warehouses
Technical requirements
Understanding the SQL Warehouse architecture
Creating and configuring SQL Warehouses
The art of SQL Warehouse sizing
Organizing and governing SQL Warehouses
Using Serverless SQL
Summary
8
Chapter 7: Using Business Intelligence Tools with Databricks SQL
Chapter 7: Using Business Intelligence Tools with Databricks SQL
Technical requirements
Connecting from validated BI tools
Connecting from non-validated BI tools
Connecting programmatically
Databricks Partner Connect
Summary
9
Part 2: Internals of Databricks SQL
Part 2: Internals of Databricks SQL
10
Chapter 8: The Delta Lake
Chapter 8: The Delta Lake
Technical requirements
Fundamentals of the Delta Lake storage format
Built-in performance-boosting features of Delta Lake
Configurable performance-boosting features of Delta Lake
Summary
11
Chapter 9: The Photon Engine
Chapter 9: The Photon Engine
Technical requirements
Understanding Photon Engine
Understanding vectorization
Discussing the Photon product roadmap
Summary
Further reading
12
Chapter 10: Warehouse on the Lakehouse
Chapter 10: Warehouse on the Lakehouse
Technical requirements
Organizing data on the Lakehouse
Implementing data modeling techniques
Summary
13
Part 3: Databricks SQL Commands
Part 3: Databricks SQL Commands
14
Chapter 11: SQL Commands – Part 1
Chapter 11: SQL Commands – Part 1
Technical requirements
Working with data definition language commands
Working with data manipulation language commands
Working with the inbuilt functions in Databricks SQL
Summary
15
Chapter 12: SQL Commands – Part 2
Chapter 12: SQL Commands – Part 2
Technical requirements
Working with Delta Lake maintenance commands
Working with data security commands
Working with metadata commands
Summary
16
Part 4: TPC-DS, Experiments, and Frequently Asked Questions
Part 4: TPC-DS, Experiments, and Frequently Asked Questions
17
Chapter 13: Playing with the TPC-DS Dataset
Chapter 13: Playing with the TPC-DS Dataset
Technical requirements
Understanding the TPC-DS dataset
Generating TPC-DS data
Running automated benchmarks
Experimenting with TPC-DS in Databricks SQL
Summary
18
Chapter 14: Ask Me Anything
Chapter 14: Ask Me Anything
Frequently asked questions
Summary
19
Index
Index
Why subscribe?
20
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts

Introduction to Databricks

Databricks is one of the most recognizable names in the big data industry. They are the providers of the lakehouse platform for data analytics and artificial intelligence (AI). This book is about Databricks SQL, a product within the Databricks Lakehouse platform that powers data analytics and business intelligence.

Databricks SQL is a rapidly evolving product. It is not a traditional data warehouse, yet its users are the traditional data warehouse and business intelligence users. It claims to provide all the functionality of data warehouses on what is essentially a data lake. This concept can be a bit jarring. It can create resistance in adoption as you might be wondering if your skills are transferrable, or if your work might be disrupted as a result of a new learning curve.

Hence, I am writing this book.

The primary intent of this book is to help you learn the fundamental concepts of Databricks SQL in a fun, follow-along interactive manner. My aim is that by the time you complete this book, you will be confident in your adoption of Databricks SQL as the enabler of your business intelligence.

This book does not intend to be a definitive guide or a complete reference, nor does it intend to be a replacement for the official documentation. It is too early for either of those. This book is your initiation into business intelligence on the data lakehouse, the Databricks SQL way.

Let’s begin!

In this chapter, we’ll cover the following topics:

  • An overview of Databricks, the company
  • An overview of the Lakehouse architecture
  • An overview of the Databricks Lakehouse platform
Previous Section
End of Section 1
Next Section