Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Big Data on Kubernetes
  • Table Of Contents Toc
Big Data on Kubernetes

Big Data on Kubernetes

By : Neylson Crepalde
close
close
Big Data on Kubernetes

Big Data on Kubernetes

By: Neylson Crepalde

Overview of this book

In today's data-driven world, organizations across different sectors need scalable and efficient solutions for processing large volumes of data. Kubernetes offers an open-source and cost-effective platform for deploying and managing big data tools and workloads, ensuring optimal resource utilization and minimizing operational overhead. If you want to master the art of building and deploying big data solutions using Kubernetes, then this book is for you. Written by an experienced data specialist, Big Data on Kubernetes takes you through the entire process of developing scalable and resilient data pipelines, with a focus on practical implementation. Starting with the basics, you’ll progress toward learning how to install Docker and run your first containerized applications. You’ll then explore Kubernetes architecture and understand its core components. This knowledge will pave the way for exploring a variety of essential tools for big data processing such as Apache Spark and Apache Airflow. You’ll also learn how to install and configure these tools on Kubernetes clusters. Throughout the book, you’ll gain hands-on experience building a complete big data stack on Kubernetes. By the end of this Kubernetes book, you’ll be equipped with the skills and knowledge you need to tackle real-world big data challenges with confidence.
Table of Contents (18 chapters)
close
close
Lock Free Chapter
1
Part 1:Docker and Kubernetes
5
Part 2: Big Data Stack
10
Part 3: Connecting It All Together

Getting started with SQL query engines

In the world of big data, the way we store and analyze data has undergone a significant transformation. Traditional data warehouses, which were once the go-to solution for data analysis, have given way to more modern and scalable approaches, such as SQL query engines. These engines, such as Trino (formerly known as Presto), Dremio, and Apache Spark SQL, offer a high-performance, cost-effective, and flexible alternative to traditional data warehouses.

Next, we are going to see the main differences between data warehouses and SQL query engines.

The limitations of traditional data warehouses

Traditional data warehouses were designed to store and analyze structured data from relational databases. However, with the advent of big data and the proliferation of diverse data sources, such as log files, sensor data, and social media data, the limitations of data warehouses became apparent. These limitations include the following:

  • Scalability...
CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Big Data on Kubernetes
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon