Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Big Data on Kubernetes
  • Table Of Contents Toc
Big Data on Kubernetes

Big Data on Kubernetes

By : Neylson Crepalde
close
close
Big Data on Kubernetes

Big Data on Kubernetes

By: Neylson Crepalde

Overview of this book

In today's data-driven world, organizations across different sectors need scalable and efficient solutions for processing large volumes of data. Kubernetes offers an open-source and cost-effective platform for deploying and managing big data tools and workloads, ensuring optimal resource utilization and minimizing operational overhead. If you want to master the art of building and deploying big data solutions using Kubernetes, then this book is for you. Written by an experienced data specialist, Big Data on Kubernetes takes you through the entire process of developing scalable and resilient data pipelines, with a focus on practical implementation. Starting with the basics, you’ll progress toward learning how to install Docker and run your first containerized applications. You’ll then explore Kubernetes architecture and understand its core components. This knowledge will pave the way for exploring a variety of essential tools for big data processing such as Apache Spark and Apache Airflow. You’ll also learn how to install and configure these tools on Kubernetes clusters. Throughout the book, you’ll gain hands-on experience building a complete big data stack on Kubernetes. By the end of this Kubernetes book, you’ll be equipped with the skills and knowledge you need to tackle real-world big data challenges with confidence.
Table of Contents (18 chapters)
close
close
Lock Free Chapter
1
Part 1:Docker and Kubernetes
5
Part 2: Big Data Stack
10
Part 3: Connecting It All Together

Big Data Processing with Apache Spark

As seen in the preceding chapter, Apache Spark has rapidly become one of the most widely used distributed data processing engines for big data workloads. In this chapter, we will cover the fundamentals of using Spark for large-scale data processing.

We’ll start by discussing how to set up a local Spark environment for development and testing. You’ll learn how to launch an interactive PySpark shell and use Spark’s built-in DataFrames API to explore and process sample datasets. Through coding examples, you’ll gain practical experience with essential PySpark data transformations such as filtering, aggregations, and joins.

Next, we’ll explore Spark SQL, which allows you to query structured data in Spark via SQL. You’ll learn how Spark SQL integrates with other Spark components and how to use it to analyze DataFrames. We’ll also cover best practices for optimizing Spark workloads. While we won&...

CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Big Data on Kubernetes
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon