In this chapter, we discussed some advanced topics of Spark toward making your Spark job's performance better. We discussed some basic techniques to tune your Spark jobs. We discussed how to monitor your jobs by accessing Spark web UI. We discussed how to set Spark configuration parameters. We also discussed some common mistakes made by Spark users and provided some recommendations. Finally, we discussed some optimization techniques that help tune Spark applications.
Apache Spark 2: Data Processing and Real-Time Analytics
By :
Apache Spark 2: Data Processing and Real-Time Analytics
By:
Overview of this book
Apache Spark is an in-memory, cluster-based data processing system that provides a wide range of functionalities such as big data processing, analytics, machine learning, and more. With this Learning Path, you can take your knowledge of Apache Spark to the next level by learning how to expand Spark's functionality and building your own data flow and machine learning programs on this platform.
You will work with the different modules in Apache Spark, such as interactive querying with Spark SQL, using DataFrames and datasets, implementing streaming analytics with Spark Streaming, and applying machine learning and deep learning techniques on Spark using MLlib and various external tools.
By the end of this elaborately designed Learning Path, you will have all the knowledge you need to master Apache Spark, and build your own big data processing and analytics pipeline quickly and without any hassle.
This Learning Path includes content from the following Packt products:
• Mastering Apache Spark 2.x by Romeo Kienzler
• Scala and Spark for Big Data Analytics by Md. Rezaul Karim, Sridhar Alla
• Apache Spark 2.x Machine Learning Cookbook by Siamak Amirghodsi, Meenakshi Rajendran, Broderick Hall, Shuen MeiCookbook
Table of Contents (23 chapters)
Title Page
Copyright
About Packt
Contributors
Preface
Free Chapter
A First Taste and What's New in Apache Spark V2
Apache Spark Streaming
Structured Streaming
Apache Spark MLlib
Apache SparkML
Apache SystemML
Apache Spark GraphX
Spark Tuning
Testing and Debugging Spark
Practical Machine Learning with Spark Using Scala
Spark's Three Data Musketeers for Machine Learning - Perfect Together
Common Recipes for Implementing a Robust Machine Learning System
Recommendation Engine that Scales with Spark
Unsupervised Clustering with Apache Spark 2.0
Implementing Text Analytics with Spark 2.0 ML Library
Spark Streaming and Machine Learning Library
Other Books You May Enjoy
Index
Customer Reviews