Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Apache Spark Graph Processing
  • Table Of Contents Toc
Apache Spark Graph Processing

Apache Spark Graph Processing

By : Rindra Ramamonjison
3.5 (2)
close
close
Apache Spark Graph Processing

Apache Spark Graph Processing

3.5 (2)
By: Rindra Ramamonjison

Overview of this book

Apache Spark is the next standard of open-source cluster-computing engine for processing big data. Many practical computing problems concern large graphs, like the Web graph and various social networks. The scale of these graphs - in some cases billions of vertices, trillions of edges - poses challenges to their efficient processing. Apache Spark GraphX API combines the advantages of both data-parallel and graph-parallel systems by efficiently expressing graph computation within the Spark data-parallel framework. This book will teach the user to do graphical programming in Apache Spark, apart from an explanation of the entire process of graphical data analysis. You will journey through the creation of graphs, its uses, its exploration and analysis and finally will also cover the conversion of graph elements into graph structures. This book begins with an introduction of the Spark system, its libraries and the Scala Build Tool. Using a hands-on approach, this book will quickly teach you how to install and leverage Spark interactively on the command line and in a standalone Scala program. Then, it presents all the methods for building Spark graphs using illustrative network datasets. Next, it will walk you through the process of exploring, visualizing and analyzing different network characteristics. This book will also teach you how to transform raw datasets into a usable form. In addition, you will learn powerful operations that can be used to transform graph elements and graph structures. Furthermore, this book also teaches how to create custom graph operations that are tailored for specific needs with efficiency in mind. The later chapters of this book cover more advanced topics such as clustering graphs, implementing graph-parallel iterative algorithms and learning methods from graph data.
Table of Contents (10 chapters)
close
close
9
Index

Modifying graph structures

The GraphX library also comes with four useful methods for changing the structure of graphs. Their method signatures are listed as follows:

class Graph[VD, ED] {
  def reverse: Graph[VD, ED]
  
  def subgraph(epred: EdgeTriplet[VD,ED] => Boolean,
               vpred: (VertexId, VD) => Boolean): Graph[VD, ED]
               
  def mask[VD2, ED2](other: Graph[VD2, ED2]): Graph[VD, ED]
  
  def groupEdges(merge: (ED, ED) => ED): Graph[VD,ED]
}

The reverse operator

As its name suggests, the reverse operator returns a new graph with all the edge directions reversed. It does not modify any vertex or edge properties, or change the number of edges. Moreover, its implementation does not produce any data movement or duplication.

The subgraph operator

Next on the list is subgraph, which is useful for filtering graphs. It takes two predicate functions as arguments that return Boolean values. The first predicate epred takes an EdgeTriplet and returns true when the triplet...

CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Apache Spark Graph Processing
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon