Book Image

Apache Spark Graph Processing

Book Image

Apache Spark Graph Processing

Overview of this book

Table of Contents (16 chapters)
Apache Spark Graph Processing
Credits
Foreword
About the Author
About the Reviewer
www.PacktPub.com
Preface
Index

Transforming the vertex and edge attributes


The map operator is a core method for transforming distributed datasets or RDDs in Spark. Similarly, property graphs also have three map operators defined as follows:

class Graph[VD, ED] {
  def mapVertices[VD2](mapFun: (VertexId, VD) => VD2): Graph[VD2, ED]
  def mapEdges[ED2](mapFun: Edge[ED] => ED2): Graph[VD, ED2]
  def mapTriplets[ED2](mapFun: EdgeTriplet[VD, ED] => ED2): Graph[VD, ED2]
}

Each of these methods is called on a property graph with vertex attribute type VD and edge attribute type ED. Each of them also takes a user-defined mapping function mapFun that performs one of the following:

  • For mapVertices, mapFun takes a pair of (VertexId, VD) as input and returns a transformed vertex attribute of type VD2.

  • For mapEdges, mapFun takes an Edge object as input and returns a transformed edge attribute of type ED2.

  • For mapTriplets, mapFun takes an EdgeTriplet object as input and returns a transformed edge attribute of type ED2.

Note

In each...