Book Image

Apache Spark Graph Processing

Book Image

Apache Spark Graph Processing

Overview of this book

Table of Contents (16 chapters)
Apache Spark Graph Processing
Credits
Foreword
About the Author
About the Reviewer
www.PacktPub.com
Preface
Index

The aggregateMessages operator


Once we have our graph ready, let's start our mission, which is aggregating the stats data in scoreGraph. In GraphX, aggregateMessages is the operator for that kind of job.

For example, let's find out the average field goals made per game by the winning teams. In other words, the games that the teams lost will not be counted. To get the average for each team, we first need to have the number of games won by the team and the total field goals that the team made in those games:

// Aggregate the total field goals made by winning teams
type FGMsg = (Int, Int)
val winningFieldGoalMade: VertexRDD[FGMsg] = scoreGraph aggregateMessages(
    // sendMsg
    triplet => triplet.sendToSrc(1, triplet.attr.winnerStats.fieldGoalMade)
    // mergeMsg
    ,(x, y) => (x._1 + y._1, x._2+ y._2)
)
// Aggregate the total field goals made by winning teams
type Msg = (Int, Int)
type Context = EdgeContext[String, FullResult, Msg] 
val winningFieldGoalMade: VertexRDD[Msg] = scoreGraph...