Book Image

Apache Spark Graph Processing

Book Image

Apache Spark Graph Processing

Overview of this book

Table of Contents (16 chapters)
Apache Spark Graph Processing
Credits
Foreword
About the Author
About the Reviewer
www.PacktPub.com
Preface
Index

Joining average stats into a graph


The previous example shows us how flexible the aggregateMessages operator is. We can define the type Msg of the messages to be aggregated to fit our needs. Moreover, we can select which nodes receive the messages. Finally, we can also define how we want to merge the messages.

As a final example, let's aggregate many statistics about each team and join this information into the nodes of the graph:

  1. To start, we create its own class for the team stats:

    // Average Stats of All Teams 
    case class TeamStat(
            wins: Int  = 0      // Number of wins
         ,losses: Int  = 0      // Number of losses
            ,ppg: Int  = 0      // Points per game
            ,pcg: Int  = 0      // Points conceded per game
            ,fgp: Double  = 0   // Field goal percentage
            ,tpp: Double  = 0   // Three point percentage
            ,ftp: Double  = 0   // Free Throw percentage
         ){
        override def toString = wins + "-" + losses
    }
  2. We collect the average stats for all teams using aggregateMessages...