We are now ready to code up the remaining pieces of our network crawler. The largest missing piece is the fetcher manager. Let's start with the companion object. As with the worker actors, this just contains the definitions of the messages that the actor can receive and a factory to create the Props instance:
// FetcherManager.scala
import scala.collection.mutable
import akka.actor._
object FetcherManager {
case class AddToQueue(login:String)
case object GiveMeWork
def props(token:Option[String], nFetchers:Int) =
Props(classOf[FetcherManager], token, nFetchers)
}The manager can receive two messages: AddToQueue, which tells it to add a username to the queue of users whose followers need to be fetched, and GiveMeWork, emitted by the fetchers when they are unemployed.
The manager will be responsible for launching the fetchers, response interpreter, and follower extractor, as well as maintaining an internal queue of usernames and a set of usernames that we...