To read configuration data from a property file, we can use the configuration library for JVM languages from typesafe. This is mostly a Scala feature, but an interesting example is running Scalding applications in two different clusters.

While during developing and testing, we execute our Scalding application in a development cluster, when the jobs are production ready, they are executed in a powerful production Hadoop cluster.

In this case, each cluster will have different configuration requirements. To solve this issue, first define two property files as shown:

  dev-cluster.properties
  production-cluster.properties

Each file contains environment and job configuration information. For example, the file dev-cluster.properties can contain the following:

  mysql=dev.mysql.company.com
  zookeeper=dev.zookeeper.company.com:2181
  reducers=30

The best way to implement this capability (keeping in mind the single responsibility principle) is in a different class...