Book Image

Spark Cookbook

By : Rishi Yadav
Book Image

Spark Cookbook

By: Rishi Yadav

Overview of this book

Table of Contents (19 chapters)
Spark Cookbook
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Index

Developing Spark applications in Eclipse with SBT


Simple Build Tool (SBT) is a build tool made especially for Scala-based development. SBT follows Maven-based naming conventions and declarative dependency management.

SBT provides the following enhancements over Maven:

  • Dependencies are in the form of key-value pairs in the build.sbt file as opposed to pom.xml in Maven

  • It provides a shell that makes it very handy to perform build operations

  • For simple projects without dependencies, you do not even need the build.sbt file

In build.sbt, the first line is the project definition:

lazy val root = (project in file("."))

Each project has an immutable map of key-value pairs. This map is changed by settings in SBT like so:

lazy val root = (project in file("."))
  settings(
    name := "wordcount"
  )

Every change in the settings leads to a new map, as it's an immutable map.

How to do it...

Here's how we go about adding the sbteclipse plugin:

  1. Add this to the global plugin file:

    $ mkdir /home/hduser/.sbt/0.13/plugins...