Apache Spark is a fast and scalable general data processing framework (http://spark.apache.org). Spark provides a very concise syntax for writing a wide range of data processing applications. Spark became a top-level Apache project in early 2014.
The Spark project was started at Berkeley as part of the Berkeley Data Analytics Stack (https://amplab.cs.berkeley.edu/software), the same project that Mesos comes from. Spark was the first data processing framework built on Mesos and effectively leverages Mesos for resource management. Spark is one of the fastest growing data analysis frameworks and aims to unify all kinds of data analysis under a single unified API. Spark provides a unified API for doing batch, streaming, and iterative data processing.
Spark makes an aggressive use of memory to accelerate computations. Spark's Directed Acyclic Graph (DAG) execution engine is suitable for a wide range of applications, including the interactive and iterative algorithms that often...