Once the application is functionally complete and passes the tests in embedded mode, it is time to take it for a test drive on the cluster. Compared to working within the IDE, execution in distributed mode requires a different approach and tools for deployment, testing and troubleshooting of the application. In this section, we will introduce YARN as the execution layer and how to setup and navigate the cluster for various tasks. Note that, as of release 3.6, Apex supports YARN as cluster manager, support for other infrastructure is likely to follow in one of the next releases.
YARN (Yet Another Resource Negotiator) originates from an effort to separate processing resource management from the application framework MapReduce, which was tightly coupled in the first version of Hadoop. Today, many of the big data processing frameworks, including Apache Spark, support YARN.