We can use one of the tools (web console, CLI, SDK, or API) to get EMR cluster details in AWS. The web console displays all of the clusters you've launched in the past two weeks (both active and terminated).
We have seen in the previous chapter that if you click on a cluster name, then the web console displays a Details pane with information about that cluster. As we will see in our next chapter, we can also find the details about a cluster from the CLI using the --describe
argument along with a Job Flow ID.
Amazon EMR and Hadoop both generate logfiles as the cluster begins execution. You can access these logfiles from several different tools, depending on the configuration specified when we launch the cluster.
Every cluster publishes log files to the /mnt/var/log/
directory on the master node. These logfiles are only available while the cluster is running.
When you launch the cluster with an Amazon S3 log path, the cluster copies...