Index
A
- ACLs
- working with / Working with ACLs
- type / Type of ACLs
- administration ACL / The administration ACL
- service-level ACL / The service-level ACL
- queue ACL / The queue ACL
- application ACL / The application ACL
- ACLs, configuring through hadoop-policy.xml file
- security.resourcetracker.protocol.acl / The service-level ACL
- security.resourcemanager-administration.protocol.acl / The service-level ACL
- security.applicationclient.protocol.acl / The service-level ACL
- security.applicationmaster.protocol.acl / The service-level ACL
- security.containermanagement.protocol.acl / The service-level ACL
- security.resourcelocalizer.protocol.acl / The service-level ACL
- ACL value
- defining / Defining an ACL value
- admin commands, ResourceManager High Availability
- yarn rmadmin -transitionToActive rm1 / High Availability admin commands
- yarn rmadmin -transitionToStandby rm1 / High Availability admin commands
- yarn rmadmin -getServiceState rm1 / High Availability admin commands
- administration ACL
- about / The administration ACL
- administration commands, Hadoop-YARN
- about / Administration commands
- ResourceManager / ResourceManager / NodeManager / ProxyServer
- NodeManager / ResourceManager / NodeManager / ProxyServer
- ProxyServer / ResourceManager / NodeManager / ProxyServer
- rmadmin / RMAdmin, Command options
- daemonlog / DaemonLog
- Ambari
- Apache Hadoop
- hardware requisites / Hardware requirements
- software requisites / Software requirements
- Apache Hadoop 2.x
- components / Starting with the basics
- master processes / Starting with the basics
- Apache Knox
- about / Apache Knox
- URL / Apache Knox
- Apache Ranger
- about / Apache Ranger
- URL / Apache Ranger
- Apache YARN documentation
- application
- writing / Writing your own application
- Java project, creating / Step 1 – Create a new project and add Hadoop-YARN JAR files
- HadoopYARN jars, creating / Step 1 – Create a new project and add Hadoop-YARN JAR files
- ApplicationMaster, defining / Step 2 – Define the ApplicationMaster and client classes, Define an ApplicationMaster
- client classes, creating / Step 2 – Define the ApplicationMaster and client classes, Define a YARN client
- project, exporting / Step 3 – Export the project and copy resources
- resources, copying / Step 3 – Export the project and copy resources
- running, bin/yarn command used / Step 4 – Run the application using bin or the YARN command
- application ACL
- about / The application ACL
- application attempts
- URI / Applications
- example / Applications
- application attempt view, ResourceManager service / The ResourceManager's view, View 3 – An application attempt
- ApplicationClientProtocol interface / ApplicationClientProtocol
- application command
- about / Application
- command options / Command options, Sample output
- sample output / Command options, Sample output
- application execution flow
- about / Understanding application execution flow
- application initialization and submission phase / Phase 1 – Application initialization and submission
- memory, allocating / Phase 2 – Allocate memory and start ApplicationMaster
- ApplicationMaster, starting / Phase 2 – Allocate memory and start ApplicationMaster
- ApplicationMaster, registering / Phase 3 – ApplicationMaster registration and resource allocation
- resource allocation / Phase 3 – ApplicationMaster registration and resource allocation
- containers, launching / Phase 4 – Launch and monitor containers
- containers, monitoring / Phase 4 – Launch and monitor containers
- application progress report / Phase 5 – Application progress report
- application completion phase / Phase 6 – Application completion
- ApplicationHistoryProtocol
- about / ApplicationHistoryProtocol
- URL / ApplicationHistoryProtocol
- application logs
- about / Application logs
- ApplicationMaster
- ApplicationMaster metadata
- URI / ApplicationMaster summary
- example / ApplicationMaster summary
- ApplicationMasterProtocol interface / ApplicationMasterProtocol
- ApplicationMaster summary, MapReduce ApplicationMaster REST APIs
- ApplicationMaster metadata / ApplicationMaster summary
- applications, NodeManager REST APIs
- about / Applications
- applications list / Applications
- single application information / Applications
- applications, ResourceManager REST APIs
- applications list / Applications
- query parameter list / Applications
- applications statistics / Applications
- single application information / Applications
- application attempts / Applications
- applications list
- URI / Applications, Applications
- example / Applications, Applications
- query parameter list / Applications, Applications
- about / Applications
- ApplicationsManager (AsM) / ResourceManager
- applications statistics
- URI / Applications
- example / Applications
- query parameter list / Applications
- ApplicationSubmissionContext class
- about / ApplicationSubmissionContext
- URL / ApplicationSubmissionContext
- application view, NodeManager service / The NodeManager's view, View 1 – Application
- application view, ResourceManager service / The ResourceManager's view, View 2 – Application
- architecture, ResourceManager High Availability / Architecture
- authorization checks
- application submission / The queue ACL
- administer queue / The queue ACL
B
- bin/yarn command
- used, for running application / Step 4 – Run the application using bin or the YARN command
C
- CapacityScheduler
- about / CapacityScheduler
- features / CapacityScheduler
- configuring / Configuring CapacityScheduler
- capacity scheduler / An introduction to scheduling in YARN
- CapacityScheduler Queue (CSQueue)
- about / CapacityScheduler Queue (CSQueue)
- properties / CapacityScheduler Queue (CSQueue)
- classes, YARN client API
- YarnClient / YARN client API
- AMRMClient / AMRMClientAsync / YARN client API
- NMClient / NMClientAsync / YARN client API
- AHSClient / TimelineClient / YARN client API
- classpath command
- about / Classpath
- client and admin interfaces, ResourceManager
- about / The client and admin interfaces
- ClientRMService class / The client and admin interfaces
- AdminService class / The client and admin interfaces
- ClientRMService
- application requests / The client and admin interfaces
- cluster metrics / The client and admin interfaces
- security / The client and admin interfaces
- URL / The client and admin interfaces
- Cloudera
- URL / Configuration changes
- Cloudera installation
- Cloudera Manager
- cluster metadata
- URI / The cluster summary
- example / The cluster summary
- cluster metrics
- URI / The cluster summary
- example / The cluster summary
- cluster metrics, ResourceManager service
- cluster summary, ResourceManager REST APIs
- cluster metadata / The cluster summary
- cluster metrics / The cluster summary
- communication protocols
- about / Communication protocols
- ApplicationClientProtocol interface / ApplicationClientProtocol
- ApplicationMasterProtocol interface / ApplicationMasterProtocol
- ContainerManagementProtocol interface / ContainerManagementProtocol
- ApplicationHistoryProtocol interface / ApplicationHistoryProtocol
- components, MapReduce system
- JobTracker / Introduction to MapReduce v1
- TaskTracker / Introduction to MapReduce v1
- components, ResourceManager
- Scheduler / ResourceManager
- ApplicationsManager (AsM) / ResourceManager
- components, YARN
- about / An overview of YARN components
- ResourceManager / ResourceManager
- NodeManager / NodeManager
- ApplicationMaster / ApplicationMaster
- container / Container
- configuration, ResourceManager High Availability
- about / Configuring ResourceManager's High Availability
- nodes, defining / Define nodes
- RM state store mechanism / The RM state store mechanism
- failover proxy provider / The failover proxy provider
- automatic failover / Automatic failover
- configuration changes, migration steps from MRv1 to MRv2
- MapReduce framework / Configuration changes
- ResourceManager host / Configuration changes
- shuffle service / Configuration changes
- scheduler configuration / Configuration changes
- resource capability / Configuration changes
- container
- about / Container
- container failure
- about / The container failure
- ContainerLaunchContext class
- about / ContainerLaunchContext
- URL / ContainerLaunchContext
- container management, NodeManager
- about / Container management
- ContainerExecutor / Container management
- ResourceLocalizationService / Container management
- ContainersLauncher / Container management
- ContainersMonitor / Container management
- auxiliary service / Container management
- LogHandler / Container management
- log aggregation / Container management
- ContainerManagementProtocol interface
- about / ContainerManagementProtocol
- containers, NodeManager REST APIs
- about / Containers
- containers list / Containers
- single container information / Containers
- containers list
- about / Containers
- URI / Containers
- example / Containers
- container view, NodeManager service / The NodeManager's view, View 2 – Container
- container view, ResourceManager service / The ResourceManager's view, View 4 – Container
- core-site.xml file
- about / The core-site.xml file
- core interfaces, ResourceManager
- about / The core interfaces
- YarnScheduler class / The core interfaces
- RMAppManager / The core interfaces
- RMStateStore / The core interfaces
- SchedulingMonitor / The core interfaces
- curl command
- about / Curl command
- URL / Curl command
- used, for calling REST API / Curl command
- syntax / Curl command
- options / Curl command
D
- daemonlog command
- about / DaemonLog
- command options / Command options
- DataNode
- about / Starting with the basics
- data processing, Spark
- deploy modes, Spark-on-YARN
- yarn-client / Running Spark on YARN
- yarn-cluster / Running Spark on YARN
- directory structure, resource localization
E
- event
- event handle
F
- failover mechanisms, ResourceManager High Availability
- manual failover / Failover mechanisms
- automatic failover / Failover mechanisms
- failures, YARN
- handling / Handling failures in YARN
- container failure / The container failure
- NodeManager failure / The NodeManager failure
- ResourceManager failure / The ResourceManager failure
- fair scheduler / An introduction to scheduling in YARN
- about / Fair scheduler
- hierarchical queues / Hierarchical queues
- schedulable / Schedulable
- scheduling policy / Scheduling policy
- URL / Scheduling policy
- configuring / Configuring a fair scheduler
- FairScheduler Queue (FSQueue)
- about / FairScheduler Queue (FSQueue)
- elements / FairScheduler Queue (FSQueue)
- Firefox
- RESTClient, using in / RESTClient plugins
- First In First Out (FIFO) / Configuration changes
G
- Ganglia
- about / Ganglia monitoring
- URL, for documentation / Ganglia daemons
- integrating, with Hadoop / Integrating Ganglia with Hadoop
- Ganglia daemons
- about / Ganglia daemons
- monitoring daemon (gmond) / Ganglia daemons
- metadata daemon (gmetad) / Ganglia daemons
- web application / Ganglia daemons
- Ganglia monitoring, YARN services
- about / Ganglia monitoring
- Ganglia daemons / Ganglia daemons
- Ganglia, integrating with Hadoop / Integrating Ganglia with Hadoop
- Giraph
- about / Projects powered by YARN
- URL / Projects powered by YARN, The integration of HAMA and Giraph with YARN
- integrating, with YARN / The integration of HAMA and Giraph with YARN
- Giraph jobs, Hadoop-YARN cluster
- URL, for documentation / The integration of HAMA and Giraph with YARN
H
- Hadoop
- URL / Projects powered by YARN
- Ganglia, integrating with / Integrating Ganglia with Hadoop
- hadoop-env.sh file
- Hadoop-YARN
- capacity scheduler / An introduction to scheduling in YARN
- fair scheduler / An introduction to scheduling in YARN
- Hadoop-YARN API
- about / An introduction to the YARN API
- Hadoop-YARN cluster
- supported platforms / Supported platforms
- Hadoop-YARN commands
- using / Using the Hadoop-YARN commands
- user commands / The user commands
- administration commands / Administration commands
- Hadoop-YARN multi-node installation
- about / The Hadoop-YARN multi-node installation
- prerequisites / Prerequisites
- steps / Installation steps
- Hadoop-YARN services
- configuring / Configuring the Hadoop-YARN services
- ResourceManager service / The ResourceManager service
- NodeManager service / The NodeManager service
- Timeline server / The Timeline server
- web application proxy server / The web application proxy server
- ports summary / Ports summary
- managing / Managing the Hadoop-YARN services
- service logs, managing / Managing service logs
- pid files, managing / Managing pid files
- Hadoop-YARN single node installation
- about / The Hadoop-YARN single node installation
- prerequisites / Prerequisites
- steps / Installation steps
- Hadoop configuration files
- core-site.xml file / The core-site.xml file
- hdfs-site.xml file / The hdfs-site.xml file
- mapred-site.xml file / The mapred-site.xml file
- yarn-site.xml file / The yarn-site.xml file
- hadoop-env.sh file / The hadoop-env.sh and yarn-env.sh files
- yarn-env.sh file / The hadoop-env.sh and yarn-env.sh files
- slaves file / The slaves file
- HAMA
- URL / Projects powered by YARN
- about / The integration of HAMA and Giraph with YARN
- integrating, with YARN / The integration of HAMA and Giraph with YARN
- components / The integration of HAMA and Giraph with YARN
- HBase
- URL / Projects powered by YARN
- HDFS
- Storm ZIP, copying to / Copy Storm ZIP to HDFS
- hdfs-site.xml file
- about / The hdfs-site.xml file
- health checker script, NodeManager / The health checker script
- high-level architecture, MapReduce framework
- MapReduce API / Introduction to MapReduce v1
- MapReduce framework / Introduction to MapReduce v1
- MapReduce system / Introduction to MapReduce v1
- high-level changes, from MRv1 to MRv2
- about / High-level changes from MRv1 to MRv2
- evolution, of MRApplicationMaster service / The evolution of the MRApplicationMaster service
- resource capability / Resource capability
- pluggable shuffle / Pluggable shuffle
- hierarchical queues / Hierarchical queues and fair scheduler
- fair scheduler / Hierarchical queues and fair scheduler
- task execution, as containers / Task execution as containers
- Hortonworks
- URL, for selecting nodes / Hardware requirements
- Hortonworks installation
I
- implementation of SecretManager, YARN
- BaseNMTokenSecretManager / The service-level ACL
- AMRMTokenSecretManager / The service-level ACL
- BaseClientToAMTokenSecretManager / The service-level ACL
- BaseContainerTokenSecretManager / The service-level ACL
- ISO image, Ubuntu Desktop
- URL, for downloading / Supported platforms
J
- jar command
- about / Jar
- Java API
- about / Java API
- JAVA API, for ContainerLaunchContext
- reference link / The application ACL
- Java compatibility, with Hadoop
- URL / Install Java
- Java Cryptography Extension (JCE) / The service-level ACL
- Java installation, on Ubuntu
- URL, for blog / Install Java
- JMX monitoring, YARN services
- about / JMX monitoring
- ResourceManager JMX beans / The ResourceManager JMX beans
- NodeManager JMX beans / The NodeManager JMX beans
- job attempts
- job configuration
- job counters
- Jobs, MapReduce ApplicationMaster REST APIs
- jobs list
- JobTracker
- functionalities / Introduction to MapReduce v1
- URL / Shortcomings of MapReducev1
- about / Introducing MRv1 and MRv2
- jps command
K
- Kerberos
- Kerberos entities
- realm / Enabling authentication using Kerberos
- principal / Enabling authentication using Kerberos
- keytab / Enabling authentication using Kerberos
- key distribution center (KDC)
L
- Linux commands / utilities
- about / Basic Linux commands / utilities
- sudo command / Sudo
- nano editor / Nano editor
- source command / Source
- jps command / Jps
- netstat command / Netstat
- man command / Man
- localized resource view, NodeManager service / The NodeManager's view, View 3 – A localized resource
- logs command
- about / Logs
- command options / Command options
- log types, container
- stderr / Application logs
- syslog / Application logs
- stdout / Application logs
M
- man command
- man pages
- about / Man
- mapred-default.xml file
- reference link / The application ACL
- mapred-site.xml file
- about / The mapred-site.xml file
- MapReduce
- about / Introduction to MapReduce v1
- URL / Projects powered by YARN
- MapReduce ApplicationMaster REST APIs
- about / MapReduce ApplicationMaster REST APIs
- ApplicationMaster summary / ApplicationMaster summary
- Jobs / Jobs
- Tasks / Tasks
- MapReduce framework
- high-level architecture / Introduction to MapReduce v1
- MapReduce HistoryServer REST APIs
- MapReduce system
- components / Introduction to MapReduce v1
- MapReduce v1
- about / Introduction to MapReduce v1
- shortcomings / Shortcomings of MapReducev1
- master processes, Apache Hadoop 2.x
- NameNode / Starting with the basics
- ResourceManager / Starting with the basics
- memory configuration settings, HDP
- reference link / Configuration changes
- methods, secure Hadoop-YARN cluster
- dedicated user group, using for Hadoop-YARN daemons / Using a dedicated user group for Hadoop-YARN daemons
- permissions, validating to YARN directories / Validating permissions to YARN directories
- HTTPS protocol, enabling / Enabling the HTTPS protocol
- authorization, enabling with Access Control Lists / Enabling authorization using Access Control Lists
- authentication, enabling with Kerberos / Enabling authentication using Kerberos
- migration steps, from MRv1 to MRv2
- about / The migration steps from MRv1 to MRv2
- configuration changes / Configuration changes
- binary / source compatibility / The binary / source compatibility
- MRApplicationMaster service
- functionalities / The evolution of the MRApplicationMaster service
- MRAppMaster class
- reference link / The evolution of the MRApplicationMaster service
- MRv1
- about / Introducing MRv1 and MRv2
- MRv1 apps
- monitoring / Running and monitoring MRv1 apps on YARN
- running, on YARN / Running and monitoring MRv1 apps on YARN
N
- NameNode
- about / Starting with the basics
- nano editor
- about / Nano editor
- URL / Nano editor
- netstat command
- node, preparing for Hadoop-YARN cluster
- about / Preparing a node for a Hadoop-YARN cluster
- Java, installing / Install Java
- Hadoop dedicated user and group, creating / Create a Hadoop dedicated user and group
- firewall, disabling / Disable firewall or open Hadoop ports
- open Hadoop ports, disabling / Disable firewall or open Hadoop ports
- domain name resolution, configuring / Configure the domain name resolution
- SSH, installing / Install SSH and configure passwordless SSH from the master to all slaves
- passwordless SSH, configuring from master to all slaves / Install SSH and configure passwordless SSH from the master to all slaves
- node command
- about / Node
- command options / Command options, Sample output
- sample output / Command options, Sample output
- NodeManager
- about / NodeManager, Starting with the basics, ResourceManager / NodeManager / ProxyServer, Understanding NodeManager
- health, monitoring / Monitoring NodeManager's health
- health checker script / The health checker script
- status updates / Status updates
- state and health management / State and health management
- container management / Container management
- security and token managers / The security and token managers
- NodeManager failure
- about / The NodeManager failure
- NodeManager interfaces
- about / The NodeManager interfaces
- NMLivelinessMonitor / The NodeManager interfaces
- ApplicationMasterLauncher / The NodeManager interfaces
- NodeManager JMX beans
- about / The NodeManager JMX beans
- JvmMetrics / The NodeManager JMX beans
- ShuffleMetrics / The NodeManager JMX beans
- RpcActivity / The NodeManager JMX beans
- NodeManagerMetrics / The NodeManager JMX beans
- UgiMetrics / The NodeManager JMX beans
- NodeManager REST APIs
- about / NodeManager REST APIs
- node summary / The node summary
- applications / Applications
- containers / Containers
- URL / Containers, Tasks
- NodeManager service
- about / The NodeManager service, An introduction to state management analogy, The NodeManager's view
- application view / The NodeManager's view, View 1 – Application
- container view / The NodeManager's view, View 2 – Container
- localized resource view / The NodeManager's view, View 3 – A localized resource
- node metadata
- URI / The node summary
- example / The node summary
- nodes, ResourceManager REST APIs
- nodes list
- node summary, NodeManager REST APIs
- node metadata / The node summary
- node view, ResourceManager service / The ResourceManager's view, View 1 – Node
- NOPASSWD option
- URL, for blog / Step 2 – Copy the Hadoop folder to all the slave nodes
O
- octal code, for directory / file permissions
- reference link / Validating permissions to YARN directories
- open Hadoop ports
- references / Disable firewall or open Hadoop ports
- OpenMPI
- URL / Projects powered by YARN
- Oracle Virtualbox
- URL / Hardware requirements
P
- pid files, Hadoop-YARN services
- managing / Managing pid files
- placeholders, YARN REST services
- http address of service / Introduction to YARN REST services
- version / Introduction to YARN REST services
- resourcepath / Introduction to YARN REST services
- projects, YARN
- Giraph / Projects powered by YARN
- Spark / Projects powered by YARN
- MapReduce / Projects powered by YARN
- OpenMPI / Projects powered by YARN
- HAMA / Projects powered by YARN
- HBase / Projects powered by YARN
- Storm / Projects powered by YARN
- properties, in yarn-default.xml file
- reference link / Configuring the Hadoop-YARN services
- ProxyServer
Q
- query parameter list, applications list
- states / Applications
- finalStatus / Applications
- user / Applications, Applications
- queue / Applications
- limit / Applications
- startedTimeBegin / Applications
- startedTimeEnd / Applications
- finishedTimeBegin / Applications
- finishedTimeEnd / Applications
- applicationTypes / Applications
- applicationTags / Applications
- state / Applications
- query parameter list, applications statistics
- States / Applications
- applicationTypes / Applications
- Queue Access Control List
- about / An overview of queues
- queue ACL
- about / The queue ACL
- QueueInfo object
- about / An overview of queues
- QueueMetrics class / An overview of queues
- queue object
- queue name / An overview of queues
- queue information / An overview of queues
- queues
- overview / An overview of queues
- types / Types of queues
- CapacityScheduler Queue (CSQueue) / CapacityScheduler Queue (CSQueue)
- FairScheduler Queue (FSQueue) / FairScheduler Queue (FSQueue)
R
- resource localization, YARN
- handling / Handling resource localization in YARN
- LocalResource / Resource localization terminologies
- ResourceState / Resource localization terminologies
- LocalizerTracker / Resource localization terminologies
- PublicLocalizer / Resource localization terminologies
- LocalizerRunner / Resource localization terminologies
- directory structure / The resource localization directory structure
- ResourceManager
- components / ResourceManager
- about / Starting with the basics, ResourceManager / NodeManager / ProxyServer, Understanding ResourceManager
- client and admin interfaces / The client and admin interfaces
- core interfaces / The core interfaces
- NodeManager interfaces / The NodeManager interfaces
- security / The security and token managers
- token managers / The security and token managers
- ResourceManager failure
- about / The ResourceManager failure
- ResourceManager High Availability
- about / Understanding ResourceManager's High Availability
- architecture / Architecture
- failover mechanisms / Failover mechanisms
- configuring / Configuring ResourceManager's High Availability
- ResourceManager JMX beans
- about / The ResourceManager JMX beans
- ClusterMetrics / The ResourceManager JMX beans
- RpcActivity / The ResourceManager JMX beans
- UgiMetrics / The ResourceManager JMX beans
- ResourceManager REST APIs
- about / ResourceManager REST APIs
- cluster summary / The cluster summary
- scheduler details / Scheduler details
- nodes / Nodes
- applications / Applications
- URL / Applications, An overview of queues
- ResourceManager service
- about / The ResourceManager service, An introduction to state management analogy, The ResourceManager's view
- node view / The ResourceManager's view, View 1 – Node
- application view / The ResourceManager's view, View 2 – Application
- application attempt view / The ResourceManager's view, View 3 – An application attempt
- container view / The ResourceManager's view, View 4 – Container
- view response / RESTClient plugins
- JSON response / RESTClient plugins
- XML response / RESTClient plugins
- ResourceManager services
- enumerations / View 1 – Node
- classes / View 1 – Node
- ResourceManager web UI
- REST API
- calling, curl used / Curl command
- RESTClient plugin
- about / RESTClient plugins
- URL / RESTClient plugins
- using, in Firefox / RESTClient plugins
- REST services
- accessing / How to access REST services
- RHEL (RedHat Enterprise Linux)
- about / Supported platforms
- rmadmin command
- about / RMAdmin
- options / Command options
- runnable jar
S
- sample application, YARN web user-interface
- running / Run a sample application
- URL / Run a sample application
- sample MapReduce application submission
- about / Submitting a sample MapReduce application
- application, submitting to cluster / Submitting an application to the cluster
- updates, in ResourceManager web UI / Updates in the ResourceManager web UI
- application process / Understanding the application process
- application details, tracking / Tracking application details
- ApplicationMaster process / The ApplicationMaster process
- cluster nodes information / Cluster nodes information
- container list / Node's container list
- YARN child process / YARN child processes
- application details, after completion / Application details after completion
- Scheduler / ResourceManager
- scheduler details, ResourceManager REST APIs
- about / Scheduler details
- URI / Scheduler details
- example / Scheduler details
- Scheduler Load Simulator (SLS), YARN / YARN Scheduler Load Simulator (SLS)
- schedulers
- about / An introduction to schedulers
- fair scheduler / Fair scheduler
- CapacityScheduler / CapacityScheduler
- scheduling
- scripts, YARN
- start-yarn.sh / Managing the Hadoop-YARN services
- stop-yarn.sh / Managing the Hadoop-YARN services
- yarn-daemon.sh / Managing the Hadoop-YARN services
- command syntax / Managing the Hadoop-YARN services
- security
- adding, to YARN cluster / Adding security to a YARN cluster
- security and token managers, NodeManager
- about / The security and token managers
- NMTokenSecretManagerInNM / The security and token managers
- NMContainerTokenSecretManager / The security and token managers
- security and token managers, ResourceManager
- about / The security and token managers
- RMAuthenticationHandler / The security and token managers
- QueueACLsManager / The security and token managers
- TokenSecretManager / The security and token managers
- security frameworks
- about / Other security frameworks
- Apache Ranger / Apache Ranger
- Apache Knox / Apache Knox
- service-level ACL
- about / The service-level ACL
- service logs, Hadoop-YARN services
- managing / Managing service logs
- services logs
- about / Services logs
- single application information
- URI / Applications, Applications
- example / Applications, Applications
- about / Applications
- single container information
- example / Containers
- URI / Containers
- single job information
- single node information
- single task attempt information
- single task information
- slaves file
- about / The slaves file
- Slider
- source command
- Spark
- about / Projects powered by YARN, The integration of Spark with YARN
- URL / Projects powered by YARN, The integration of Spark with YARN
- integrating, with YARN / The integration of Spark with YARN
- running, on YARN / Running Spark on YARN
- Spark-Hadoop bundle
- URL, for downloading / Running Spark on YARN
- Spark-on-YARN
- URL / Running Spark on YARN
- standalone Zookeeper quorum
- state
- state and health management, NodeManager
- about / State and health management
- NodeHealthCheckerService / State and health management
- NMStateStoreService / State and health management
- state management analogy
- state transition
- status updates, NodeManager
- about / Status updates
- NodeStatusUpdater / Status updates
- NodeManagerMetrics / Status updates
- steps, Hadoop-YARN multi-node installation
- about / Installation steps
- master node, configuring as single-node Hadoop-YARN installation / Step 1 – Configure the master node as a single-node Hadoop-YARN installation
- Hadoop folder, copying to slave nodes / Step 2 – Copy the Hadoop folder to all the slave nodes
- environment variables, configuring on slave nodes / Step 3 – Configure environment variables on slave nodes
- NameNode, formatting / Step 4 – Format NameNode
- Hadoop daemons, starting / Step 5 – Start Hadoop daemons
- steps, Hadoop-YARN single node installation
- about / Installation steps
- Hadoop bundle, downloading / Step 1 – Download and extract the Hadoop bundle
- Hadoop bundle, extracting / Step 1 – Download and extract the Hadoop bundle
- environment variables, configuring / Step 2 – Configure the environment variables
- Hadoop configuration files, configuring / Step 3 – Configure the Hadoop configuration files
- NameNode, formatting / Step 4 – Format NameNode
- Hadoop daemons, starting / Step 5 – Start Hadoop daemons
- Storm
- URL / Projects powered by YARN, The integration of Storm with YARN
- about / The integration of Storm with YARN
- integrating, with YARN / The integration of Storm with YARN
- URL, for official documentation / The integration of Storm with YARN
- running, on YARN / Running Storm on YARN
- managing, on YARN / Managing Storm on YARN
- Storm-Yarn
- Storm-YARN cluster
- launching / Launching the Storm-YARN cluster
- storm.yaml file
- configuring / Configuring the storm.yaml file
- StormAMRMClient service
- reference link / Managing Storm on YARN
- Storm bundle
- URL, for downloading / Download, extract, and prepare the Storm bundle
- Storm cluster mode deployment
- URL, for official documentation / Running Storm on YARN
- StormMasterCommand class
- reference link / Managing Storm on YARN
- Storm ZIP
- copying, to HDFS / Copy Storm ZIP to HDFS
- sudo command
- system hostname, modifying in Linux
- URLs, for blog / Configure the domain name resolution
T
- task attempt counters
- task attempts
- task counters
- Tasks, MapReduce ApplicationMaster REST APIs
- tasks list
- TaskTracker
- functionalities / Introduction to MapReduce v1
- Timeline server
- about / The Timeline server
- reference link / The Timeline server
- transitions, analyzing through logs
- about / Analyzing transitions through logs
- NodeManager registration, with ResourceManager / NodeManager registration with ResourceManager
- application submission / Application submission
- container resource allocation / Container resource allocation
- resource localization / Resource localization
- Twill
U
- Ubuntu
- user commands, Hadoop-YARN
- about / The user commands
- jar / Jar
- application / Application
- node / Node, Sample output
- logs / Logs, Command options
- classpath / Classpath
- version / Version
V
- version command
- about / Version
W
- web application proxy server
- Web Application proxy server
- web user interfaces
- overview / An overview of web user interfaces
Y
- YARN
- components / An overview of YARN components
- benefits / How YARN satisfies big data needs
- projects / Projects powered by YARN
- scripts / Managing the Hadoop-YARN services
- MRv1 apps, running on / Running and monitoring MRv1 apps on YARN
- about / An introduction to the YARN API
- Timeline server / The YARN Timeline server
- Scheduler Load Simulator (SLS) / YARN Scheduler Load Simulator (SLS)
- resource localization, handling / Handling resource localization in YARN
- Spark, integrating with / The integration of Spark with YARN
- Spark, running on / Running Spark on YARN
- Storm, integrating with / The integration of Storm with YARN
- Storm, running on / Running Storm on YARN
- Storm, managing on / Managing Storm on YARN
- HAMA, integrating with / The integration of HAMA and Giraph with YARN
- Giraph, integrating with / The integration of HAMA and Giraph with YARN
- yarn-env.sh file
- yarn-site.xml file
- about / The yarn-site.xml file
- YARN application logging
- about / YARN application logging
- services logs / Services logs
- application logs / Application logs
- YARN architecture
- about / The YARN architecture
- YARNChild class
- reference link / Task execution as containers
- YARN client API
- about / YARN client API
- classes / YARN client API
- URL / YARN client API
- YARN cluster
- security, adding to / Adding security to a YARN cluster
- YARNConfiguration class
- about / YARNConfiguration
- resources, loading / Load resources
- final properties / Final properties
- variable expansion / Variable expansion
- URL, for documentation / Variable expansion
- YARN ResourceManager
- web-interface / An overview of web user interfaces
- YARN REST services
- about / Introduction to YARN REST services
- placeholders / Introduction to YARN REST services
- HTTP request / HTTP request and response
- HTTP response / HTTP request and response
- successful response / Successful response
- response, with error / Response with an error
- YARN schedulers
- YARN services
- monitoring / Monitoring the YARN services
- monitoring, JMX used / JMX monitoring
- monitoring, Ganglia used / Ganglia monitoring
Z
- Zookeeper
- URL, for documentation / The RM state store mechanism, Automatic failover
- Zookeeper ACLs
- about / The administration ACL
- URL / The administration ACL
- Zookeeper Quorum
- creating / Create a Zookeeper quorum