Index
A
- Access Control List (ACL) / Configuring service-level authentication
- ACL properties / There's more...
- Amazon EMR
- about / There's more...
- data processing with / Data processing with Amazon Elastic MapReduce
- Amazon Machine Image (AMI)
- about / Introduction
- creating / Creating an Amazon Machine Image (AMI), How to do it...
- creating, from existing AMI / Creating an AMI from an existing AMI
- Amazon Resource Name (ARN) / How to do it...
- Ambari
- about / Introduction
- URL / Monitoring a Hadoop cluster with Ambari
- configuring, for Hadoop cluster monitoring / Monitoring a Hadoop cluster with Ambari, Getting ready , How to do it...
- Apache Avro
- about / Apache Avro
- URL / Apache Avro
- Apache Flume
- about / Apache Flume, Introduction
- URL / Apache Flume
- Apache Free Software (AFS) / There’s more...
- Apache HBase
- about / Apache HBase
- URL / Apache HBase
- Apache Hive
- about / Apache Hive
- URL / Apache Hive
- Apache Mahout
- about / Apache Mahout
- URL / Apache Mahout
- Apache Oozie
- about / Apache Oozie
- URL / Apache Oozie
- Apache Pig
- about / Apache Pig
- URL / Apache Pig
- Apache Sqoop
- about / Apache Sqoop
- URL / Apache Sqoop
- Apache ZooKeeper
- about / Apache ZooKeeper
- URL / Apache ZooKeeper
- audit logging
- about / Configuring Hadoop audit logging
- configuring / Getting ready
- working / How it works...
- AWS
- registering with / Registering with Amazon Web Services (AWS), How to do it...
- security credentials, managing / Managing AWS security credentials, How to do it...
B
- balancer
- running / Running balancer
- benchmark commands
- about / How it works…
- block size
- selecting / Choosing a proper block size, How to do it...
C
- CapacityScheduler
- about / Configuring CapacityScheduler
- configuring / Getting ready, How to do it...
- working / How it works...
- queue configuration properties / How it works...
- CentOS 6.3 / Getting ready
- check_jmx Nagios plugin / Getting ready
- Chukwa
- about / Introduction, Monitoring a Hadoop cluster with Chukwa
- installing / Getting ready
- URL / Getting ready
- configuring, for Hadoop monitoring / How to do it...
- working / How it works...
- features / There’s more...
- Cloudera
- about / How it works…
- URL / How it works…
- cluster administrator machine
- cluster attribute / How to do it...
- cluster network
- designing / Designing the cluster network, How to do it...
- working / How it works...
- compression / Using compression for input and output
- configuration files, for pseudo-distributed mode
- hadoop-env.sh / How it works...
- core-site.xml / How it works...
- hdfs-site.xml / How it works...
- mapred-site.xml / How it works...
- masters file / How it works...
- slaves file / How it works...
- core-site.xml
- about / How it works...
- current live threads / How to do it...
D
- data
- importing, to HDFS / Importing data to HDFS, Getting ready, There's more...
- data blocks
- balancing, for Hadoop cluster / Balancing data blocks for a Hadoop cluster, How to do it..., How it works…
- Data Delivery subsystem / HPCC
- data local / Balancing data blocks for a Hadoop cluster
- DataNode
- about / Introduction
- decommissioning / Decommissioning DataNode, How it works...
- Data Refinery subsystem / HPCC
- data skew / Balancing data blocks for a Hadoop cluster
- decompression / Using compression for input and output
- dfs.data.dir property / How it works...
- dfs.replication property / How it works...
- dfsadmin command / How it works...
- DHCP
- configuring, for network booting / Configuring DHCP for network booting
E
- EBS-backed AMI
- creating / Creating an EBS-backed AMI
- EC2
- about / Introduction
- EC2 connection
- local machine, preparing / Preparing a local machine for EC2 connection, How to do it...
- erroneous iptables configuration
- about / Erroneous iptables configuration
- erroneous SELinux configuration
- about / Erroneous SELinux configuration
- erroneous SSH settings
- about / Erroneous SSH settings
F
- Fair Scheduler
- about / Configuring Fair Scheduler
- configuring / Getting ready, How to do it..., How it works...
- properties / How it works...
- files
- manipulating, on HDFS / Manipulating files on HDFS, How to do it..., How it works…
- folder, Rumen
- about / Analyzing job history with Rumen
- fs.default.name property / How it works...
- fsck command / How it works...
- fully-distributed mode
G
- Ganglia
- about / Introduction, Monitoring a Hadoop cluster with Ganglia
- configuring, for monitoring Hadoop cluster / Monitoring a Hadoop cluster with Ganglia, How to do it...
- monitoring daemon / Monitoring a Hadoop cluster with Ganglia
- metadata daemon / Monitoring a Hadoop cluster with Ganglia
- web UI / Monitoring a Hadoop cluster with Ganglia
- working / How it works...
- GitHub
- URL / How it works...
- GNU wget
- about / How it works...
- Gold Trace
- about / How to do it...
- GraphLab
- about / How it works…
- URL / How it works…
- GridMix
- about / Benchmarking a Hadoop cluster with GridMix
- used, for benchmarking Hadoop / Benchmarking a Hadoop cluster with GridMix, How to do it...
- working / How it works...
- GridMix1
- used, for benchmarking Hadoop / Benchmarking Hadoop cluster with GridMix1
- GridMix2
- streamSort / How it works...
- javaSort / How it works...
- webdataSort / How it works...
- combiner / How it works...
- monsterSort / How it works...
- GridMix2 benchmarks
- getting / How to do it...
- GridMix3
- used, for benchmarking Hadoop / Benchmarking Hadoop cluster with GridMix3
- Gzip codec / How to do it...
H
- Hadapt
- about / How it works…
- URL / How it works…
- Hadoop
- configuring, in pseudo-distributed mode / Configuring Hadoop in pseudo-distributed mode, Getting ready, How to do it...
- working, in pseudo-distributed mode / How it works...
- configuring, in fully-distributed mode / Configuring Hadoop in fully-distributed mode, Getting ready, How to do it..., There's more...
- job management commands / More job management commands
- tasks, managing / Managing tasks
- job management, from web UI / Managing jobs through the web UI
- upgrading / Upgrading Hadoop, How to do it...
- hadoop-env.sh file
- about / How it works...
- Hadoop-specific monitoring systems
- Ambari / Introduction
- Chukwa / Introduction
- hadoop.tmp.dir property / How it works...
- Hadoop alternatives
- selecting from / Choosing from Hadoop alternatives
- working / How it works…
- Hadoop audit logging
- configuring / Configuring Hadoop audit logging, How to do it...
- working / How it works...
- Hadoop cluster
- configuring / Introduction
- HDFS cluster / Introduction
- MapReduce cluster / Introduction
- hardening / Introduction
- securing, with Kerberos / Securing a Hadoop cluster with Kerberos, Getting ready, How to do it...
- monitoring, JMX used / Monitoring a Hadoop cluster with JMX, How to do it...
- monitoring, Ganglia used / Monitoring a Hadoop cluster with Ganglia, How to do it...
- monitoring, Nagios used / Monitoring a Hadoop cluster with Nagios, Getting ready, How to do it...
- monitoring, Ambari used / Monitoring a Hadoop cluster with Ambari, Getting ready , How to do it...
- monitoring, Chukwa used / Getting ready, How to do it..., How it works...
- benchmarking / Benchmarking and profiling a Hadoop cluster
- benchmarking, GridMix used / Benchmarking a Hadoop cluster with GridMix, How to do it...
- benchmarking, GridMix1 used / Benchmarking Hadoop cluster with GridMix1
- benchmarking, GridMix3 used / Benchmarking Hadoop cluster with GridMix3
- data blocks, balancing / Balancing data blocks for a Hadoop cluster, How to do it...
- input and output data compression, configuring / Using compression for input and output, How to do it...
- memory configuration properties, configuring / Getting ready, How to do it...
- configuring, with new AMI / Configuring a Hadoop cluster with the new AMI, How to do it...
- Hadoop cluster benchmarks
- HDFS benchmarks, performing / How to do it...
- MapReduce cluster, benchmarking / How to do it...
- working / How it works…
- Hadoop common
- about / Hadoop common
- Hadoop configuration problems
- HDFS daemons starting / Can't start HDFS daemons
- cluster, missing in slave nodes / Cluster is missing slave nodes
- MapReduce daemons starting issues / MapReduce daemons can't be started
- Hadoop daemon logging
- configuring / Configuring Hadoop daemon logging, Getting ready, How to do it..., How it works...
- configuring, hadoop-env.sh used / Configuring Hadoop logging with hadoop-env.sh
- Hadoop data compression properties / How it works...
- Hadoop distribution
- release version number / Getting ready
- version number / Getting ready
- major revision number / Getting ready
- minor revision number / Getting ready
- hadoop fs command / How it works…
- Hadoop Infrastructure Care Center (HICC) / How to do it...
- Hadoop installation
- validating / Validating Hadoop installation, How to do it..., How it works...
- Hadoop logs file naming conventions
- Hadoop NameNode
- about / Configuring SecondaryNameNode
- Hadoop performance tuning
- about / Introduction
- Hadoop releases
- about / How to do it...
- reference link / See also
- Hadoop security logging
- configuring / Configuring Hadoop security logging
- Hadoop Vaidya
- about / Using Hadoop Vaidya to identify performance problems
- using / How to do it...
- working / How it works...
- Hadoop version
- selecting / Choosing a Hadoop version
- Haloop
- about / How it works…
- URL / How it works…
- hardware, for cluster nodes
- HBase
- about / Installing HBase
- installing / Getting ready, How to do it...
- downloading / Getting ready
- working / How it works...
- hdfs-site.xml
- about / How it works...
- HDFS cluster
- about / Introduction
- managing / Managing the HDFS cluster, How to do it..., How it works..., There's more…
- data, importing / Importing data to HDFS, How to do it..., There's more...
- files, manipulating / Manipulating files on HDFS, How to do it..., How it works…
- HDFS federation / How to do it...
- configuring / Configuring HDFS federation, How to do it..., How it works...
- HDFS quota
- configuring / Configuring the HDFS quota, How to do it...
- heartbeat
- about / How it works...
- HiBench / There's more...
- Hive
- about / Installing Hive
- downloading / Getting ready
- installing / How to do it...
- Hortonworks
- about / How it works…
- URL / How it works…
- HPCC
I
- IAM role / How to do it...
- input and output data compression
- configuring / Using compression for input and output, How to do it...
- installation
- HBase / How to do it...
- Hive / How to do it...
- Pig / How to do it...
- Mahout / How to do it...
J
- J2SE platform 5.0 / Monitoring a Hadoop cluster with JMX
- Java
- installing / Installing Java and other tools, Getting ready, How to do it..., How it works..., There's more...
- downloading, from Oracle / Getting ready
- JMX
- about / Introduction
- used, for monitoring Hadoop cluster / Monitoring a Hadoop cluster with JMX, How to do it...
- job authorization
- configuring, with ACL / Configuring job authorization with ACL, How to do it...
- job command
- about / How it works...
- job history
- checking, from web UI / Checking job history from the web UI, How to do it..., How it works...
- analyzing, Rumen used / Analyzing job history with Rumen, Getting ready, How to do it...
- job management commands
- about / More job management commands
- jobs
- managing, from web UI / Managing jobs through the web UI
- JobTracker configuration
- tuning / Tuning the JobTracker configuration, How to do it...
- properties / How it works…
- JobTracker daemon
- about / Introduction
- journal node / How to do it...
- JVM parameters
- tuning / Tuning JVM parameters, How to do it...
- JVM Reuse
- configuring / Configuring JVM Reuse, Getting ready
- about / Configuring JVM Reuse
K
- Kerberos
- about / Securing a Hadoop cluster with Kerberos
- used, for securing Hadoop cluster / Securing a Hadoop cluster with Kerberos
- configuring, for Hadoop cluster / How to do it...
- kickstart file
- creating / Getting ready
- using / How to do it...
- working / How it works...
L
- (Local Area Network (LAN) / How to do it...
- Linux operating system
- installing / Installing the Linux operating system, How it works...
- local client machine
- preparing, for EC2 connection / Preparing a local machine for EC2 connection, How to do it...
- Log4j
- about / How it works...
- logging levels / How it works...
M
- Mahout
- about / Installing Mahout
- downloading / Getting ready
- installing / How to do it...
- map/reduce slots, TaskTracker
- MapR
- about / How it works…
- URL / How it works…
- mapred-site.xml
- about / How it works...
- mapred.job.tracker property / How it works...
- mapred.map.child.java.opts property / How it works...
- mapred.reduce.child.java.opts property / How it works...
- mapred.tasktracker.map.tasks.maximum property / How it works...
- mapred.tasktracker.reduce.tasks.maximum property / How it works...
- mapredtest benchmark / How to do it...
- about / How it works…
- MapReduce cluster
- about / Introduction, Managing the MapReduce cluster
- managing / Managing the MapReduce cluster, How to do it...
- MapReduce jobs
- managing / Managing MapReduce jobs, How to do it..., There's more...
- masters file
- about / How it works...
- memory configuration properties
- configuring / Getting ready, How to do it...
- listing / How it works...
- merge
- configuring / How to do it...
- MPI
- mradmin command / How it works...
- mrbench command / How to do it...
- multicast / How to do it...
N
- Nagios
- about / Introduction, Monitoring a Hadoop cluster with Nagios
- configuring, for monitoring Hadoop cluster / Getting ready, How to do it...
- working / How it works...
- URL / How it works...
- Nagios Remote Plugin Executor (NRPE) package
- installing / Getting ready
- NameNode
- recovering, from SecondaryNameNode checkpoint / Recovering NameNode from the checkpoint of a SecondaryNameNode
- decommissioning, from cluster / Decommissioning a NameNode from the cluster
- adding / Adding a new NameNode
- NameNode failure
- recovering from / Recovering from NameNode failure, How to do it..., There's more...
- NameNode HA / How to do it...
- configuring / Configuring NameNode high availability, How to do it...
- NameNode HA configuration
- testing / How to do it...
- working / How it works...
- NameNode resilience
- with multiple hard drives / NameNode resilience with multiple hard drives
- NameNodes
- about / Introduction
- Network Mapper (nmap)
- about / How it works...
- nnbench
- about / How it works…
- number of parallel copies
- configuring / How to do it...
- Nutch / There's more...
O
- open source cluster monitoring systems
- Ganglia / Introduction
- Nagios / Introduction
P
- PageRank / There's more...
- peak live threads / How to do it...
- Phoenix
- URL / How it works…
- about / How it works…
- Pig
- about / Installing Pig
- downloading / Getting ready
- installing / How to do it...
- platform as a service (PaaS)
- about / Introduction
- Preboot Execution Environment (PXE) method / Configuring DHCP for network booting
- pseudo-distributed mode
Q
- queue ACLs
- about / There's more...
- reference link / There's more...
- queue command
- about / How it works...
- quota
- about / Configuring the HDFS quota
- configuring / Configuring the HDFS quota
R
- randomwriter
- about / There's more...
- reducer initialization time
- configuring / Getting ready
- Remote Procedure Calls (RPC) / Securing a Hadoop cluster with Kerberos
- rsync
- about / How it works...
- Rumen
- about / Analyzing job history with Rumen
- used, for analyzing job history / Analyzing job history with Rumen, Getting ready, How to do it...
- TraceBuilder / Analyzing job history with Rumen
- folder / Analyzing job history with Rumen
S
- S3
- about / Introduction
- configuring, for data storage / Using S3 to host data, How to do it...
- SecondaryNameNode
- configuring / Configuring SecondaryNameNode, How to do it...
- secured ZooKeeper
- configuring / There's more...
- security credentials, AWS
- Security Enhanced Linux (SELinux)
- about / Erroneous SELinux configuration
- service-level authentication
- configuring / Configuring service-level authentication, Getting ready, How to do it...
- working / How it works...
- shuffle
- configuring / How to do it...
- slave node
- replacing / Replacing a slave node, How to do it...
- slaves file
- about / How it works...
- sort
- about / How to do it...
- sorting parameters
- configuring / Tuning shuffle, merge, and sort parameters, How to do it...
- properties / How it works…
- Spark
- about / How it works…
- URL / How it works…
- speculative execution / Introduction
- about / Configuring speculative execution
- configuring / Configuring speculative execution, How to do it...
- working / How it works...
- SPNEGO
- about / How it works...
- URL / How it works...
- SSH
- about / Configuring SSH
- configuring / Configuring SSH, How to do it..., There's more...
- start-all.sh script
- about / How it works...
- start-dfs.sh script
- about / How it works...
- start-mapred.sh script
- about / How it works...
- stop-all.sh script
- about / How it works...
- stop-dfs.sh script
- about / How it works...
- stop-mapred.sh script
- about / How it works...
- Storm
- about / How it works…
- URL / How it works…
- system monitoring
- about / Introduction
T
- tasks
- managing / Managing tasks
- TaskTracker configuration
- tuning / Tuning the TaskTracker configuration, Getting ready
- properties, configuring / How to do it...
- properties / How it works…
- TaskTrackers
- about / Introduction, Managing TaskTracker
- blacklist / Managing TaskTracker
- gray list / Managing TaskTracker
- excluded list / Managing TaskTracker
- managing / Managing TaskTracker, Getting ready, How to do it...
- working / How it works...
- heartbeat / How it works...
- testbigmapoutput benchmark
- about / How it works…
- testfilesystem benchmark
- about / How it works…
- TFTP
- configuring, for network booting / Configuring TFTP for network booting:
- threadedmapbench
- about / How to do it...
- TraceBuilder
- about / Analyzing job history with Rumen
U
- udp_recv_channel attribute / How to do it...
- udp_send_channel attribute / How to do it...
- unicast / How to do it...
- USB boot media
- creating / How to do it...
W
- web UI
- job history, checking from / How to do it...
- web UI authentication
- configuring / Configuring web UI authentication, How to do it...
- working / How it works..., There's more...
Y
- yum command / Getting ready
Z
- znode / How to do it...
- ZooKeeper
- about / Configuring ZooKeeper
- downloading / Getting ready
- configuring / How to do it...