In this chapter, we'll look at Hadoop services and try to understand how and on what ports they communicate. The aim of this chapter is not to configure the Hadoop cluster, but to understand it from the perspective of monitoring. Hadoop is a distributed platform with various services running across the cluster. The coordination between services and the way they communicate plays a very important role in the working of the cluster. The communication can be done using TCP/IP or RPC over TCP, or it could be simply done using HTTP.
In this chapter, we will look at the communication between Hadoop components.
The following topics will be covered in this chapter:
Important services, ports used by Hadoop and how they communicate
Common issues faced by various daemons
Host level checks
Hadoop is highly configurable, and we can configure it to work optimally. Each of the Hadoop components has configuration files with which we can control service ports, data directories...