In this section, we will dig into the HDFS Java interfaces and APIs that can be used to interact with the HDFS File System. We will also to focus on the method implementation of FileSystem
classes so that you can understand how to write your own implementation if needed. This will also help you when you test your program against certain test cases.
We will talk about two approaches to reading a file and will discuss when to use what. In the first approach, we can use the URL class, which is part of the java.net
package and can be used to read files stored on HDFS. The URL calls setURLStreamHandlerFactory()
, which requires an instance of FsUrlStreamHandlerFactory()
. This initialization is a part of a static block that is executed before any instance creation. This method is in a static block because it can only be called once per JVM and hence, if any third-party program sets URLStreamHandlerFactory
, we won't be able to use it for reading files from HDFS:
static...