If our Big Data is on the local filesystem, we need to move it to HDFS. In this section, we will list steps to move data from the local filesystem to the HDFS filesystem.
We assume that our Hadoop cluster has been properly configured and all the Hadoop daemons are running without any issues. And we assume that the data on the local system is in the directory /data
.
Perform the following steps to import data to HDFS:
Use the following command to create a data directory on HDFS:
hadoop fs -mkdir data
This command will create a directory
/user/hduser/data
in the HDFS filesystem.Copy the data file from the local directory to HDFS using the following command:
hadoop fs -cp file:///data/datafile /user/hduser/data
Alternatively, we can use the command
hadoop fs -put /data/datafile /user/hduser/data
.Verify the data file on HDFS with the following command:
hadoop fs -ls /user/hduser/data
Move the data file from the local directory to HDFS with the following...