In this recipe, we are going to take a look at how to set the block size for a specific file only.
In the previous recipe, we learned how to change the block size at the cluster level. But this is not always required. HDFS provides us with the facility to set the block size for a single file as well. The following command copies a file called myfile
to HDFS, setting the block size to 1MB:
hadoop fs -Ddfs.block.size=1048576 -put /home/ubuntu/myfile /
Once the file is copied, you can verify whether the block size is set to 1MB and has been broken into exact chunks:
hdfs fsck -blocks /myfile Connecting to namenode via http://localhost:50070/fsck?ugi=ubuntu&blocks=1&path=%2Fmyfile FSCK started by ubuntu (auth:SIMPLE) from /127.0.0.1 for path /myfile at Thu Oct 29 14:58:00 UTC 2015 .Status: HEALTHY Total size: ...