LZO compression can be split; the LZO block format allows us to decompress the block at a random offset of the file. As the compression has many small blocks of compressed data, it allows the MapReduce jobs to be split along the boundaries thus allowing parallel threads to work on it. Hence, we get incredible speed to write and read.
This is the library that is shipped with most of the Linux distribution. In case you don't have it, you can download it from http://www.oberhumer.com/opensource/lzo/.
Based on the Linux version, you have to download appropriately:
> port fetch lzo2 # If for some reason LZO2 is already installed, please uninstall first before doing this > port edit lzo2 # A vim editor should open // Add the following block of text in the file and save the file. variant x86_64 description "Build the 64-bit." { configure.args-delete --build=x86-apple-darwin ABI=standard configure.cflags-delete -m32 configure.cxxflags-delete -m32...