In this recipe, you will learn how to use a bucket sort merge map join in Hive.
A bucket sort merge map join is an advanced version of a bucket map join. If the data in the tables is sorted and bucketed on the join columns at the same time then a bucket sort merge map join comes into the picture. In this type of join, all the tables must have an equal number of buckets as each mapper will read a bucket from each table and will perform a bucket sort merge map join.
It is mandatory for the data to be sorted in this join condition. The following parameter needs to be set to true for sorting the data or data can be sorted manually:
Set hive.enforce.sorting = true;
Note
If data in the buckets is not sorted then there is a possibility that a wrong result or output is generated as Hive does not check whether the buckets are sorted or not.
The following parameters need to be set for:
set hive.input.format = org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat; set...