In this section, we'll implement a custom partitioner and create a partitioner that takes a list of parses with ranges. If our key falls into a specific range, we will assign the partition number index of the list.
We will cover the following topics:
- Implementing a custom partitioner
- Implementing a range partitioner
- Testing our partitioner
We will implement the logic range partitioning based on our own range partitioning and then test our partitioner. Let's start with the black box test without looking at the implementation.
The first part of the code is similar to what we have used already, but this time we have keyBy amount of data, as shown in the following example:
val keysWithValuesList =
Array(
UserTransaction("A", 100),
UserTransaction("B", 4),
UserTransaction("A", 100001),
UserTransaction("...