The use cases selected for elucidating the Spark SQL way of programming with DataFrame are given as follows:
The transaction records come as comma-separated values.
Filter out only the good transaction records from the list. The account number should start with
SB
and the transaction amount should be greater than zero.Find all the high-value transaction records with a transaction amount greater than 1000.
Find all the transaction records where the account number is bad.
Find all the transaction records where the transaction amount is less than or equal to zero.
Find a combined list of all the bad transaction records.
Find the total of all the transaction amounts.
Find the maximum of all the transaction amounts.
Find the minimum of all the transaction amounts.
Find all the good account numbers.
This is exactly the same set of use cases that were used in the previous chapter as well, but here the programming model is totally different. Using this set of use cases, two types of...