Next, we want to create a Hive external table on top of S3 logs and use EMR to compute the results. We can do this using the following three different methods:
- Using EMR CLI
- Using EMR console
- Using web GUI
Creating sample data
How to do it...
It depends on your preferences. In my example, I will use EMR CLI. We should already be connected to the EMR cluster via SSH. Let's start to work with Hive:
- In EMR CLI type hive and it will launch Hive.
- Next, we can execute SQL commands. Let's create the table on top of the CloudFront logs that are stored in the S3 bucket. We will run this DDL, as follows:
hive>CREATE EXTERNAL TABLE IF NOT EXISTS cloudfront_logs (
DateObject Date,
Time STRING,
Location STRING,
Bytes...