Apache Hive Cookbook

Apache Hive Cookbook

Overview of this book

Hive was developed by Facebook and later open sourced in Apache community. Hive provides SQL like interface to run queries on Big Data frameworks. Hive provides SQL like syntax also called as HiveQL that includes all SQL capabilities like analytical functions which are the need of the hour in today’s Big Data world. This book provides you easy installation steps with different types of metastores supported by Hive. This book has simple and easy to learn recipes for configuring Hive clients and services. You would also learn different Hive optimizations including Partitions and Bucketing. The book also covers the source code explanation of latest Hive version. Hive Query Language is being used by other frameworks including spark. Towards the end you will cover integration of Hive with these frameworks.

Apache Hive Cookbook

Credits

About the Authors

About the Reviewer

www.PacktPub.com

Preface

Free Chapter

Developing Hive

Introduction

Deploying Hive on a Hadoop cluster

Deploying Hive Metastore

Installing Hive

Configuring HCatalog

Understanding different components of Hive

Compiling Hive from source

Hive packages

Debugging Hive

Running Hive

Changing configurations at runtime

Services in Hive

Introducing HiveServer2

Understanding HiveServer2 properties

Configuring HiveServer2 high availability

Using HiveServer2 clients

Introducing the Hive metastore service

Configuring high availability of metastore service

Introducing Hue

Understanding the Hive Data Model

Introduction

Using numeric data types

Using string data types

Using Date/Time data types

Using miscellaneous data types

Using complex data types

Using operators

Partitioning

Partitioning a managed table

Partitioning an external table

Bucketing

Hive Data Definition Language

Introduction

Creating a database schema

Dropping a database schema

Altering a database schema

Using a database schema

Showing database schemas

Describing a database schema

Altering table properties

Creating views

Dropping views

Altering the view properties

Altering the view as select

Showing tables

Showing partitions

Show the table properties

Showing create table

HCatalog

WebHCat

Hive Data Manipulation Language

Introduction

Loading files into tables

Inserting data into Hive tables from queries

Inserting data into dynamic partitions

Writing data into files from queries

Enabling transactions in Hive

Inserting values into tables from SQL

Updating data

Deleting data

Hive Extensibility Features

Introduction

Serialization and deserialization formats and data types

Exploring views

Exploring indexes

Hive partitioning

Creating buckets in Hive

Analytics functions in Hive

Windowing in Hive

File formats

Joins and Join Optimization

Understanding the joins concept

Using a left/right/full outer join

Using a left semi join

Using a cross join

Using a map-side join

Using a bucket map join

Using a bucket sort merge map join

Using a skew join

Statistics in Hive

Bringing statistics in to Hive

Table and partition statistics in Hive

Column statistics in Hive

Top K statistics in Hive

Functions in Hive

Using built-in functions

Using the built-in User-defined Aggregation Function (UDAF)

Using the built-in User Defined Table Function (UDTF)

Creating custom User-Defined Functions (UDF)

Hive Tuning

Enabling predicate pushdown optimizations in Hive

Optimizations to reduce the number of map

Sampling

Hive Security

Securing Hadoop

Authorizing Hive

Configuring the SQL standards-based authorization

Authenticating Hive

Hive Integration with Other Frameworks

Working with Apache Spark

Working with Accumulo

Working with HBase

Working with Google Drill

Index

Customer Reviews

5 star

4 star

3 star

2 star

1 star

Compiling Hive from source

In this recipe, we will see how to compile Hive from source.

Getting ready

Apache Hive is an open source framework available for compilation and modification by any user. Hive source code is a maven project. The source has intermittent scripts executed on a UNIX platform during compilation.

The following prerequisites need to be installed:

UNIX OS: UNIX is preferable for Hive source compilation. Although the source could also be compiled on Windows, you need to comment out the intermittent scripts execution.
Maven: The following are the steps to configure maven:
1. Download the Apache maven binaries for Linux (.tar.gz) from https://maven.apache.org/download.cgi.
```
wget http://mirror.olnevhost.net/pub/apache/maven/maven-3/3.3.3/binaries/apache-maven-3.3.3-bin.tar.gz
```
2. Extract the tar file:
```
tar -xzvf apache-maven-3.3.3-bin.tar.gz
```
3. Create a folder and move maven binaries to that folder:
```
sudo mkdir –p /usr/lib/maven
mv apache-maven-3.3.3-bin/usr/lib/maven/
```
4. Open /etc/environment:
```
sudo nano /etc/profile
```
5. Add the following variable for the environment PATH:
```
export M2_HOME=/usr/lib/maven/apache-maven-3.3.3-bin
export M2=$M2_HOME/bin
export PATH=$M2:$PATH
```
6. Use the command source /etc/environment to add variables to PATH without restart:
```
source /etc/environment
```
7. Check whether maven is properly installed or not:
```
mvn –version
```

How to do it...

Follow these steps to compile Hive on a Unix OS:

Download the latest version of the Hive source tar file:

sudo wget http://a.mbbsindia.com/hive/hive-1.2.1/apache-hive-1.2.1-src.tar.gz

Extract the source folder:

tar –xzvf apache-hive-1.2.1-src.tar.gz

Move to the Hive directory:
```
cd apache-hive-1.2.1-src
```
To import Hive packages in eclipse, run the following command:
```
mvn eclipse:eclipse
```
To compile Hive with Hadoop 2 binaries, run the following command:
```
mvn clean install -Phadoop-2,dist
```
In case you want to skip tests execution, run the earlier command with the following switch:
```
mvn clean install –DskipTests -Phadoop-2,dist
```
To generate a tarball file from the source code, run the following command:
```
mvn clean package -DskipTests -Phadoop-2 -Pdist
```

Apache Hive Cookbook

Apache Hive Cookbook

Overview of this book

Related Content you might be interested in

Current Title:

Apache Hive Cookbook

Compiling Hive from source

Getting ready

How to do it...