Book Image

Apache Hive Cookbook

Book Image

Apache Hive Cookbook

Overview of this book

Hive was developed by Facebook and later open sourced in Apache community. Hive provides SQL like interface to run queries on Big Data frameworks. Hive provides SQL like syntax also called as HiveQL that includes all SQL capabilities like analytical functions which are the need of the hour in today’s Big Data world. This book provides you easy installation steps with different types of metastores supported by Hive. This book has simple and easy to learn recipes for configuring Hive clients and services. You would also learn different Hive optimizations including Partitions and Bucketing. The book also covers the source code explanation of latest Hive version. Hive Query Language is being used by other frameworks including spark. Towards the end you will cover integration of Hive with these frameworks.
Table of Contents (19 chapters)
Apache Hive Cookbook
Credits
About the Authors
About the Reviewer
www.PacktPub.com
Preface
Index

Using the built-in User-defined Aggregation Function (UDAF)


Hive provides a set of functions to do aggregation on a dataset. These functions operate on a range of data (rows) and provide the cumulative or relative result.

How to do it…

The built-in functions could be used directly in the query. The following are some of the examples of aggregated functions available in Hive:

Function Name

Return Type

Description

avg(col)

DOUBLE

It is used to calculate the average of all values of a particular column.

avg(DISTINCT col)

DOUBLE

It is used to calculate the average of unique values of a particular column.

collect_list(col)

ARRAY

It will return a list of all values of a particular column in an array.

collect_set(col)

ARRAY

It will return a list of unique values of a particular column in an array. Duplicate values are eliminated.

corr(col1, col2)

DOUBLE

It is used to calculate the Pearson coefficient of correlation between two columns.

count(*)

BIGINT

It will return the total...