Apache Hivemall (https://hivemall.incubator.apache.org/) is a collection of Hive UDFs for machine learning. It contains a number of ML algorithm implementations across classification, regression, recommendations, loss functions, and feature engineering, all as UDFs. This allows end users to use SQL and only SQL to apply machine learning algorithms to a large volume of training data. Perform the following steps to set it up:
- Download Hivemall from https://hivemall.incubator.apache.org/download.html and put it into HDFS:
$ hdfs fs -mkdir -p /apps/hivemall
$ hdfs fs -put hivemall-all-xxx.jar /apps/hivemall
- Create permanent functions using script here (https://github.com/apache/incubator-hivemall/blob/master/resources/ddl/define-all-as-permanent.hive):
> CREATE DATABASE IF NOT EXISTS hivemall; -- create a db for the
udfs
> USE hivemall...