Book Image

Apache Hive Cookbook

Book Image

Apache Hive Cookbook

Overview of this book

Hive was developed by Facebook and later open sourced in Apache community. Hive provides SQL like interface to run queries on Big Data frameworks. Hive provides SQL like syntax also called as HiveQL that includes all SQL capabilities like analytical functions which are the need of the hour in today’s Big Data world. This book provides you easy installation steps with different types of metastores supported by Hive. This book has simple and easy to learn recipes for configuring Hive clients and services. You would also learn different Hive optimizations including Partitions and Bucketing. The book also covers the source code explanation of latest Hive version. Hive Query Language is being used by other frameworks including spark. Towards the end you will cover integration of Hive with these frameworks.
Table of Contents (19 chapters)
Apache Hive Cookbook
Credits
About the Authors
About the Reviewer
www.PacktPub.com
Preface
Index

Working with Apache Spark


In this recipe, you will learn how to integrate Hive with Apache Spark. Apache Spark is an open source cluster computing framework. It is used as a replacement of the MapReduce framework.

Getting ready

In this topic, we will cover the use of Hive and Apache Spark. You must have Apache Spark installed on your system before going further in the topic.

  1. Once the Spark is installed, start the Spark master server by executing the following command:

    ./sbin/start-master.sh
    
  2. Check whether the Spark master server has been started or not by issuing the URL mentioned later on the web browser:

    http://<ip_address>:<port_number>
    
  3. The exact URL is present at the following path:

    /spark-1.6.0-bin-hadoop2.6/logs/spark-hadoop-org.apache.spark.deploy.master.Master-1-node1.out
    
  4. The following screenshot shows the result of the URL:

  5. Once the master server is started, start the slave service by executing the following command:

    ./sbin/start-slave.sh <master-spark-URL>
    
  6. Refresh...