With every Data Lake initiative, over 50 percent of the project time is spent in transforming the data from its raw format to something that can be consumed by the decision makers. In this chapter, we looked at the key tools available for data transformation; Hive and Pig both provide a layer of abstraction over MapReduce and are easy for business users to adopt. Pig has a scripting interface and Hive has a SQL interface. Azure PowerShell provides the capability to orchestrate these various jobs in a sequence. In the next chapter, we will review how to utilize and visualize the data generated by this transformed process.
HDInsight Essentials - Second Edition
By :
HDInsight Essentials - Second Edition
By:
Overview of this book
Table of Contents (16 chapters)
HDInsight Essentials Second Edition
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Free Chapter
Hadoop and HDInsight in a Heartbeat
Enterprise Data Lake using HDInsight
HDInsight Service on Azure
Administering Your HDInsight Cluster
Ingest and Organize Data Lake
Transform Data in the Data Lake
Analyze and Report from Data Lake
HDInsight 3.1 New Features
Strategy for a Successful Data Lake Implementation
Index
Customer Reviews