Book Image

MySQL 8 for Big Data

By : Shabbir Challawala, Chintan Mehta, Kandarp Patel, Jaydip Lakhatariya
Book Image

MySQL 8 for Big Data

By: Shabbir Challawala, Chintan Mehta, Kandarp Patel, Jaydip Lakhatariya

Overview of this book

With organizations handling large amounts of data on a regular basis, MySQL has become a popular solution to handle this structured Big Data. In this book, you will see how DBAs can use MySQL 8 to handle billions of records, and load and retrieve data with performance comparable or superior to commercial DB solutions with higher costs. Many organizations today depend on MySQL for their websites and a Big Data solution for their data archiving, storage, and analysis needs. However, integrating them can be challenging. This book will show you how to implement a successful Big Data strategy with Apache Hadoop and MySQL 8. It will cover real-time use case scenario to explain integration and achieve Big Data solutions using technologies such as Apache Hadoop, Apache Sqoop, and MySQL Applier. Also, the book includes case studies on Apache Sqoop and real-time event processing. By the end of this book, you will know how to efficiently use MySQL 8 to manage data for your Big Data applications.
Table of Contents (17 chapters)
Title Page
Credits
About the Authors
About the Reviewers
www.PacktPub.com
Customer Feedback
Preface

Integrating Apache Sqoop with MySQL and Hadoop


Apache Sqoop can only work if Hadoop is installed on the server. Apache Sqoop requires Linux based operating system to work . ForHadoop and Sqoop to work on the Linux server, Java must be installed on the server. Once Sqoop is installed on the server, we will need to download Sqoop's MySQL connector which will allow JDBC driver to connect with MySQL database for transferring data with Hadoop.

Hadoop

is an open source, Big Data framework to process and analyze large amount of data sets quickly by using a cluster of environment. Because of Hadoop's multiple slave nodes environment, it's easy to avoid system failure or data loss if one or more nodes go off. Hadoop basically works with multiple modules such as Yet Another Resource Negotiator (YARN), Hadoop distributed file system (HDFS), and MapReduce. Hadoop's MapReduce algorithm is used for parallel processing of the data. MapReduce is used to convert unstructured data to a structured format using...