Book Image

MySQL 8 for Big Data

By : Shabbir Challawala, Chintan Mehta, Kandarp Patel, Jaydip Lakhatariya
Book Image

MySQL 8 for Big Data

By: Shabbir Challawala, Chintan Mehta, Kandarp Patel, Jaydip Lakhatariya

Overview of this book

With organizations handling large amounts of data on a regular basis, MySQL has become a popular solution to handle this structured Big Data. In this book, you will see how DBAs can use MySQL 8 to handle billions of records, and load and retrieve data with performance comparable or superior to commercial DB solutions with higher costs. Many organizations today depend on MySQL for their websites and a Big Data solution for their data archiving, storage, and analysis needs. However, integrating them can be challenging. This book will show you how to implement a successful Big Data strategy with Apache Hadoop and MySQL 8. It will cover real-time use case scenario to explain integration and achieve Big Data solutions using technologies such as Apache Hadoop, Apache Sqoop, and MySQL Applier. Also, the book includes case studies on Apache Sqoop and real-time event processing. By the end of this book, you will know how to efficiently use MySQL 8 to manage data for your Big Data applications.
Table of Contents (17 chapters)
Title Page
Credits
About the Authors
About the Reviewers
www.PacktPub.com
Customer Feedback
Preface

MySQL Applier overview


MySQL Applier provides real-time data processing between MySQL and Hadoop. This is quite an efficient way to load data because of its performance and real-time processing. This will load the data which has been changed rather than loading whole data. Hence, No More Bulk Transfer needed!

MySQL Applier replicates rows inserted into MySQL to the HDFS with help of MySQL binlog. This will use binary log and insert data in real-time; based on events in MySQL. All the events occurred in MySQL Server is available in binlog and Applier takes these changes from the events and applies same in Hadoop. This way we can rapidly acquire new data from the MySQL.

MySQL Applier uses API provided by libhdfs which is C library. This library is pre-compiled with Hadoop distributions for connecting MySQL master or read a binlog file of MySQL. It is responsible for various operations as listed in the following points:

  • Get the events (insert, update, delete) occurred on the MySQL Server.
  • Decodes...