In this final chapter, we looked at the various Hadoop ecosystem technologies. We set up Pig with ES-Hadoop and developed the script in order to interact with Elasticsearch. You learned how to integrate Hive in order to use SQL to write and read data to/from Elasticsearch. We saw how to flow the data through cascading pipes, make it sink to Elasticsearch, and flow it out from the Elasticsearch source. You also learned how to use ES-Hadoop to integrate Elasticsearch into Spark and empower it with a powerful SQL engine: SparkSQL.

We concluded with a small introduction on how to seamlessly integrate Elasticsearch to run it on the YARN resource manager in order to reduce the administration overhead.

Throughout the book, you learned how Elasticsearch empowers you to get a quick insight into your data, which can be in any form in the Hadoop ecosystem, such as HDFS, Parquet, Avro, S3, RDBMS, or streaming data. You can build real-time analytics systems that solve data mining problems, such...