Book Image

Ceph Cookbook - Second Edition

By : Vikhyat Umrao, Karan Singh, Michael Hackett
Book Image

Ceph Cookbook - Second Edition

By: Vikhyat Umrao, Karan Singh, Michael Hackett

Overview of this book

Ceph is a unified distributed storage system designed for reliability and scalability. This technology has been transforming the software-defined storage industry and is evolving rapidly as a leader with its wide range of support for popular cloud platforms such as OpenStack, and CloudStack, and also for virtualized platforms. Ceph is backed by Red Hat and has been developed by community of developers which has gained immense traction in recent years. This book will guide you right from the basics of Ceph , such as creating blocks, object storage, and filesystem access, to advanced concepts such as cloud integration solutions. The book will also cover practical and easy to implement recipes on CephFS, RGW, and RBD with respect to the major stable release of Ceph Jewel. Towards the end of the book, recipes based on troubleshooting and best practices will help you get to grips with managing Ceph storage in a production environment. By the end of this book, you will have practical, hands-on experience of using Ceph efficiently for your storage requirements.
Table of Contents (15 chapters)

Integrating RADOS Gateway with Hadoop S3A plugin 

For data analytics applications that require Hadoop Distributed File System (HDFS) access, the Ceph object gateway can be accessed using the Apache S3A connector for Hadoop. The S3A connector is an open source tool that presents S3 compatible object storage as an HDFS file system with HDFS file system read and write semantics to the applications while data is stored in the Ceph object gateway.

Ceph object gateway Jewel version 10.2.9 is fully compatible with the S3A connector that ships with Hadoop 2.7.3.

How to do it...

You can use client-node1 to configure Hadoop S3A client.

  1. Install Java packages in the client-node1:
        # yum install java* -y
  1. Download the...