Book Image

Ceph Cookbook

Book Image

Ceph Cookbook

Overview of this book

Ceph is a unified, distributed storage system designed for excellent performance, reliability, and scalability. This cutting-edge technology has been transforming the storage industry, and is evolving rapidly as a leader in software-defined storage space, extending full support to cloud platforms such as Openstack and Cloudstack, including virtualization platforms. It is the most popular storage backend for Openstack, public, and private clouds, so is the first choice for a storage solution. Ceph is backed by RedHat and is developed by a thriving open source community of individual developers as well as several companies across the globe. This book takes you from a basic knowledge of Ceph to an expert understanding of the most advanced features, walking you through building up a production-grade Ceph storage cluster and helping you develop all the skills you need to plan, deploy, and effectively manage your Ceph cluster. Beginning with the basics, you’ll create a Ceph cluster, followed by block, object, and file storage provisioning. Next, you’ll get a step-by-step tutorial on integrating it with OpenStack and building a Dropbox-like object storage solution. We’ll also take a look at federated architecture and CephFS, and you’ll dive into Calamari and VSM for monitoring the Ceph environment. You’ll develop expert knowledge on troubleshooting and benchmarking your Ceph storage cluster. Finally, you’ll get to grips with the best practices to operate Ceph in a production environment.

Related Content you might be interested in

Current Title:

Small book image
Ceph Cookbook
Feb, 2016 | 326 pages
No titles found
Table of Contents (18 chapters)
Ceph Cookbook
Ceph Cookbook
Credits
Credits
Foreword
Foreword
About the Author
About the Author
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Ceph – Introduction and Beyond
Ceph – Introduction and Beyond
Introduction
Ceph – the beginning of a new era
RAID – the end of an era
Ceph – the architectural overview
Planning the Ceph deployment
Setting up a virtual infrastructure
Installing and configuring Ceph
Scaling up your Ceph cluster
Using Ceph cluster with a hands-on approach
2
Working with Ceph Block Device
Working with Ceph Block Device
Introduction
Working with Ceph Block Device
Configuring Ceph client
Creating Ceph Block Device
Mapping Ceph Block Device
Ceph RBD resizing
Working with RBD snapshots
Working with RBD Clones
A quick look at OpenStack
Ceph – the best match for OpenStack
Setting up OpenStack
Configuring OpenStack as Ceph clients
Configuring Glance for Ceph backend
Configuring Cinder for Ceph backend
Configuring Nova to attach Ceph RBD
Configuring Nova to boot instances from Ceph RBD
3
Working with Ceph Object Storage
Working with Ceph Object Storage
Introduction
Understanding Ceph object storage
RADOS Gateway standard setup, installation, and configuration
Creating the radosgw user
Accessing Ceph object storage using S3 API
Accessing Ceph object storage using the Swift API
Integrating RADOS Gateway with OpenStack Keystone
Configuring Ceph federated gateways
Testing the radosgw federated configuration
Building file sync and share service using RGW
4
Working with the Ceph Filesystem
Working with the Ceph Filesystem
Introduction
Understanding Ceph Filesystem and MDS
Deploying Ceph MDS
Accessing CephFS via kernel driver
Accessing CephFS via FUSE client
Exporting Ceph Filesystem as NFS
ceph-dokan – CephFS for Windows clients
CephFS a drop-in replacement for HDFS
5
Monitoring Ceph Clusters using Calamari
Monitoring Ceph Clusters using Calamari
Introduction
Ceph cluster monitoring – the classic way
Monitoring Ceph clusters
Introducing Ceph Calamari
Building Calamari server packages
Building Calamari client packages
Setting up Calamari master server
Adding Ceph nodes to Calamari
Monitoring Ceph clusters from the Calamari dashboard
Troubleshooting Calamari
6
Operating and Managing a Ceph Cluster
Operating and Managing a Ceph Cluster
Introduction
Understanding Ceph service management
Managing the cluster configuration file
Running Ceph with SYSVINIT
Running Ceph as a service
Scale-up versus scale-out
Scaling out your Ceph cluster
Scaling down your Ceph cluster
Replacing a failed disk in the Ceph cluster
Upgrading your Ceph cluster
Maintaining a Ceph cluster
7
Ceph under the Hood
Ceph under the Hood
Introduction
Ceph scalability and high availability
Understanding the CRUSH mechanism
CRUSH map internals
Ceph cluster map
High availability monitors
Ceph authentication and authorization
Ceph dynamic cluster management
Ceph placement group
Placement group states
Creating Ceph pools on specific OSDs
8
Production Planning and Performance Tuning for Ceph
Production Planning and Performance Tuning for Ceph
Introduction
The dynamics of capacity, performance, and cost
Choosing the hardware and software components for Ceph
Ceph recommendation and performance tuning
Ceph erasure coding
Creating an erasure coded pool
Ceph cache tiering
Creating a pool for cache tiering
Creating a cache tier
Configuring a cache tier
Testing a cache tier
9
The Virtual Storage Manager for Ceph
The Virtual Storage Manager for Ceph
Introduction
Understanding the VSM architecture
Setting up the VSM environment
Getting ready for VSM
Installing VSM
Creating a Ceph cluster using VSM
Exploring the VSM dashboard
Upgrading the Ceph cluster using VSM
VSM roadmap
VSM resources
10
More on Ceph
More on Ceph
Introduction
Benchmarking the Ceph cluster
Disk performance baseline
Baseline network performance
Ceph RADOS bench
RADOS load-gen
Benchmarking the Ceph block device
Benchmarking Ceph RBD using FIO
Ceph admin socket
Using the ceph tell command
Ceph REST API
Profiling Ceph memory
Deploying Ceph using Ansible
The ceph-objectstore tool
Index
Index

Installing and configuring Ceph

To deploy our first Ceph cluster, we will use the ceph-deploy tool to install and configure Ceph on all three virtual machines. The ceph-deploy tool is a part of the Ceph software-defined storage, which is used for easier deployment and management of your Ceph storage cluster. In the previous section, we created three virtual machines with CentOS7, which have connectivity with the Internet over NAT, as well as private host-only networks.

We will configure these machines as Ceph storage clusters, as mentioned in the following diagram:

Creating Ceph cluster on ceph-node1

We will first install Ceph and configure ceph-node1 as the Ceph monitor and the Ceph OSD node. Later recipes in this chapter will introduce ceph-node2 and ceph-node3.

How to do it…

  1. Install ceph-deploy on ceph-node1:

    # yum install ceph-deploy -y

  2. Next, we will create a Ceph cluster using ceph-deploy by executing the following command from ceph-node1:

    # mkdir /etc/ceph ; cd /etc/ceph
# ceph-deploy new ceph-node1

    The new subcommand in ceph-deploy deploys a new cluster with ceph as the cluster name, which is by default; it generates a cluster configuration and keying files. List the present working directory; you will find the ceph.conf and ceph.mon.keyring files:

  3. To install Ceph software binaries on all the machines using ceph-deploy, execute the following command from ceph-node1:

    # ceph-deploy install ceph-node1 ceph-node2 ceph-node3

    The ceph-deploy tool will first install all the dependencies followed by the Ceph Giant binaries. Once the command completes successfully, check the Ceph version and Ceph health on all the nodes, as shown as follows:

    # ceph -v

  4. Create first the Ceph monitor in ceph-node1:

    # ceph-deploy mon create-initial

    Once the monitor creation is successful, check your cluster status. Your cluster will not be healthy at this stage:

    # ceph -s

  5. Create OSDs on ceph-node1:

    1. List the available disks on ceph-node1:

      # ceph-deploy disk list ceph-node1

      From the output, carefully select the disks (other than the OS partition) on which we should create the Ceph OSD. In our case, the disk names are sdb, sdc, and sdd.

    2. The disk zap subcommand would destroy the existing partition table and content from the disk. Before running this command, make sure that you are using the correct disk device name:

      # ceph-deploy disk zap ceph-node1:sdb ceph-node1:sdc ceph-node1:sdd

    3. The osd create subcommand will first prepare the disk, that is, erase the disk with a filesystem that is xfs by default, and then it will activate the disk's first partition as data partition and its second partition as journal:

      # ceph-deploy osd create ceph-node1:sdb ceph-node1:sdc ceph-node1:sdd

    4. Check the Ceph status and notice the OSD count. At this stage, your cluster would not be healthy; we need to add a few more nodes to the Ceph cluster so that it can replicate objects three times (by default) across cluster and attain healthy status. You will find more information on this in the next recipe:

      # ceph -s
Previous Section
End of Section 8
Next Section