Book Image

Ceph Cookbook - Second Edition

By : Vikhyat Umrao, Karan Singh, Michael Hackett
Book Image

Ceph Cookbook - Second Edition

By: Vikhyat Umrao, Karan Singh, Michael Hackett

Overview of this book

Ceph is a unified distributed storage system designed for reliability and scalability. This technology has been transforming the software-defined storage industry and is evolving rapidly as a leader with its wide range of support for popular cloud platforms such as OpenStack, and CloudStack, and also for virtualized platforms. Ceph is backed by Red Hat and has been developed by community of developers which has gained immense traction in recent years. This book will guide you right from the basics of Ceph , such as creating blocks, object storage, and filesystem access, to advanced concepts such as cloud integration solutions. The book will also cover practical and easy to implement recipes on CephFS, RGW, and RBD with respect to the major stable release of Ceph Jewel. Towards the end of the book, recipes based on troubleshooting and best practices will help you get to grips with managing Ceph storage in a production environment. By the end of this book, you will have practical, hands-on experience of using Ceph efficiently for your storage requirements.

Related Content you might be interested in

Current Title:

Small book image
Ceph Cookbook - Second Edition
Vikhyat Umrao | Karan Singh | Michael Hackett
Nov, 2017 | 466 pages
Small book image
Learning Ceph
Anthony DAtri | Vaibhav Bhembre | Karan Singh
Oct, 2017 | 340 pages
Small book image
Mastering Ceph,
Nick Fisk
Mar, 2019 | 356 pages
Small book image
Mastering Ceph
Nick Fisk
May, 2017 | 240 pages
Table of Contents (15 chapters)
Preface
Preface
What this book covers
What you need for this book
Who this book is for
Sections
Conventions
Reader feedback
Customer support
Free Chapter
1
Ceph – Introduction and Beyond
Ceph – Introduction and Beyond
Introduction
Ceph – the beginning of a new era
RAID – the end of an era
Ceph – the architectural overview
Planning a Ceph deployment
Setting up a virtual infrastructure
Installing and configuring Ceph
Scaling up your Ceph cluster
Using the Ceph cluster with a hands-on approach
2
Working with Ceph Block Device
Working with Ceph Block Device
Introduction
Configuring Ceph client
Creating Ceph Block Device
Mapping Ceph Block Device
Resizing Ceph RBD
Working with RBD snapshots
Working with RBD clones
Disaster recovery replication using RBD mirroring
Configuring pools for RBD mirroring with one way replication
Configuring image mirroring
Configuring two-way mirroring
Recovering from a disaster!
3
Working with Ceph and OpenStack
Working with Ceph and OpenStack
Introduction
Ceph – the best match for OpenStack
Setting up OpenStack
Configuring OpenStack as Ceph clients
Configuring Glance for Ceph backend
Configuring Cinder for Ceph backend
Configuring Nova to boot instances from Ceph RBD
Configuring Nova to attach Ceph RBD
4
Working with Ceph Object Storage
Working with Ceph Object Storage
Introduction
Understanding Ceph object storage
RADOS Gateway standard setup, installation, and configuration
Creating the radosgw user
Accessing the Ceph object storage using S3 API
Accessing the Ceph object storage using the Swift API
Integrating RADOS Gateway with OpenStack Keystone
Integrating RADOS Gateway with Hadoop S3A plugin
5
Working with Ceph Object Storage Multi-Site v2
Working with Ceph Object Storage Multi-Site v2
Introduction
Functional changes from Hammer federated configuration
RGW multi-site v2 requirement
Installing the Ceph RGW multi-site v2 environment
Configuring Ceph RGW multi-site v2
Testing user, bucket, and object sync between master and secondary sites
6
Working with the Ceph Filesystem
Working with the Ceph Filesystem
Introduction
Understanding the Ceph Filesystem and MDS
Deploying Ceph MDS
Accessing Ceph FS through kernel driver
Accessing Ceph FS through FUSE client
Exporting the Ceph Filesystem as NFS
Ceph FS – a drop-in replacement for HDFS
7
Monitoring Ceph Clusters
Monitoring Ceph Clusters
Introduction
Monitoring Ceph clusters – the classic way
Introducing Ceph Metrics and Grafana
Installing and configuring Ceph Metrics with the Grafana dashboard
Monitoring Ceph clusters with Ceph Metrics with the Grafana dashboard
8
Operating and Managing a Ceph Cluster
Operating and Managing a Ceph Cluster
Introduction
Understanding Ceph service management
Managing the cluster configuration file
Running Ceph with systemd
Scale-up versus scale-out
Scaling out your Ceph cluster
Scaling down your Ceph cluster
Replacing a failed disk in the Ceph cluster
Upgrading your Ceph cluster
Maintaining a Ceph cluster
9
Ceph under the Hood
Ceph under the Hood
Introduction
Ceph scalability and high availability
Understanding the CRUSH mechanism
CRUSH map internals
CRUSH tunables
Ceph cluster map
High availability monitors
Ceph authentication and authorization
I/O path from a Ceph client to a Ceph cluster
Ceph Placement Group
Placement Group states
Creating Ceph pools on specific OSDs
10
Production Planning and Performance Tuning for Ceph
Production Planning and Performance Tuning for Ceph
Introduction
The dynamics of capacity, performance, and cost
Choosing hardware and software components for Ceph
Ceph recommendations and performance tuning
Ceph erasure-coding
Creating an erasure-coded pool
Ceph cache tiering
Creating a pool for cache tiering
Creating a cache tier
Configuring a cache tier
Testing a cache tier
Cache tiering – possible dangers in production environments
11
The Virtual Storage Manager for Ceph
The Virtual Storage Manager for Ceph
Introductionc
Understanding the VSM architecture
Setting up the VSM environment
Getting ready for VSM
Installing VSM
Creating a Ceph cluster using VSM
Exploring the VSM dashboard
Upgrading the Ceph cluster using VSM
VSM roadmap
VSM resources
12
More on Ceph
More on Ceph
Introduction
Disk performance baseline
Baseline network performance
Ceph rados bench
RADOS load-gen
Benchmarking the Ceph Block Device
Benchmarking Ceph RBD using FIO
Ceph admin socket
Using the ceph tell command
Ceph REST API
Profiling Ceph memory
The ceph-objectstore-tool
Using ceph-medic
Deploying the experimental Ceph BlueStore
13
An Introduction to Troubleshooting Ceph
An Introduction to Troubleshooting Ceph
Introduction
Initial troubleshooting and logging
Troubleshooting network issues
Troubleshooting monitors
Troubleshooting OSDs
Troubleshooting placement groups
14
Upgrading Your Ceph Cluster from Hammer to Jewel
Upgrading Your Ceph Cluster from Hammer to Jewel
Introduction
Upgrading your Ceph cluster from Hammer to Jewel

RADOS load-gen

A bit similar to the rados bench, RADOS load-gen is another interesting tool provided by Ceph, which runs out-of-the-box. As the name suggests, the RADOS load-gen tool can be used to generate load on a Ceph cluster and can be useful to simulate high load scenarios.

How to do it...

  1. Let's try to generate some load on our Ceph cluster with the following command:
        # rados -p rbd load-gen --num-objects 50 --min-object-size 4M 
        --max-object-size 4M --max-ops 16 --min-op-len 4M --max-op-len 4M 
        --percent 5 --target-throughput 2000 --run-length 60

How it works...

...
Previous Section
End of Section 6
Next Section