Book Image

High Availability MySQL Cookbook

By : Alexander Davies
Book Image

High Availability MySQL Cookbook

By: Alexander Davies

Overview of this book

High Availability is something that all web sites hope to achieve, especially those that are linked to big companies.MySQL, an open source relational database management system (RDBMS), can be made highly available to protect from corruption, hardware failure, software crashes, and user error. Running a MySQL setup is quite simple. Things start getting complex when you start thinking about the best way to provide redundancy. There are a large number of techniques available to add 'redundancy' and 'high availability' to MySQL, but most are both poorly understood and documented.This book will provide you with recipes showing how to design, implement, and manage a MySQL Cluster and achieve high availability using MySQL replication, block level replication, shared storage, and the open source Global File System (GFS).This book covers all the major techniques available for increasing availability of your MySQL databases. It demonstrates how to design, implement, troubleshoot and manage a highly available MySQL setup using any one of several techniques, which are shown in different recipes. It is based on MySQL Cluster 7.0, MySQL (for non clustered recipes) 5.0.77, and CentOS / RedHat Enterprise Linux 5.3.The book starts by introducing MySQL Cluster as a technology and explaining how to set up a simple cluster. It will help you to master the options available for backing up and restoring a file in the MySQL Cluster. By following the practical examples in this book, you will learn how to manage the MySQL Cluster. Further, we will discuss some troubleshooting aspects of the MySQL Cluster.We also have a look at achieving high availability for MySQL databases with the techniques of MySQL Replication, block level replication, shared storage (a SAN or NAS), and DRBD.Finally, you will learn the principles of Performance tuning and tune MySQL database for optimal performance.
Table of Contents (17 chapters)
High Availability MySQL Cookbook
Credits
About the Author
About the Reviewers
Preface
Base Installation
Index

How to do it...


Installing mylvmbackup on a RedHat Enterprise Linux or CentOS 5 system is simple and it is shown in this section:

Firstly, install perl-Config-IniFiles and perl-TimeDate:

Note

perl-Config-IniFiles is available only in the EPEL repository. This was covered earlier in this book in Chapter 3, MySQL Cluster Management; you can read the simple install guide for this repository at http://fedoraproject.org/wiki/EPEL/FAQ#How_can_I_install_the_packages_from_the_EPEL_software_repository.3F.

[root@node2 ~]# yum -y install perl-Config-IniFiles perl-TimeDate
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * epel: www.mirrorservice.org

Installed: perl-Config-IniFiles.noarch 0:2.39-6.el5 perl-TimeDate.noarch 1:1.16-5.el5
Complete!

Download mylvmbackup and extract the tar.gz file as follows:

[root@node2 ~]# cd /usr/src/
[root@node2 src]# wget http://lenzg.net/mylvmbackup/mylvmbackup-0.13.tar.gz
--18:16:11--  http://lenzg.net/mylvmbackup/mylvmbackup-0.13.tar.gz
Resolving lenzg.net... 213.83.63.50
Connecting to lenzg.net|213.83.63.50|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 37121 (36K) [application/x-tar]
Saving to: `mylvmbackup-0.13.tar.gz'

100%[=======================================================================================================>] 37,121      --.-K/s   in 0.06s

18:16:11 (593 KB/s) - `mylvmbackup-0.13.tar.gz' saved [37121/37121]

[root@node2 src]# tar zxvf mylvmbackup-0.13.tar.gz

mylvmbackup-0.13/
mylvmbackup-0.13/ChangeLog
mylvmbackup-0.13/COPYING
mylvmbackup-0.13/CREDITS
mylvmbackup-0.13/hooks/
mylvmbackup-0.13/hooks/backupfailure.pm
mylvmbackup-0.13/hooks/logerr.pm
mylvmbackup-0.13/hooks/preflush.pm
mylvmbackup-0.13/INSTALL
mylvmbackup-0.13/Makefile
mylvmbackup-0.13/man/
mylvmbackup-0.13/man/mylvmbackup.pod
mylvmbackup-0.13/man/mylvmbackup.1
mylvmbackup-0.13/mylvmbackup
mylvmbackup-0.13/mylvmbackup.conf
mylvmbackup-0.13/mylvmbackup.pl.in
mylvmbackup-0.13/mylvmbackup.spec
mylvmbackup-0.13/mylvmbackup.spec.in
mylvmbackup-0.13/README
mylvmbackup-0.13/TODO

Change to the new directory and install mylvmbackup as follows:

[root@node2 src]# cd mylvmbackup-0.13
[root@node2 mylvmbackup-0.13]# make install
test -d /usr/local/bin || /usr/bin/install -d /usr/local/bin
test -d /usr/local/man/man1 || /usr/bin/install -d /usr/local/man/man1
test -d /etc || /usr/bin/install -d /etc
test -d /usr/local/share/mylvmbackup || /usr/bin/install -d /usr/local/share/mylvmbackup
/usr/bin/install -m 755 mylvmbackup /usr/local/bin
/usr/bin/install -m 644 man/mylvmbackup.1 /usr/local/man/man1/mylvmbackup.1
if test -f /etc/mylvmbackup.conf ; then /bin/mv /etc/mylvmbackup.conf /etc/mylvmbackup.conf.bak ; fi
/usr/bin/install -m 600 mylvmbackup.conf /etc
for HOOK in hooks/backupfailure.pm hooks/logerr.pm hooks/preflush.pm ; do if [ ! -f /usr/local/share/mylvmbackup/$HOOK ] ; then /usr/bin/install -m 644 -v $HOOK /usr/local/share/mylvmbackup ; fi ; done
`hooks/backupfailure.pm' -> `/usr/local/share/mylvmbackup/backupfailure.pm'
`hooks/logerr.pm' -> `/usr/local/share/mylvmbackup/logerr.pm'
`hooks/preflush.pm' -> `/usr/local/share/mylvmbackup/preflush.pm'

mylvmbackup requires two things to work: the MySQL data directory must be on a logical volume and the volume group that contains this logical volume must have some spare space to hold the snapshot copy for writing. Check these with the following commands:

Firstly, run the vgs command to confirm that there is spare space available. In this case, there is 9.66G spare space in the volume group:

[root@node1 ~]# vgs
  VG     #PV #LV #SN Attr   VSize  VFree
  system   1   3   0 wz--n- 29.41G 9.66G

Secondly, check that there is a logical volume for MySQL with the lvs command and note the volume group (system in the following example):

[root@node1 ~]# lvs
  LV    VG     Attr   LSize  Origin Snap%  Move Log Copy%  Convert
  log   system -wi-ao  1.94G
  mysql system -wi-ao 10.00G
  root  system -wi-ao  7.81G

Finally, check that the MySQL data directory is mounted on this logical volume:

[root@node1 ~]# df -h | grep mysql
/dev/mapper/system-mysql
                      9.9G  172M  9.2G   2% /var/lib/mysql

With these things verified, you are ready to configure /etc/mylvmbackup.conf file.

Start with the self-explanatory [mysql] section that gives a user the privileges to lock tables.

In the [lvm] section, define the vgname and lvname from the output of vgs / lvs commands used earlier (in our example, the vgname is system and the lvname is mysql). The parameter lvsize determines how much space to keep for the snapshot—the volume group must have that much space free, and this is used to hold the data that is changed while the snapshot is open. The correct setting for this parameter depends on how busy your server is and how long backups take; you can start with 20 percent of the logical volume size.

An example configuration file is as follows:

#
# mylvmbackup configuration file
#

[mysql]
user=USER
password=PASSWORD
host=localhost
port=
socket=
mycnf=/etc/my.cnf

#
# LVM-specific options
#
[lvm]
vgname=dataVol
lvname=mysql
backuplv=
# lvsize must be able to cope with differences
lvsize=1G

#
# File system specific options
#
[fs]
xfs=0
mountdir=/var/tmp/mylvmbackup/mnt/
backupdir=/var/tmp/mylvmbackup/backup/
relpath=

#
# Names of required external utilities
# Make sure the $PATH is set accordingly, especially for cron jobs!
#
[tools]
lvcreate=lvcreate

lvremove=lvremove
lvs=lvs
mount=mount
tar=tar
compress=gzip
rsync=rsync
rsnap=rsnap
umount=umount

#
# Other configuration options
#

[misc]
backuptype=tar
prefix=backup-ibactive
suffix=_mysql
tararg=cvf
tarsuffixarg=
tarfilesuffix=.tar.gz
compressarg=--stdout --verbose --best
rsyncarg=-avWP 
rsnaparg=7
datefmt=%Y%m%d_%H%M%S
innodb_recover=0
pidfile=/var/tmp/mylvmbackup_recoverserver.pid
skip_flush_tables=0
extra_flush_tables=0

skip_mycnf=0
hooksdir=/usr/share/mylvmbackup 
skip_hooks=0
keep_snapshot=0
keep_mount=0
quiet=0

#
# Logging options. The Sys::Syslog module is required for syslog option
# See "perldoc Sys::Syslog" for more information.
#
[logging]
# 'console' (STDOUT, STDERR) or 'syslog' or 'both'.
log_method=console
# 'native', 'tcp', 'udp'. Default is 'native'
syslog_socktype=native
syslog_facility=
# If using remote syslog, don't forget to change the socket type to tcp or udp.
syslog_remotehost=

With the configuration file created, execute mylvmbackup to test your backup as follows:

[root@node1 ~]# mylvmbackup
20091206 18:39:24 Info: Connecting to database...
20091206 18:39:24 Info: Flushing tables with read lock...
20091206 18:39:24 Info: Taking position record into /tmp/mylvmbackup-backup-20091206_183924_mysql-mxAD4A.pos...
20091206 18:39:24 Info: Running: lvcreate -s --size=5G --name=mysql_snapshot /dev/system/mysql
File descriptor 4 (socket:[17769919]) leaked on lvcreate invocation. Parent PID 20180: /usr/bin/perl
  Logical volume "mysql_snapshot" created
20091206 18:39:25 Info: DONE: taking LVM snapshot
20091206 18:39:25 Info: Unlocking tables...
20091206 18:39:25 Info: Disconnecting from database...
20091206 18:39:25 Info: Mounting snapshot...

20091206 18:39:25 Info: Running: lvremove -f /dev/system/mysql_snapshot
  Logical volume "mysql_snapshot" successfully removed
20091206 18:39:26 Info: DONE: Removing snapshot
[root@node1 ~]# echo $?
0

In this case, the tables were locked for less than a second.

Various parts of this procedure can fail for the following reasons:

  • The MySQL server can be down

  • The read lock cannot be successfully acquired

  • Insufficient spare space in the volume group to create the snapshot

  • The volume group or logical volume name does not exist (that is, a typo in the configuration file)

It is therefore essential in automated scripts to check the exit code ($?) and ensure that it is 0—and if it is not, you must schedule a manual look at the backup job.

If the script exits with an exit code of 0, the backup can be found in /var/tmp/mylvmbackup/backup/, which is ready for sending to a backup location. For example, you could scp or rsync it to an offsite location.