Installing and starting storage nodes

From the download page at http://dev.mysql.com, all files for MySQL Cluster for RedHat and CentOS 5 can be found in the Red Hat Enterprise Linux 5 RPM section. It is recommended that the following two RPMs should be installed on each storage node:

Once these packages are downloaded, we will show in an example how to install the nodes, start the storage nodes, and check the status of the cluster.

Firstly, download the two files required on each storage node (that is, complete this on all storage nodes simultaneously):

[root@node1 ~]# cd ~/
[root@node1 ~]# mkdir mysql-storagenode
[root@node1 ~]# cd mysql-storagenode/
[root@node1 mysql-storagenode]# wget http://dev.mysql.com/get/Downloads/MySQL-Cluster-7.0/MySQL-Cluster-gpl-storage-7.0.6-0.rhel5.x86_64.rpm/from/http://a.mirror/
--21:17:04--  http://dev.mysql.com/get/Downloads/MySQL-Cluster-7.0/MySQL-Cluster-gpl-storage-7.0.6-0.rhel5.x86_64.rpm/from/http://a.mirror/
Resolving dev.mysql.com... 213.136.52.29
<snip>
21:18:06 (9.25 MB/s) - `MySQL-Cluster-gpl-storage-7.0.6-0.rhel5.x86_64.rpm' saved [4004834/4004834]

[root@node1 mysql-storagenode]# wget http://dev.mysql.com/get/Downloads/MySQL-Cluster-7.0/MySQL-Cluster-gpl-tools-7.0.6-0.rhel5.x86_64.rpm/from/http://a.mirror/
--21:19:12--  http://dev.mysql.com/get/Downloads/MySQL-Cluster-7.0/MySQL-Cluster-gpl-tools-7.0.6-0.rhel5.x86_64.rpm/from/http://a.mirror/
<snip>
21:20:14 (9.67 MB/s) - `MySQL-Cluster-gpl-tools-7.0.6-0.rhel5.x86_64.rpm' saved [9524521/9524521]

Once both the files are downloaded, install these two packages using the same command as it was used in the previous recipe:

[root@node1 mysql-storagenode]# rpm -ivh MySQL-Cluster-gpl-tools-7.0.6-0.rhel5.x86_64.rpm MySQL-Cluster-gpl-storage-7.0.6-0.rhel5.x86_64.rpm 
Preparing...                ########################################### [100%]
   1:MySQL-Cluster-gpl-stora########################################### [ 50%]
   2:MySQL-Cluster-gpl-tools########################################### [100%]

Now, using your favorite text editor, insert the following into /etc/my.cnf file, replacing 10.0.0.5:1186 with the hostname or IP address of the already installed management node:

[mysql_cluster]
ndb-connectstring=10.0.0.5:1186

Now, as we have installed the storage node client and configured /etc/my.cnf file to allow a starting storage node process to find its management node, we can start our cluster.

To join storage nodes to our cluster, following requirements must be met:

To start our storage nodes, on each node create the data directory that was configured for each storage node in the config.ini file on the management node, and run ndbd with the –-initial flag:

[root@node1 ~]# mkdir -p /var/lib/mysql-cluster
[root@node1 ~]# ndbd --initial
2009-07-06 00:31:57 [ndbd] INFO     -- Configuration fetched from '10.0.0.5:1186', generation: 1
[root@node1 ~]#

If you fail to create the data directory (/var/lib/mysql-cluster in our example in the previous recipe), you may well get a Cannot become daemon: /var/lib/mysql-cluster/ndb_3.pid: open for write failed: No such file or directory error. If you get this, run the mkdir command again on the relevant storage node

Once you have started ndbd on each node, you can run the management client, ndb_mgm, from any machine as long as it can talk to port 1186 on the management node with which it will work.

The --initial flag to ndbd tells ndbd to initialize the DataDir on the local disk, overwriting any existing data (or in this case, creating it for the first time).

You should only use --initial the first time you start a cluster or if you are deliberately discarding the local copy of data. If you do it at other times, you risk losing all the data held in the cluster, if there is no online node in the same nodegroup that stays online, long enough, to update the starting node.

The cluster will go through various stages as it starts. If you run the ALL STATUS command in the management client while the nodes are starting, you will see that they start off as unconnected, then go through the startup phases, and finally are marked as started.

Because often a node starting must apply a large number of database transactions either from other nodes or from its local disk, this process can take some time in clusters with data. Although, in the case of an initial start, this process should be relatively fast. The following output shows the management client when all the storage nodes have started:

[root@node5 ~]# ndb_mgm
-- NDB Cluster -- Management Client --

ndb_mgm> SHOW
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=3 @10.0.0.1  (mysql-5.1.34 ndb-7.0.6, Nodegroup: 0, Master)
id=4 @10.0.0.2  (mysql-5.1.34 ndb-7.0.6, Nodegroup: 0)

[ndb_mgmd(MGM)] 1 node(s)
id=1 @10.0.0.5  (mysql-5.1.34 ndb-7.0.6)

[mysqld(API)] 4 node(s)
id=11 (not connected, accepting connect from 10.2.0.2)
id=12 (not connected, accepting connect from 10.2.0.3)
id=13 (not connected, accepting connect from any host)
id=14 (not connected, accepting connect from any host)
ndb_mgm> ALL STATUS
Node 3: started (mysql-5.1.34 ndb-7.0.6)
Node 4: started (mysql-5.1.34 ndb-7.0.6)

At this point, this cluster has one management node and two storage nodes that are connected. You are now able to start the SQL nodes.

In the case you have any problems, look at the following points:

These two logfiles should give you a pretty good idea of what is causing a problem.

If one node remains in phase 1, when others are in phase 2, this likely indicates a network problem—normally, a firewall between storage nodes causes this issue. In such cases, double check that there are no software or hardware firewalls between the nodes.

For convenience (particularly, when writing scripts to manage large clusters), you may want to start the ndbd process on all storage nodes to the point that they get configuration data from the management node and are able to be controlled by it but you may not want to completely start them. This is achieved with the --nostart or -n flag:

On the storage nodes (and assuming that ndbd is not already running):

[root@node1 ~]# ndbd –n
2009-07-09 20:59:49 [ndbd] INFO     -- Configuration fetched from '10.0.0.5:1186', generation: 1

Then, on the management node you should see that the nodes are not started (this is different from not connected) as shown here:

ndb_mgm> ALL STATUS
Node 3: not started (mysql-5.1.34 ndb-7.0.6)
Node 4: not started (mysql-5.1.34 ndb-7.0.6)
Node 5: not started (mysql-5.1.34 ndb-7.0.6)
Node 6: not started (mysql-5.1.34 ndb-7.0.6)

It is then possible to start all the nodes at the same time from the management client with the command <nodeid> START as follows:

NDB_MGM> 3 START
Database node 3 is being started.
ndb_mgm> NODE 3: START INITIATED (VERSION 7.0.6)
NODE 3: DATA USAGE DECREASED TO 0%(0 32K PAGES OF TOTAL 2560)

You can start all storage nodes in a not started state with the command ALL START:

NDB_MGM> ALL START
NDB Cluster is being started.
NDB Cluster is being started.
NDB Cluster is being started.
NDB Cluster is being started.

During the start up of storage nodes, you may find the following list of phases useful, if nodes fail or hang during a certain start up phase: