Sometimes it is necessary to rebuild a backend database. This process differs depending on the database backend. For instance, with a SQL backend, it might entail dumping, dropping, and re-creating tables in the database.
Note
Moving to a new server and transferring contents to a new slave server are also processes similar to rebuilding a database, and the differences are mentioned within the text here.
The most commonly-used backends for OpenLDAP are the HDB and BDB backends (both based on the Berkeley DB lightweight database). In this section, I want to cover the process of rebuilding these databases.
This process consists of five steps:
Stop SLAPD
Dump the directory data into a file
Delete the old directory files
Create a new database
Start SLAPD
None of these steps is particularly difficult. In fact, for a small to medium-sized directory, this process can be done in less than ten minutes.
Note
Moving from Server to Server
Moving a directory from one server to another is done by a process very similar to that described here. Only step three, as mentioned later, differs. In this case, instead of deleting directory files, the LDIF file would be transferred from the original server to the new server. Steps one and two would be run on the original server, and steps four and five would be done on the new server.
The purpose of stopping the server is to prevent additional changes to the directory information tree while we are working on it.
Note
If you are just dumping the contents of a master directory to import into a shadow server that will use SyncRepl, you need not stop the server. Any changes that happen after the directory has been dumped will be retrieved by the shadow server during its first LDAP synchronization operation.
This can be done either by killing the server's process ID, or by running the startup script with the stop command:
$ sudo invoke-rc.d slapd stop
Now that the server is stopped, we can dump the database.
In Chapter 3 I covered the OpenLDAP utilities. One of the tools I discussed was the slapcat
program, which is a tool for dumping the contents of the directory into an LDIF file. That is the program we will use in this step.
Why use slapcat
instead of an ldapsearch
? There are two reasons.
First, slapcat
preserves all of the attributes (and records for that matter) that the LDAP server uses, including the operational attributes that are stored. (Those operational attributes that are generated at runtime are not generated by slapcat
, and that is good. We wouldn't want to import those, anyway.)
Second, slapcat
accesses the database directly, instead of opening an LDAP connection to the server. That means that ACLs, time and size limits, and other by products of the LDAP connection are not evaluated, and hence will not alter the data.
The BDB/HDB database is stored in a small set of files located at /var/lib/ldap
(or /usr/local/var/openldap-data
if you built from source). Usually access to those files is restricted to only the ID of the SLAPD user. By default this is root
or ldap
. In order to extract information using slapcat
, you will need to have access to those files.
We have this command:
$ sudo slapcat -l /tmp/backup.ldif
This command executes slapcat
as root. The -l
flag is used to pass in the name of the output file. In this case the file backup.ldif
will be created in the /tmp
directory.
Note
You may prefer putting the LDIF file in a folder other than /tmp
, especially if you plan on keeping the LDIF file for more than a few minutes.
In most cases the -l
flag is the only one you will need. If you have more than one backend and you only want to dump one, you can use the -n
flag to specify which backend to dump.
Once the slapcat
is complete, we are done with this step.
Before continuing however, you may want to check the contents of the LDIF file to make sure that it is not corrupt. Do this before deleting the database files.
If you are re-building a database you will want to delete the old database files before building new ones.
Note
You do not need to do this if you are either migrating from an old server to a new server or configuring SyncRepl shadow servers.
These files are stored at /var/lib/ldap
(or /usr/local/var/openldap-data
if you built from source). However, not all of the files in that directory should be deleted. We only want to delete:
The index files: files that end in '
.bdb
'.The main database files: files named
__db.???
, where the question marks are replaced by numbers in sequence (__db.001
,__db.002
, and so on).The
alock
file: a file used internally for storing locking information. (Usually, this can be left with no negative consequences, but if SLAPD crashed, this can be left in an unstable state.)The BDB log files: files named
log.??????????
, where the ten question marks are replaced by numbers in sequence:log.0000000001
,log.0000000002
, and so on.
There is one file we definitely do not want to delete. This is our database configuration file, DB_CONFIG
. Deleting it would cause the BDB engine to use its default settings, which are not tuned to our needs, and generally cause OpenLDAP to perform poorly.
So, to delete the files, we can do the following:
$ cd /var/lib/ldap $ sudo rm __db.* *.bdb alock log.*
To reduce the risk of data loss, you may want to backup the __db.*
, *.bdb
, and log.*
files before removing them. Or instead of doing an rm
, you may use mv
to move the files to a different location:
$ cd /var/lib/ldap $ sudo mkdir backup/ $ sudo mv *.bdb log.* alock __db.* backup/
Now the database directory has been cleared. We are ready to create new database files.
The new database can be created and populated with the data all in one step, using the slapadd
utility that we covered in Chapter 3. Still in the OpenLDAP data directory, run the following command:
$ sudo slapadd -l /tmp/backup.ldif
This will create all of the necessary files, import the LDIF file, and handle all of the data indexing as well.
Note
If you are running your LDAP server as a user other than root (and it is a good idea to do so), you will also need to use chown
to change the ownership on all of the files at /var/lib/ldap
to be owned by the SLAPD userID: sudo
chown
openldap
*.bdb
log.*
__db.*
.
All we need to do now is restart the server.
If you stopped the server in step 1 you will need to restart it.
Restart the server in one of the usual ways. Using the init script is usually the best way:
$ sudo invoke-rc.d slapd start
That's all there is to it. Now you should have SLAPD running with a fresh copy of the database.
As long as the LDIF file exported with slapcat
is good, there is not much that can go wrong in this process. Even if you have to delete and recreate several times, as long as the LDIF file is safe, no important data is at risk.
If SLAPD is running as a user other than root
, the main problem with importing is usually the permissions on the database files at /var/lib/ldap
. Permissions on the configuration files in /etc/ldap
directory may also be the source of SLAPD failures. Make sure they are owned by the appropriate user.
When switching versions of OpenLDAP, occasionally an old LDIF file will not be valid in the new server (this happened between OpenLDAP 2.0 and OpenLDAP 2.2, and again between 2.2 and 2.3; it could happen again in the future). While the standard schemas are fairly stable over time, operational attributes, which are not usually standardized, are more volatile, and do change from release to release.
Often, the fix will be tweaking records in the LDIF file to match the attributes used in new version. One other common issue has to do with starting up the server. Sometimes, when using the init script, you will not be able to get the server to start, but no informative message will be sent to the console or the log files. (One common reason for the failure to start is the permissions issue I noted earlier).
A good first step in solving startup problems is to run slapd
from the command line, with debugging enabled: sudo slapd -d trace
.