IONET is a Wellington, New Zealand-based company that has specialized in Lotus Notes and Domino solutions since v3.0. They concentrate on innovative, low-cost products to enhance the usability of any Lotus Notes/Domino environment.
For a demonstration copy of the Incremental Archiver, or more information on IONET or our products, please visit http://www.ionetsoftware.com.
The IONET Incremental Archiver complements R8 as an automatic archive/restore tool for Lotus Notes data. In the Archiver, we use standard Lotus Notes technologies, but in an innovative way, which is the focus of this section.
Briefly, the Archiver automatically installs, archives, allows users to restore document versions and deletions themselves, and removes production data (including mail). It does this via a combination of Notes replication and a host of other functions that may be useful for any Notes/Domino environment. Coupled with the usability of R8, this makes a powerful platform for your archiving requirements.
The Archiver has been designed to allow users to restore their own data from any date (including document versions), using the source Notes application they are familiar with, be it mail or any other Notes database. This means that the hierarchy of the data (for example, responses) is preserved and therefore familiar to the user. It also means that the IT department no longer has to juggle backup tapes to restore user data. The user can also search Archives, providing a "back through time" look at document versions in the Archive.
All facets of the archiving process are automatically enabled and controlled by the Administrator, Notes security is observed, and full backups can still be taken for Disaster Recovery. In addition, Notes databases no longer need custom archiving solutions, as data can be simply deleted from the database and remains in the Archive, including document versions. That means you can safely use the "Remove documents not modified in the last x days" setting to control database size (the Archiver can also remove documents for you according to combinations of dates and @formulas). For the full product description, features, and downloads, please visit http://www.ionetsoftware.com/archiving.
Let's have a look at the main components in more detail. They are:
To make it easier to administer, the Archiver installation is automated as much as possible. The setup process below is followed for each database included in the Archive:
Add three design elements (an Agent, View, and Form) to the production database design. This is performed using the excellent DBDesign LotusScript class from Damian Katz (IRIS). This class treats Notes Design elements as Notes Documents, thus allowing them to be copied between databases using the
CopyToDatabase
method of theNotesDocument
class. This library is available via the Sandbox on the Lotus Developers Domain.Create a replica on the Archive Server. This is performed using the
CreateReplica
method.Set the Replica to not replicate deletions, via the
NotesReplication
LotusScript class.Set the Replica to only replicate documents matching a user-defined @formula, using the
NotesReplicationEntry
LotusScript class.Create a copy of the newly created replica on the Archive Server (this copy contains document versions). This is performed using the
CreateCopy
method of theNotesDatabase
class.Check there is no scheduled replication (by trawling directory documents), and that the Replica cannot write back to the Production database (via NotesACL LotusScript methods). These checks are also performed before each archiving operation to ensure the integrity of data.
Create Full Text Indices of the newly created databases on the Archive Server. This function is not directly possible via LotusScript. However, it is possible to create a separate LotusScript agent (called BuildIndex) that opens a database locally (using
"
as the server argument) and creates an index using theCreateFTIndex
method of theNotesDatabase
class. Using a second agent, you then instantiate this agent and call theRunOnServer
method, passing a NotesDocument containing the database information. The result is to create a Full Text Index on the server. This is shown below.
Sub Initialize Dim s As New NotesSession Dim db As NotesDatabase Dim targetDB As New NotesDatabase("", "") Dim agent As NotesAgent Dim doc As NotesDocument Set db = s.CurrentDatabase Set agent = s.CurrentAgent Set doc = db.GetDocumentByID(agent.ParameterDocID) If Not (doc Is Nothing) Then replicaID$ = doc.DBReplicaID(0) If targetDB.OpenByReplicaID("", replicaID$) Then Call targetDB.CreateFTIndex(23, False) End If End If End Sub
During setup, we created a replica on the Archive Server (the Archive Replica) and a copy of that replica (the Archive Store), for each production database.
A scheduled agent in the Archiver checks each Production database for any changes made to eligible documents (those matching the Archive @formula for the database) since the last time replication occurred, using the Search
method of the NotesDatabase
class. The Search
method allows reasonable search performance using time/date criteria within databases that may or may not be Full Text Indexed.
For any document that has been modified, the corresponding document is located in the Archive Replica (where it contains the previous content). This document is copied to the Archive Store database. Because we're copying data on the same file system and not traversing a network, this is a reasonably fast process.
During the copy process, the user has the choice to automatically zip attachments. This is done by calling a Java agent that uses the java.util.zip
package. Briefly, the steps are:
Make an array of all eligible attachments in the document and write this to the document.
Extract the attachments to disk and remove them from the document.
Call the Java agent, again using the
NotesAgent
classRunOnServer
method and passing the documentNoteID
. The Java agent locates the files on disk according to information in the document, zips them, then writes a flag once finished. The initial agent sees the flag and reattaches the ZIP files to the document.
Set doc = ... Set the document object containing information on the files to ZIP Call session.SetEnvironmentVar("ZipAgentStatus","") Set agent = db.GetAgent("(ZipFiles)") Call agent.RunOnServer(doc.NoteID) chkFinished$ = session.GetEnvironmentString("ZipAgentStatus") While chkFinished$ = "" chkFinished$ = session.GetEnvironmentString("ZipAgentStatus") Sleep 1 Wend
import lotus.domino.*; import java.io.*; import java.util.*; import java.util.zip.*; import java.text.*; public class JavaAgent extends AgentBase { private ZipInputStream inZipFile; private String aDBServer; private String aDBFileName; private String aDBNoteID; private String zipFileNameInput; private String zipFileNameOutput; public void NotesMain() { try { Session session = getSession(); //Instantiate NotesSession AgentContext agentContext = session.getAgentContext(); //Instantiate AgentContext Database db = agentContext.getCurrentDatabase(); //Instantiate CurrentDatabase Agent agent = agentContext.getCurrentAgent(); //Instantiate CurrentAgent Document callingDoc = db.getDocumentByID(agent.getParameterDocID()); //Get the doc calling this agent String aDBServer = callingDoc.getItemValueString("IOZipDBServer"); //Get the server of the target database String aDBFileName = callingDoc.getItemValueString("IOZipDBFileName"); //Get the filename of the targetdatabase String aDBNoteID = callingDoc.getItemValueString("IOZipNoteID"); //Get the Note ID of the target document Database aDB = session.getDatabase(aDBServer, aDBFileName);//Open the target database Document doc = aDB.getDocumentByID(aDBNoteID); //Get the target document Vector zipFileInput = doc.getItemValue("IOZipInput"); //Source uncompressed files Vector zipFileOutput = doc.getItemValue("IOZipOutput"); //Target compressed files byte[] buffer = new byte[18024]; for (int i = 0; i < zipFileInput.size(); i++) { // Associate a file input stream for the current file String zipFileNameOutput = (String)zipFileOutput.elementAt(i); ZipOutputStream out = new ZipOutputStream(new FileOutputStream(zipFileNameOutput)); out.setLevel(Deflater.DEFAULT_COMPRESSION); String zipFileNameInput = (String)zipFileInput.elementAt(i); FileInputStream in = new FileInputStream(zipFileNameInput); // Add ZIP entry to output stream. out.putNextEntry(new ZipEntry(zipFileNameInput)); int len; while ((len = in.read(buffer)) > 0) { out.write(buffer, 0, len); } // Close the current entry out.closeEntry(); // Close the current file input stream in.close(); out.close(); } session.setEnvironmentVar("ZipAgentStatus", "Completed", false); session.recycle(); agentContext.recycle(); db.recycle(); agent.recycle(); doc.recycle(); } catch (IllegalArgumentException iae) { iae.printStackTrace(); } catch (FileNotFoundException fnfe) { fnfe.printStackTrace(); } catch (IOException ioe) { ioe.printStackTrace(); } catch(Exception e) { e.printStackTrace(); } } }
When all eligible documents have been zipped and copied, the agent initiates normal Notes replication between the Production database and Archive Replica to update the Archive Replica. It does this via the Replicate
method of the NotesDatabase
class.
So by identifying modified documents before they are updated via replication, an accurate versioning of the documents is achieved.
Because deletions are not replicated to the Archive Replica, all documents are permanently retained. Replication and ACL settings are verified before every Archive procedure to ensure deletions are not replicated back into Production when deletion stubs have been purged.
The final part of the Archive process is to remove documents from the Production database, if they match the criteria specified for the database, using a combination of document age, last accessed date, last modified date, and @formulas.
During the setup process, a "Dialogbox Form" and a "Restore Document" agent are automatically added to the Production database. When a user wants to restore a document, they run this agent. They are prompted for the date and it is retrieved from either the Archive Replica or Archive Store (depending on the required date), and opened.
The following screenshot shows the Restore operation. Note the handy use of the R8 function to modify the right-hand menu.
The user then has to select the type of Restore:
If the user wants to restore a deleted document, they instead automatically open the Archive Replica (where deletions are not removed), locate the document they want, set the date, and repeat the restore process, retrieving the document from either the Archive Replica or Archive Store depending on the required date. Users can then restore deleted documents to the Production database.
The user is also able to search the Archive Replica and Archive Store databases for the data they want.
This allows a "back through time" search facility that includes the date archived. This functionality is provided via our FT Search Manager, which allows simultaneous searching of multiple Full Text Indices.
Housekeeping agents optionally clear out Archived data after a configurable period.
A separate database option allows the restore process to use a separate directory, so that if a restore of data from five years ago is required, the Archive Replica and Archive Store tape backups can be restored into this directory and the process works from there instead of the normal location. This is so that "normal" full backups to tape can be performed if required (for example, for Disaster Recovery, reducing disk space usage, and so on)
Another benefit of the Archiver is that no other form of archiving is required (no custom data archives need be done by the IT department), documents can be simply deleted from the Production database and they will remain available in the Archive Replica for restore.