Book Image

IBM WebSphere eXtreme Scale 6

By : Anthony Chaves
Book Image

IBM WebSphere eXtreme Scale 6

By: Anthony Chaves

Overview of this book

A data grid is a means of combining computing resources. Data grids provide a way to distribute object storage and add capacity on demand in the form of CPU, memory, and network resources from additional servers. All three resource types play an important role in how fast data can be processed, and how much data can be processed at once. WebSphere eXtreme Scale provides a solution to scalability issues through caching and grid technology. Working with a data grid requires new approaches to writing highly scalable software; this book covers both the practical eXtreme Scale libraries and design patterns that will help you build scalable software. Starting with a blank slate, this book assumes you don't have experience with IBM WebSphere eXtreme Scale. It is a tutorial-style guide detailing the installation of WebSphere eXtreme Scale right through to using the developer libraries. It covers installation and configuration, and discusses the reasons why a data grid is a viable middleware layer. It also covers many different ways of interacting with objects in eXtreme Scale. It will also show you how to use eXtreme Scale in new projects, and integrate it with relational databases and existing applications. This book covers the ObjectMap, Entity, and Query APIs for interacting with objects in the grid. It shows client/server configurations and interactions, as well as the powerful DataGrid API. DataGrid allows us to send code into the grid, which can be run where the data lives. Equally important are the design patterns that go alongside using a data grid. This book covers the major concepts you need to know that prevent your client application from becoming a performance bottleneck. By the end of the book, you'll be able to write software using the eXtreme Scale APIs, and take advantage of a linearly scalable middleware layer.
Table of Contents (15 chapters)
IBM WebSphere eXtreme Scale 6
Credits
About the Author
About the Reviewers
Preface

Hello, world!


An object cache stores objects as key/value pairs. The first thing we should do is define a class that we want to store in the cache. Let's store credit card payments for now:

public class Payment implements Serializable {
private int id;
private String cardNumber;
private BigDecimal amount;
private long version = 0L;
// getters and setters omitted for brevity...
}

The Payment class is a simple POJO with getters and setters. As it may be used in a distributed eXtreme Scale deployment, it implements Serializable. That's it! All we need to use a class with eXtreme Scale's object cache is for it to implement Serializable.

Objects of type Payment will be the value part of the key/value pair when we store them in the BackingMap. The key should also implement Serializable if it is a class. The key, if it is not a primitive type, should also implement reasonable equals(Object obj) and hashCode() methods.

Now that we know what we will store in the cache, let's see what it takes to actually store it. In order to put objects in a BackingMap, we need the instance of com.ibm.websphere.objectgrid.ObjectMap. We don't interact directly with objects in a BackingMap. Instead, we do it by using a proxy ObjectMap.

We obtain a reference to an ObjectMap from the com.ibm.websphere.objectgrid.Session#getMap(String mapName) method. A Session lets us perform operations, like GET and PUT operations, in the context of a transaction.

A Session object is returned from the com.ibm.websphere.objectgrid.ObjectGrid#getSession() method. We get an instance of ObjectGrid from the ObjectGridManager#createObjectGrid() method.

public class PaymentLoader {
ObjectGrid grid;
private void initialize() throws ObjectGridException {
ObjectGridManager ogm = ObjectGridManagerFactory.getObjectGridManager();
grid = ogm.createObjectGrid();
BackingMap bm = grid.defineMap("Payment");
}
// other methods omitted for now...
}

The PaymentLoader class has an instance variable grid which holds a reference to our ObjectGrid instance (seen above). The initialize() method sets up the ObjectGrid instance and defines one BackingMap used within the grid.

Let's take it one step at a time and walk through what we did. We want to interact with an ObjectGrid instance. The ObjectGrid interface is the gateway to interacting with WebSphere eXtreme Scale. It allows us to define BackingMaps and create Sessions. ObjectGrid instances are created with the ObjectGridManager interface. We get an ObjectGridManager reference by calling the helper class ObjectGridManagerFactory.getObjectGridManager(). ObjectGridManager is a singleton which provides access to methods which create local ObjectGrid instances, or connect to remote ObjectGrid instances. For now, we call the createObjectGrid() method on the ObjectGridManager. It returns an ObjectGrid instance which is exactly what we were looking for. There are several createObjectGrid methods on the ObjectGridManager that take varying arguments for naming grids and configure them through XML files. Right now this is unnecessary, though we will eventually need to use them. For now, createObjectGrid() meets our needs.

The ObjectGrid.createObjectGrid() method creates a local instance of an ObjectGrid. This means the grid lives inside the application process along with our business logic. At this point, the grid is API equivalent to any WebSphere eXtreme Scale topology we create. No matter how interesting our deployment becomes with partitions, shards, and catalog servers, we always use the same APIs to interact with it.

After creating the grid, we must define maps within it (as seen above). We store our application data inside the maps defined in the grid. Creating a map to store Payment objects is done by calling grid.defineMap("Payment"). This method creates and returns a BackingMap which lives in the grid and holds our Payments. If we were to store different classes in the grid, then we would call the grid.defineMap(String mapName) method for each one. We aren't limited to one BackingMap per class though. If we were to split up our Payment by card type, then our maps would be defined by:

BackingMap bmapVisa = grid.defineMap("VisaPayments"); BackingMap bmapMC = grid.defineMap("MasterCardPayments"); BackingMap bmapAmEx = grid.defineMap("AmExPayments");

Defining a BackingMap gives it a place to live inside the ObjectGrid instance. ObjectGrid instances manage more than one BackingMap. Creating the previous BackingMaps would give us a runtime grid that looked like this:

All BackingMaps must be defined before the call to grid.initialize(). An explicit call to initialize() is optional since the grid.getSession() method calls it if it has not been called by our application. A call to grid.defineMap(String mapName) will throw an IllegalStateException if the initialize() method has already been called. In the sample code, we rely on the implicit call to grid.initialize(), rather than explicitly calling it. This approach is acceptable as long as all BackingMaps are defined before the first call to grid.getSession().

The most important thing to remember about BackingMaps is that they contain objects which live inside the ObjectGrid instance. Once an object is outside of the grid instance, we are no longer dealing with the BackingMap. We never directly interact with an object while it is inside a BackingMap. So how do we interact with objects in the grid?

Any interaction with objects in the gird must be done through an instance of ObjectMap. An ObjectMap is the application-side representation of the objects in a particular BackingMap. ObjectMap instances are obtained through the getMap(String mapName) method in the Session interface:

Session session = grid.getSession();
ObjectMap instancesobtaining, (String mapName) method usedObjectMap paymentsMap = session.getMap("Payments");

Sessions are used to gain access to ObjectMaps backed by BackingMaps that live inside an ObjectGrid instance. The ObjectMap instance can be thought of as a "near cache". Objects copied from remote ObjectGrid instances live inside the near cache when a "get" operation is performed. Objects in the near cache are synchronized to main ObjectGrid cache when a transaction commits. The following diagram should make the relationship clearer:

The previous diagram shows the BackingMap named Payment defined in our application. This BackingMap exists inside the ObjectGrid instance, and we cannot directly add, remove, or modify the objects inside it. Our sample code calls the grid.getSession() method (which actually creates the Payment BackingMap with an implicit call to grid.initialize()). The Session interface is used to create the ObjectMap object that lets us interact with the objects in the BackingMap named Payment with the call to session.getMap("Payment"). session.getMap("Payment") throws UndefinedMapException if it is passed the name of a BackingMap that does not exist.

Now that we have an ObjectMap, we can start adding objects to the cache. Interacting with an instance of ObjectMap is similar to interacting with java.util.Maps. ObjectMap contains methods to put objects in, and get objects out. The two simplest methods to use in this interface are put(Object key, Object value) and get(Object key). While ObjectMap contains other methods to put and get objects in bulk, we'll use these two methods in our example:

private void persistPayment(Payment payment) throws ObjectGridException {
Session session = grid.getSession();
ObjectMap paymentMap = session.getMap("Payment");
session.begin();
paymentMap.put(payment.getId(), payment);
session.commit();
}

The persistPayment(Payment payment) method uses the instance variable grid to get a Session. The Session instance can get a reference to the ObjectMap used to interact with the Payment BackingMap. We call the Session#getMap(String mapName) to get a reference to an ObjectMap. Once we have an ObjectMap, we can interact with it using GET and PUT methods.

When we want to put or get objects from an ObjectMap, we must do so under the context of a transaction. A transaction in WebSphere eXtreme Scale is similar in concept to a database transaction. The Session interface is responsible for transaction management, with explicit calls session.begin() and session.commit() or session.rollback() to start and end transactions. If a put or get on an ObjectGrid does not take place under an explicitly created transaction, then a transaction will begin and commit implicitly. Though implicit transactions may be usable for occasional one-off reads or writes, it is considered poor form to use them, and you are encouraged to call session.begin() and session.commit() in order to utilize transactions better and improve access to your BackingMaps:

session.begin();
paymentMap.put(payment.getId(), payment);
session.commit();

Starting a transaction alerts the grid that we are about to read from, or write to an ObjectMap. In this case, we are simply putting a Payment into the ObjectMap named Payment. Right now, that object only exists in our application context. The ObjectGrid instance does not know about it yet. The call to session.commit() signals that we are finished with our actions, and any changes made to any ObjectMap inside the transaction may be safely written out to their BackingMap inside the grid:

Eventually, we're going to get data out of the grid. In the event that we already have a reference to an ObjectMap, we can begin a new transaction in the Session and read from the ObjectMap using the get(Object) method. Our example shows what we need to do in the event we do not have a reference to an ObjectGrid on hand:

private Payment findPayment(int id) throws ObjectGridException {
ObjectGrid instanceget(Object) methodSession session = grid.getSession();
ObjectMap paymentMap = session.getMap("Payment");
session.begin();
Payment payment = (Payment)paymentMap.get(id);
session.rollback(); return payment;
}

The findPayment(int id) method shows how to get a Payment out of the Payment BackingMap. Just like the persistPayment(Payment payment) method, findPayment(int id) obtains a reference to a Session and an ObjectMap for Payments. The ObjectMap#get(Object key) method returns an Object with the key ID. If that key does not exist in the map, then the get method returns null. We cast the Object returned from the get method into a Payment and return it after rolling back the transaction. We roll back because we did not change the payment object at all, and we only read from the map.

When we're done using the grid, we should make sure we don't leave the grid open to accidental operations. We call the tearDown() method to make sure our reference to ObjectGrid doesn't work anymore:

private void tearDown() {
ObjectGrid instancetearDown() methodgrid.destroy();
}

Finally, grid.destroy() frees any resources used by the grid. Any attempt to get a Session, or begin a transaction after calling grid.destroy(), results in a thrown IllegalStateException.

For completeness, here is the entire PaymentLoader class.package wxs.sample:

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.math.BigDecimal;
import com.ibm.websphere.objectgrid.BackingMap;
import com.ibm.websphere.objectgrid.ObjectGrid;
import com.ibm.websphere.objectgrid.ObjectGridException;
import com.ibm.websphere.objectgrid.ObjectGridManager;
import com.ibm.websphere.objectgrid.ObjectGridManagerFactory;
import com.ibm.websphere.objectgrid.ObjectMap;
import com.ibm.websphere.objectgrid.Session;
public class PaymentLoader {
ObjectGrid grid;
static int pId = 0;
private void initialize() throws ObjectGridException {
ObjectGridManager ogm = ObjectGridManagerFactory.getObjectGridManager();
grid = ogm.createObjectGrid();
BackingMap bm = grid.defineMap("Payment");
}
public static void main(String[] args) {
PaymentLoader pl = new PaymentLoader();
try {
pl.initialize();
pl.loadPayments(args[0]);
pl.tearDown();
} catch (ObjectGridException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
System.out.println("All done!");
}
private void loadPayments(String filename) throws IOException, ObjectGridException {
BufferedReader br = null;
try {
br = new BufferedReader(new FileReader(filename));
String line;
while ((line = br.readLine()) != null) {
Payment payment = createPayment(line);
persistPayment(payment);
findPayment(payment.getId());
}
} finally {
if (br != null) {
br.close();
}
}
}
private Payment findPayment(int id) throws ObjectGridException {
Session session = grid.getSession();
ObjectMap paymentMap = session.getMap("Payment");
session.begin();
Payment payment = (Payment)paymentMap.get(id);
session.rollback();
return payment;
}
private void persistPayment(Payment payment) throws ObjectGridException {
Session session = grid.getSession();
ObjectMap paymentMap = session.getMap("Payment");
session.begin();
paymentMap.put(payment.getId(), payment);
session.commit();
}
private Payment createPayment(String line) {
String[] tokens = line.split(":");
Payment payment = new Payment();
payment.setId(pId++);
payment.setCardNumber(tokens[0]);
payment.setAmount(new BigDecimal(tokens[4]));
return payment;
}
private void tearDown() {
grid.destroy();
}
}

So far, we have created the ObjectGrid instance and BackingMaps in our application code. This isn't always the best way to do it, and it adds clutter to our code. Only local ObjectGrid instances are configurable through the programmatic interface. If we were to continue creating grids like this, we would not be able to take advantage of many of the features that make WebSphere eXtreme Scale so powerful. Instead of the programmatic configuration, we can use XML configuration files to keep information about our ObjectGrid deployment, and then load it when our application runs. This will eventually allow us to build the linearly scalable grids which we have discussed in this chapter. Let's take a look at what the XML configuration for our sample program looks like:

<?xml version="1.0" encoding="UTF-8"?>
<objectGridConfig xmlns:xsi="http://www.w3.org/2001/ XMLSchema-instance"xsi:schemaLocation="http://ibm.com/ws/ objectgrid/config../objectGrid.xsd"xmlns="http://ibm.com/ws/
objectgrid/config">
<objectGrids>
<objectGrid name="MyGrid">
<backingMap name="Payment"/>
</objectGrid>
</objectGrids>
</objectGridConfig>

This is about as simple an XML configuration file we can get. The<objectGridConfig> tag encompasses everything else in the file. In this file, we can define multiple ObjectGrid instances in the<objectGrids>. However, in this file we define just one. ObjectGrids defined in an XML configuration file, must have a name set with the name attribute. When we load the ObjectGrid from the config file in our code, we must provide a name to the createObjectGrid(...) method so that it can return the correct grid instance. This is a departure from using anonymous ObjectGrids, which is what we have done in the rest of this chapter. Our Payment BackingMap is defined in a tag nested under the ObjectGrid instance to which it belongs. Defining an ObjectGrid instance in an XML configuration file changes the way we obtain a reference to it in our code:

private void initialize() throws ObjectGridException, MalformedURLException {
ObjectGridManager ogm = ObjectGridManagerFactory.getObjectGridManager();
String filename = "c:/objectgrid.xml";
URL configFile = new URL("file://" + filename);
grid = ogm.createObjectGrid("MyGrid",configFile);
}

The main difference here is that the createObjectGrid(...) method has changed from giving us an anonymous ObjectGrid instance, to requesting the ObjectGrid instance named MyGrid that is defined in the XML file. Notice how the grid.defineMap("Payment") call disappears. We have already defined the Payment BackingMap for the new ObjectGrid instance in the XML file. Once we have a reference to the ObjectGrid instance, we have everything we need to get to work.