Today's enterprise is not confined to the physical boundaries of an organization. Open systems and an open IT infrastructure strives to provide interoperability not only between platforms, runtimes, and languages, but also across enterprises. When our concerns shift from networked systems to networked enterprises, a whole lot of opportunities open up to interact with enterprise applications. Whether it is for trading partners to collaborate through their back-end systems, or for multichannel deployments where consumers can use a whole lot of user agents like web and mobile handsets, the opportunities are endless. This also introduces the issues and concerns to be addressed by network, integration, and application architects. Today, companies that have been built through mergers or rapid expansions have Line of Businesses (LOB) and systems within a single enterprise that were not intended to interact together. More often than not these interactions fail and are discredited.
Let's begin with a quick tour of enterprise integration and the associated issues so that we can better understand the problem which we are trying to solve, rather than follow a solution for an unknown problem. At the end of this chapter, you should be in a position to identify what we are trying to aim with this book. We also introduce Enterprise Service Bus (ESB) architecture, and compare and contrast it with other integration architectures. Then we can better understand how Java Business Integration (JBI) helps us to define ESB-based solutions for integration problems.
In this chapter we will cover the following:
Problems faced by today's enterprises
Enterprise Application Integration (EAI)
Different integration architectures
ESB
Compare and contrast service bus and message bus
Need for an ESB-based architecture
Jack Welch, of General Electric (GE), invented the boundary-less organization. Meaning that, organizations should not only be boundary-less, but should also be made permeable. "Integrated information" and "integrated access to integrated information" are two key features that any enterprise should aim for, in order to improve their organizational business processes. Boundary-less doesn't mean that organization has no boundaries; it means that there's a smooth and efficient flow of information across boundaries. While enterprise portals and web services give a new face for this emerging paradigm, the skeleton of this "open enterprise" is provided by an open IT infrastructure. The flesh is provided by emerging trends in Enterprise Application Integration (EAI) practice.
Let us briefly visit a few selected issues faced by today's enterprises.
In a home business or in small scale ventures, we start up with systems and applications in silos which cater to the entire setup. When the business grows, we add more verticals, which are functionally separated entities within the same organization. It goes without saying that, each of these verticals or LOB will have their own systems. People ask why they have different systems. The answer is because they are doing functionally different operations in an enterprise and hence they need systems carrying out different functionality for them. For example, an HR system will manage and maintain employee related data whereas the marketing relations will use Customer Relationship Management (CRM) systems. These systems may not be interconnected and hence impede information flow across LOBs. Adding to this, each LOB will have their own budgeting and cost constraints which determine how often systems are upgraded or introduced.
Multiple LOBs will have their own systems, and hence their own data schemas, and a way of accessing the data. This leads to duplication of data, which in turn leads to multiple services providing views for the same entity in different ways.
Let's consider the example shown in the following figure. There is one LOB system, Product Information System, which will keep track of customers who have registered their interest for a particular product or service. Another LOB system, Event Registration System, will provide membership management tools to customers, for a sales discount program. Sales Information System will have to track all customers who have registered their interest for any upcoming products or services. Thus, we can see there is no unified view of the Customer entity at the enterprise-level. Each LOB will have its own systems and their own view of Customer. Some will have more attributes for the Customer, while others a few. Now the question is which system owns the Customer entity? Rather, how do we address the data stewardship issue(This is represented in the figure using the symbols "?" and "?") ?
Note
Data stewardship roles are common when organizations attempt to exchange data, precisely and consistently, between computer systems, and reuse data-related resources where the steward is responsible for maintaining a data element in a metadata registry.
Now the question is how long can these systems remain isolated? Rather, can we bring all these systems under a central, single point of control? The fact is, systems cannot remain isolated, and also they cannot be controlled centrally. This is because every system needs data from every (or most) other systems. Can we then integrate everything together into a single big system so that we don't have the problem of integration at all? The question seems tempting, but this is analogous to attempting to boil sea water.
Different departments or verticals within the same organization need autonomy and so do their systems. Without the required level of autonomy, there is no freedom which will hinder innovation. Constant innovation is the key to success, in any walk of life. So, departments and their systems need to be autonomous. But, since they require each other's data, they need to integrate. This leads to the necessity for a farm of autonomous systems, which are all federated to the required level to facilitate information flow.
The following figure represents systems that are autonomous, but federated to facilitate information flow:
Integrating different functional departments within an organization is not a big hurdle, since everything is under the control of the organization. But the picture changes the moment the organization grows beyond a certain limit. Twenty first century organizations are growing beyond a single location; many of them are truly global organizations. They have operations around the globe, in multiple countries, with multiple legal, technical, and operational environments. The good news is that we have technologies like Internet, Wide Area Networks, and Virtual Private Networks for connectivity. This means global organizations will have their systems federated across the globe. All these systems will evolve due to constant innovation and business changes. They have to integrate across the firewall, and not every protocol and format can be easily traversed across the firewall.
Organizations conduct business with other organizations. An online retailer's system has to partner with its wholesaler's inventory systems. These systems are not federated in any manner due to the fact that they belong to multiple organizations. There exists no federation due to the competitive nature between organizations too. But they have to collaborate; otherwise there is no business at all. So, information has to flow across organizational systems in the Internet. This is what we mean by a permeable organization boundary which allows for constant information exchange on 24 X 7 bases, with adequate Quality of Service (QOS) features. Thus the necessity is to extend the enterprise over the edge (firewall), and this activity has to happen based on pre-plans and not as an afterthought.
The following figure explains a trading partners system that is not federated but the information flow is through the Internet:
