Book Image

Business Process Driven SOA using BPMN and BPEL

5 (1)
Book Image

Business Process Driven SOA using BPMN and BPEL

5 (1)

Overview of this book

Table of Contents (13 chapters)
Business Process Driven SOA using BPMN and BPEL
About the Authors
About the Reviewer

Technical Introduction to SOA

The development of composite applications is one of the major contributions of SOA, which brings us to the technical aspects of SOA. In this section, we will look at the technical background of SOA.

To be able to develop composite applications, that is, to compose business processes out of services, we need technologies that will allow us to develop services. The obvious answer to this are 'web services', as they are best aligned with the concepts of SOA. With this, we want to emphasize that SOA is not bound to specific technologies. What matters are the concepts. The most important thing from the perspective of a developer is the service description, the WSDL (Web Services Description Language).

We also need a language and a technology to perform the composition of services into processes, and an environment in which to execute the processes. You may argue that we could use well-known programming languages such as Java or C# for this. It turns out that the process composition differs somewhat from traditional programming. With composition, we merge services into larger services and processes. In other words, we represent the high-level state transition logic of a process. Using programming languages such as Java, C#, and so on for these purposes is likely to result in inflexible solutions, particularly because there is no clear separation between the process flow and the business logic, and these should not be tightly coupled. However, we continue to use traditional programming languages to develop services.

In addition to these facts, the composition of business processes has other specific requirements such as support for many process instances, long-running processes, compensation, fault handling, correlation, parallel flows, complex dependencies, asynchronous invocations, and so on. All of these make it reasonable to use a specific language and a specific environment for business processes.

In SOA, the language used is BPEL and the environments for executing processes are called process servers (sometimes called BPEL engines). Process servers provide additional valuable features, such as an overview of the running processes, completed processes, faulted processes, and so on. They provide a complete history of process execution and insight into process activities.

BPEL is an executable language. We are emphasizing this because, in order to execute BPEL processes, we have to specify details such as interfaces, messages, variables, types, and so on. These details are usually not needed at the time when we model the business processes. Therefore, for modeling, we will use a modeling notation, the BPMN. BPMN has been designed explicitly for process executions. Therefore, it provides round-trip mapping from BPMN to BPEL.

This brings us to the simplified SOA architecture, as shown in the following figure:

Let us now look briefly at the different building blocks. First we will look at the BPMN, then at the BPEL, and finally at the services.


BPMN is a graphical notation used for business process modeling. We use BPMN to draw business process diagrams. These diagrams present the activities and tasks of a process and their relations. The diagrams use flowchart concepts to represent the logic of business processes.

BPMN is a visual language, and uses a set of graphical elements. Activities are represented as rectangles, and decisions as diamonds. BPMN successfully joins the simplicity of the diagrams with expressive power, which allows BPMN to be used for complex processes and specification of details.

We will take a closer look at the syntax of BPMN in the next chapter.


BPMN has been designed specifically for SOA.


BPEL has been adopted as the de facto standard for developing executable business processes. The main goal of BPEL is to standardize process automation between services.


With BPEL, we can describe business processes in two distinct ways: we can either specify the exact details of business processes, or specify the public message exchange between parties. The former processes are called executable business processes and can be executed by a process server. The latter processes are called abstract processes. They do not include the internal details of process flows, and are not executable. In the real world, BPEL has been used predominantly for executable processes.

Executable business processes are processes that comprise a set of existing services and specify the exact algorithm of activities, along with the input and output messages. With BPEL, we can define business processes that make use of services, and business processes that externalize their functionality as services. When we define an executable business process in BPEL, we actually define a new service that is a composition of existing services.

Within enterprises, BPEL is used to standardize enterprise application integration, and extend the integration to previously isolated systems among business partners. Between enterprises, BPEL enables easier and more effective integration with business partners. Further, BPEL encourages enterprises to define their business processes, which in turn leads to business process optimization, re-engineering, and selection of the most appropriate processes, thus further optimizing the organization. Definitions of business processes described in BPEL do not influence existing systems, thus stimulating upgrades. BPEL is the key technology in environments where functionalities already are, or will be, exposed via services.

BPEL represents a convergence of two early workflow languages, WSFL (Web Services Flow Language) by IBM, and XLANG by Microsoft. IBM, BEA, and Microsoft jointly developed the first version of BPEL in August 2002. Since then, several other partners have joined. Version 1.1 was adopted in March 2003. In April 2003, BPEL was submitted to OASIS (Organization for the Advancement of Structured Information Standards) for standardization purposes, where the WSBPEL TC (Web Services Business Process Execution Language Technical Committee) has been formed. In 2007, WS-BPEL version 2.0 was adopted.

BPEL uses an XML-based vocabulary to specify and describe business processes, and is based on WSDL, XML Schema, and XPath specifications. Familiarity with these specifications will be helpful while learning BPEL.


With BPEL, we can define both simple and complex business processes. BPEL offers constructs such as loops, branches, variables, assignments, and so on, which allow us to define business processes in an algorithmic way. The most important BPEL constructs are related to the invocation of services. BPEL makes it easy to invoke operations of services either synchronously or asynchronously. We can invoke operations either in sequence, or in parallel. We can also wait for callbacks. BPEL provides a rich vocabulary for fault handling, which is very important, as robust business processes need to react to failures in a smart way. BPEL also provides support for long-running process and compensation, and allows us to undo partial work done by a process that has not terminated successfully. The following are the most important features that BPEL provides:

  • Describing the logic of business processes through composition of services

  • Composing larger business processes by combining smaller processes and services

  • Handling synchronous and asynchronous (often long-running) operation invocations on services, and managing callbacks that occur later

  • Invoking service operations in sequence, or in parallel

  • Selectively compensating completed activities in case of failures

  • Maintaining multiple long-running transactional activities, which are also interruptible

  • Resuming interrupted or failed activities to minimize re-work

  • Routing incoming messages to the appropriate processes and activities

  • Correlating requests within and across business processes

  • Scheduling activities based on the execution time and defining their order of execution

  • Executing activities in parallel and defining how parallel flows merge based on synchronization conditions

  • Handling message-related and time-related events


SOA and BPEL approaches rely on services. We compose services into processes. To be able to take advantage of this approach, we need services. Services are discrete self-contained building blocks of SOA that contain business logic. Services expose business logic through well-defined interfaces.

Services provide business functionalities such as applications for business travel, applications for loans, and so on. This differs considerably from technology-oriented functionalities such as retrieving or updating a table in a database. Services in SOA must provide business value, hide implementation details, and must be autonomous. Therefore, in SOA we usually organize services in several layers. In the top-most layer, we have business services.


Business services should represent activities of a business process. At the same time, they should be generic, so that they can be used in different business processes.

Such business services cannot be developed in an ad-hoc manner. They require a lot of planning and designing. The success of SOA relies on services and the ability to define these services at the right level of abstraction, to make them re-usable, and to use them in different processes. From this perspective, we could say that SOA is the development of the overall IT architecture.

Business servers are usually composed of lower-level services. At the bottom, we have technical services. For example, these can be stored procedures, exposed from the database, or functions or methods of an existing application, and so on.

Defining the right services and positioning in the right layers requires some planning and designing. It also requires us to be familiar with good practices and patterns. This should lead us to the architecture as shown in the following figure:


From the technical perspective, processes and services do not differ. They both expose interfaces, which are described in WSDL. Therefore, we can develop composed services using different approaches (BPEL, programming languages, and so on). We can also re-use services and/or processes. As they look similar from outside, they give us the opportunity to develop multiple layers of services and processes, from the simplest services to the complex end-to-end processes.

Service Interfaces

It is also important to know how service interfaces are defined. The interface of a service defines a set of public operation signatures. The interface is a contract between the service provider and the service consumer. Interface is separated from the implementation, is self-describing, and is platform-independent. In order to define business services, we need to focus on the correct granulation of operations. SOA services are best modeled with coarse granulation.


Service operations are defined as a set of messages. Messages specify the data to be exchanged and describe it in a platform-and language-independent way, using schemas. Services exchange only data, which differs considerably from object oriented and component approaches, where behavior (implementation code) can also be exchanged. Operations should also be idempotent (operations are idempotent if repeated invocations have the same effect as one invocation).


Service consumers can use synchronous or asynchronous communication modes to invoke operations of services. In synchronous mode, a service operation responds to the service consumer after the processing is complete. The service consumer has to wait for its completion. Usually, we use synchronous mode when operations complete processing after a short period of time. In asynchronous mode, a service operation does not respond to the consumer, although it may return an acknowledgement so that the consumer knows the operation has been invoked successfully. If a response is needed, a callback from the service to consumer is used. In such a scenario, message correlation is needed.

Loose Coupling

Loose coupling of services is achieved through self-describing interfaces, coarse granulation, the exchange of data structures, and support for synchronous and asynchronous communication modes. Loosely-coupled services are services that expose only the necessary dependencies and reduce all kinds of artificial dependencies. This is particularly important where services are subject to frequent changes. When a service is modified, minimal dependencies ensure that changes made to other services are minimal. Such an approach improves robustness, makes systems more resilient to changes, and promotes re-use of services.

Quality of Service

For developing truly re-usable services, it is also important that we care about attributes, such as availability, performance, security, and so on. These are called quality of service attributes. Quality of service attributes are important in large information systems. In web services, quality of service attributes are covered by WS-* specifications such as WS-Security, WS-Addressing, WS-Coordination, and so on.

Other Important Parts of SOA

The scenario described above with BPMN, BPEL, and services covers only the most essential parts of SOA. For a full-blown SOA, other important parts are required. These include:

  • Enterprise Service Bus

  • Registry and repository

  • Rules Engine

  • BAM

  • User interactions (human workflow)

SOA is also related to the following:

  • Unification of presentation layer, which is related to portals

  • Security and identity management

These layers are shown in the following figure:

In the coming sections, we will describe the building blocks of a full-blown SOA architecture, starting with the Enterprise Service Bus.

Enterprise Service Bus

The Enterprise Service Bus (ESB) addresses the communication between the services and the processes. We have already mentioned that for services, we can use various technologies. In most cases however, we will use web services. Web services communicate using the SOAP protocol, which can be mapped to one of the transport protocols, most likely to HTTP. HTTP is an unreliable protocol, and is therefore not well-suited to mission-critical applications.

ESB addresses this problem and provides a reliable mechanism for communication between services. In addition, it offers protocol mapping capabilities. This means that ESB can translate different protocols, such as SOAP to JMS (Java Message Service) or vice versa. This is important because in enterprise information systems, we usually have some components already developed in different technologies. We might even have a kind of middleware, such as a message broker. ESB helps to connect and integrate these systems.

ESB can also enable reliable message delivery and provide other value-added features, which are very useful in mission critical systems. Usually, ESB also addresses aspects of security and transaction management. It allows us to configure secure access to the services and processes, and to monitor the access. It also allows the definition of services that should participate in transactions. Some ESBs even allow us to choose between ACID (WS-AtomicTransactions) and compensating transactions (WS-BusinessActivity).

Services in SOA are described in WSDL, and use input and output messages, which are formatted using XML Schema. In the ideal world, all services would use the same schema definitions. In the real world however, this is not the case. Therefore, we often have to transform the XML payload from the output schema of one service to the input schema of another service. For such transformations, we often use XSLT, XPath, or XQuery. ESB provides the ability to do such transformation transparently, on the bus, without having to modify existing services or processes. This capability to transform XML payloads is particularly valuable when new versions of services or processes are deployed. New versions might change the interface or the message schemas somehow. With ESB, we can mask these differences without having to modify all related clients.

Related to transformations is message enhancement. Sometimes, it is not enough just to transform the XML payload. We may need to add or even alter certain information in order to make the message compliant with the service definition. Such alterations may can be simple, such as altering the representation of date, or perform more complex operations such as calculating data, translating types, and so on.

ESB can also serve as a message router. Sometimes, we might want to route messages to certain services. Such routing is usually done based on some deterministic business rules, or can be based on message content, user information (which user created the message, or which user is the receiver of a message), or the time when the message was created. Routing messages on the bus gives us the advantage of doing this in the Good ESBs are integrated with Rules Engines, and allow us to use the rules from the engine for routing.

Routing is related to service mapping. ESBs can also be used to dynamically map a service implementation to the corresponding WSDL interface. On ESBs, this can be done in run time, which gives us greater flexibility over which implementation we will use. If this feature is used with care, it can improve overall flexibility.

In future, we believe that SOA will introduce another important concept: business events. This will extend the way services and processes interoperate. Instead of invoking services by calling their operations, we will simply trigger a business event. Such events will be consumed by those services and processes that have subscribed to the event. ESB will play a central role in this scenario, because it will monitor, route, and manage events in the same way as service invocations.

Registry and Repository

A very important aspect of SOA is re-use. Re-use at the service level means that when we compose executable processes (in BPEL), we re-use existing services as far as possible. Successful re-use is not easy to achieve and requires at least the following:

  • The developers are motivated to develop re-usable services. Developing a re-usable service initially takes longer than developing a single-purpose service. Re-use pays off in the second step, when the service is actually re-used. The more it is re-used, the more justifiable this initial longer development becomes.

  • Developers also have to be motivated to search for services and to identify the most appropriate services to use. They may also need to modify the behavior of the service slightly. In such a case, they need to know who else uses the services, and the boundaries within which they can make changes.

  • To achieve maximum re-use and keep the architecture sound, it is useful to have some sort of governance in place. Governance procedures require information on who uses a service, how many times a service has been re-used, what level of re-use is achieved in processes, and so on.

Another very important aspect is the deployment process. A professional deployment has three environments:

  • development

  • test

  • production environment

When an application is ready for production use, it is deployed into the production environment. Controlled deployment of processes requires that a process starts using the production version of services instead of the development or test version. If references to services (partner links) are hard-coded into the BPEL processes, such migration can be very painful, as it might, in the worst case scenario, require manual changes to URL addresses. When deploying a service, we might want to know which processes use this service, because we might want to retest those processes.

Addressing all of the aspects mentioned above is very difficult if we have no list of the available services, or when processes and services are tightly coupled (for example, when a process uses the direct URL of a service in the partner link).

Registries and repositories address these aspects. They are used to register services in a central location. Once registered, we can search and locate appropriate services at design time and at run-time. The more metadata we include for a service, the better the search capabilities that the registry and the repository can provide.

The rule of thumb is that once we have more than 50 services, we will desperately start needing the registry and the repository. However, sometimes it might be wise to introduce it from the beginning, because once developers get used to a certain development process, it will be very difficult to change their behavior.


The current notion about the role of registries and repositories in SOA has changed considerably. A few years ago, we believed that a relatively simple registry—UDDI would cover all our needs. Today, we have identified that a registry alone is not powerful enough for SOA, because in many cases it makes sense to store service metadata as well (WSDL interfaces, XSD schemas, and many more). Therefore, today we talk about registries and repositories.

A powerful registry and repository should have the following features:

  • Classification capabilities to categorize and classify services and processes based on one or more classification schemas. This simplifies queries and enables easier location of the most appropriate services for re-use.

  • Governance functions to enable the definition of proprietary service/process lifecycles together with the conditions to go from one stage of the life cycle to another. Stage transition can trigger automatic actions such as execution of validators.

  • Access control to define who can perform which operations against the registry and repository, and for which registered services/processes. Such access control could be based on XACML (eXtensible Access Control Markup Language).

  • User, programming, and administration interfaces.

Rules Engines

Business rules are an important part of each enterprise. Experience has shown that business rules change; sometimes very often, sometimes not that often. Today, business rules are coded into different applications and are tightly-coupled with the implementation of an application system. Changing business rules is therefore very difficult, and requires modifications to applications. Each modification requires testing and deployment, which makes things even more complicated. The major challenge today, however, is to identify the applications into which the same business rule is coded, and to modify all applications where this rule has been used. If we forget some applications, this could lead to unpredictable results, which we do not want to happen.

Business rules are very common in business processes. The idea of SOA has been to extract these rules from the executable code (from BPEL, Java, C#, and so on) and to store them in a central place, where:

  • Business rules can be re-used from different processes, services, and applications

  • User friendly interfaces exist, which enable us to change and modify business rules

Often, business processes manage business rules. Therefore, putting business rules into a central place, the Rules Engine, is appropriate because the same rule is used very often in more than one process. This might not be obvious at the start, when we will have just a few processes automated, but it will become important when the number of processes start to grow. However, similar as it is to the registry and repository, it makes sense to introduce business rules early. This is because if rules engines are introduced after we have implemented several processes, we will have to:

  • Change the way development team works

  • Extract the rules from the existing processes and services, which will require significant changes to these processes and services

With the introduction of Rules Engines, SOA now addresses the particular challenge of changes in business rules.

Business Activity Monitoring

Business Activity Monitoring (BAM) helps in answering the question "Where to optimize business processes? Once a process is automated end-to-end, we can use the SOA platform to provide quantitative data about the performance of the process.

A BAM solution gathers information about the time required to complete different activities of a process. It can then provide reports with average values for each activity. Having these numbers can be very helpful, because we can use them to identify which activities take the longest time to execute. With this, we can identify those parts of a process, where we will realize the largest benefits from optimization.

BAM solutions can provide other useful information. They can show us how many process instances are active at a specific time, and how long, on an average, it takes to complete a process. They can also show us which users (employees) have started how many process instances, and so on.

Please note that BAM is not just related to automatic activities (those implemented by services). It can be related to human activities as well. In such a case, we can use BAM to observe the productivity of the employees.

This brings us to the next important part of SOA—the human activities or user tasks: user interactions.

User Interactions

User interactions in business processes are very common. It is true that SOA's objective is to automate as many activities as possible. However, this cannot be done overnight. Therefore, we will need to model certain activities of a process as user interactions—human tasks.

User interactions in business processes can be simple, such as approving certain tasks or decisions, or complex, such as delegation, renewal, escalation, nomination, or chained execution. Task approval is the simplest and probably the most common user interaction. For example, in a business process, say for opening a new account, a user interaction might be required to decide whether the user is allowed to open the account. If the situation is more complex, a business process might require several users to make approvals, either in sequence, or in parallel. In sequential scenarios, the next user often wants to see the decision made by the previous user. Sometimes, particularly in parallel user interactions, users are not allowed to see the decisions taken by other users. This improves the decision's potential. Sometimes, a user does not even know which other users are involved, or even whether any other users are involved at all.

A common scenario for involving more than one user is workflow with escalation. Escalation is typically used in situations where an activity doesn't fulfill a time constraint. In such a case, a notification is sent to one or more users. Escalations can be chained, going first to the first-line employees and advancing to senior staff if the activity is not fulfilled.

Sometimes it is difficult or impossible to define in advance which user should perform an interaction. In this case, a supervisor might manually assign the task to an employees. A group of users or a decision-support system can also assign a task.

In other scenarios, a business process may require a single user to perform several steps that can be defined in advance, or during the execution of the process instance. Even the simplest processes might require that one workflow is continued with another workflow.

User interactions are not limited only to approvals. They may also include data entries or process management issues such as process initiation, suspension, and exception management. This is particularly true for long-running business processes where, for example, user exception handling can prevent costly process termination and related compensation for those activities that have already been successfully completed.

As a best practice for human workflows, it is usually not wise to associate human interactions directly with specific users. It is better to connect tasks to roles and then associate those roles with the individual users. This gives business processes greater flexibility, allowing any user with a certain role to interact with the process, and allowing changes to users and roles to be made dynamically.

User Interaction in BPEL

To interleave user interactions with BPEL processes, we can use a workflow service, that interacts with BPEL processes using standard WSDL interfaces, as any other service. This way, the BPEL process can assign user tasks and wait for responses by invoking the workflow service using the same syntax as any other service. The BPEL process can also perform more complex operations such as updating, completing, renewing, routing, and escalating tasks.

After the BPEL process has assigned tasks to users, users can act on the tasks by using the appropriate applications. The applications communicate with the workflow service by using WSDL interfaces or another API (such as Java) to acquire the list of tasks for selected users, render the appropriate user interfaces, and return the results to the workflow service, which then forwards them to the BPEL process. User applications can also perform other tasks such as reassigning, escalating, routing, suspending, resuming, and withdrawing tasks. Finally, the workflow service may supports other communication channels such as email and SMS, as shown in the following figure:


In the coming years, it is likely that we will see further development of user interactions in business processes with the objective of standardizing the explicit inclusion of human tasks in BPEL processes. This standardization effort is called the BPEL4People specification, which was originally developed jointly by IBM and SAP.

The most important extensions introduced in BPEL4People are people activities and people links. A people activity is a new BPEL activity used to define user interactions, in other words, tasks that a user has to perform. For each people activity, the BPEL server must create work items and distribute them to users eligible to execute them. People activities can have input and output variables and can specify deadlines.

To specify the implementation of people activities, BPEL4People introduced tasks. Tasks specify actions that users must perform. Tasks can have descriptions, priorities, deadlines, and other properties. To assign tasks to users, we need a client application that provides a user interface and interacts with the tasks. The interface can query available tasks, claim and revoke them, and complete or fail them.

To associate people activities and their related tasks with users or groups of users, BPEL4People introduced people links. People links are somewhat similar to partner links -they associate users with one or more people activities. People links are usually associated with generic human roles such as process initiators, process stakeholders, owners, and administrators.

The actual users that are associated with people activities can be determined at design, deployment, or run time. BPEL4People anticipates the use of directories such as LDAP, (lightweight Directory Access Protocol) to select users. However, it does not define the query language used to select users. Rather, it foresees the use of LDAP filters, SQL, XQuery, or other methods.