Now that we have defined what a business process is, we can start understanding how to manage the way processes interact with our organization. To do so, we will define six different stages that involve business process discovery, modeling, formalizing, execution, monitoring, and improvement. They constitute an iterative lifecycle for our business processes and the way they relate to their context. The following figure shows all the relevant participants and the cyclic nature of BPM:
BPM cycle and its participants
The BPM discipline's scope and main goal is to improve the current business situation by planning iterations to solve well-defined problems, and it is not about coding Java or software development at all. You may be wondering how BPM achieves such a difficult task and also how it is related to jBPM6.
After the first three chapters of this book, the content will be really technical. However, it will be clearly shown how all these concepts solve real-life situations. We will see that the jBPM6 project structure, APIs, and designs are backed up by these concepts, and you will understand the importance of knowing them for future designs.
The following sections describe the stages proposed by the BPM disciplines. These stages highlight the most important points that need to be understood to start using BPM systems.
Discovery of new processes is started most of the time by business analysts. It involves a certain level of knowledge engineering; a branch of requirement gathering involved in correctly merging different knowledge representation strategies, such as business rules, process definitions, and so on, with the knowledge from domain experts.
This stage has added weight when you're implementing the first iteration of the BPM cycle, which is choosing a starting point to demonstrate the importance of BPM for the business. I recommend choosing a small process to start, with a noncritical objective from the business perspective, since the results of the first iteration will surely teach you a lot of ways to do it better in the next iteration. It is important to take time to evaluate the learned lessons at the end of each of the iterations.
You'll notice that business analysts alone are not enough to perform this stage. Most companies that reach a certain level of maturity on BPM end up having a Process Improvement department, which involves technical people and a project leader solely dedicated to discovering and improving company business processes. They also include or collaborate with business experts, sponsors, and BPM champions (highly ranked people in the company, by title or merit, who are dedicated to encouraging the use of BPM throughout the company). Making BPM an enterprise-wide discipline is extremely important to make it work successfully; therefore, communication and acceptance of all areas of the company involved in BPM becomes a priority.
After identifying a goal to build a process around, this stage consists of performing interviews with business experts, representatives of operation, and anyone who is involved (or that should be involved) in the process. To achieve effective interviews, you have to prepare a questionnaire for each person/role involved in the process. Always target your questions to each role in the company. It is also advisable to have a wide set of questions to ask each interviewee as well as more specific questions, in case complex activities arise.
Some questions that I've found useful for these interviews are as follows:
What is your role in process X?
Which screens do you use to complete the activity X?
Are you doing paper work? What kind of forms are you filling out?
Is the activity related to the review of information sent by another person or system?
Are you an expert of a specific topic? Are you the only one responsible for that activity? Are there more people trained for that specific activity?
How many activities are you doing inside a specific business process?
How many activities related to different business processes are you currently doing? (per day/week/month)
Do you need to move information from one place to another by moving paper forms to different departments or using the postal service?
Do you use e-mail/chat as a communication channel to send information to customers or to other business units?
Do you interact directly with customers/clients face to face?
Are you aware of the BPM practice and why the company wants to adopt it?
Are you handling duplicate or unnecessary information?
What are the well-known flaws of your activities?
How do you deal with exceptional situations, missing pieces of information, or new cases?
If they answer that they do paper work, you should get a scanned version of all the forms that have to be filled out. If they are using different systems/screens/applications, get those screenshots.
Always let interviewees know the goal of the process being analyzed and the purpose of improving the processes and how that would help them in their everyday work. This will increase collaboration on their side and reduce the stress associated with the fear of being scrutinized on work performance. You're not there to evaluate them, but to learn from them. Let them know that.
The information the interviews provide will help you understand what is being done to achieve the business goal, and it will result in a list of all the activities that the company executes related to that particular process.
Once you have all the different answers, you will be able to cross-reference them to determine the main path of the business process at hand. An example that could be the outcome of this first stage can be seen in the following figure:
The preceding figure shows a brief graphical description of the relevant steps in approving a loan, but still doesn't adhere to any specific format. This is something to be dealt with in a later stage.
During this stage, you might also find new terms related to the activities in the process. Try to disambiguate those terms as much as possible by creating and updating a business dictionary with all the related terminology of the particular process.
Finally, find hidden activities related to company-wide processes (the most usual case is related to batch processes whose result end up impacting process activities). You'll need some expertise in the domain, which is why you need to check with domain experts to point out vocabulary inconsistencies, missing activities, and ambiguous terms, as well as to identify irrelevant business activities, duplicated activities, and so on.
When the business process, its owner, and the business goal have been identified, we can start working in a formal, unambiguous representation of the business process.
Formalizing processes is done using a predefined language. The purpose of the language is to be able to share the model with other people in a way that can only be understood in one way, as long as other people understand said language. There are many languages that have been designed over the last 20 years for this purpose; most of them are based on different graphical representations to help people quickly understand the activities needed to achieve the business goal.
When picking a language, you should always consider the ones that define the most widely accepted standard to make sure that most people can understand it. In 2014, the most widely accepted standard happens to be BPMN 2.0. We will learn the graphical representation and execution semantics of BPMN 2.0 in depth in Chapter 3, Using BPMN 2.0 to Model Business Scenarios. For the time being, let's just say that it provides a widely accepted formal language to represent processes that is not just implemented by jBPM6, but by many other process engines. So, even if you decide to use another engine, writing your processes in BPMN 2.0 would still be a good idea.
In this stage, business analysts trained in BPMN 2.0 will model the business processes. They should choose the level of accuracy of the process representation, depending on the time and information available for this stage. Since BPM is an iterative discipline, they can improve on that accuracy in later iterations.
This stage is also a very good point to start testing our processes through simulations. By determining the resources assigned to our process activities (people, time, and money), we can predict in a statistical way which activities would consume the most of each resource and plan ahead on assigning appropriate resources based on those simulations.
The resulting artifacts from this stage will be the formalized process with a graphical representation that can be shared and understood by different roles in the company.
In the third stage, the Development team works with the Business Analyst team to add all the technical details that will allow the process to run.
By now, we have a very important asset already implemented, that is, the business process' formal representation. This will act as common ground for exchange of ideas, improvement, formalized contract between areas, and also as documentation of what is being done. For that reason, it is important to keep it safe, versioned, and centralized. For this purpose, we usually set up a knowledge repository to store them.
Also, at this stage, the process definition needs to be enriched with all the information that its activities will handle and manipulate to achieve the business goal. There are different options to achieve this, and we'll discuss them in detail in the following sections.
We will select a model to work with, and our executable entity model will be created based on discovery stage results.
We usually have an inherited business model from legacy systems. When that's the case, you have three options:
The first option is using the inherited model for your process activities. Its pros and cons are as follows:
Pros: It is fast to implement, especially if the developers of your organization already know the model.
Cons: Inherited models could carry an unnecessary complexity for the process' required level of data. Also, any change needed to be done to the process' internal model might impact a third-party project.
The second option is storing external keys for the real entities inside your business process. The data will remain consistent and updated as long as it can be managed by a master data source from the process' engine point of view. Its pros and cons are follows:
Pros: You can reuse your old model and even its persistence, and only store a key to get the related information when needed.
Cons: If you store a key, you will need to query another data source every time the information is needed, and depending on the case, you might need to modify external information (with all the problems regarding permissions, communication, transactions, and so on). Depending on the amount of concurrent process executions, this could cause a performance issue.
The third option is creating an understandable wrapper model that abstracts the legacy model, data sources, and communication strategies required by your business processes' model objects. Its pros and cons are follows:
Pros: You can map any outdated terms or concepts to a business process' relevant names and structures, and define underlying strategies to fetch information from external systems or to produce the needed data.
Cons: Writing and maintaining the integrity of the wrapper model will take time, and an expert on both models will have to worry about keeping everything in sync.
Once you define and know how to get and update information from your business model, you will need to bind each bit of information with the correspondent activity in your process. We usually do this with an expression language that allows us to express, in a declarative way, the information that we need without saying where it can be obtained. One example of this could be #{ambulance.doctor.speciality}.
This expression will be evaluated at runtime, and an internal mechanism will be used to retrieve the information.
The business process provides us with a set of activities that need inputs and produce outputs. As technical developers, we will have to provide technological assets to provide such functionality, from creating form renderers for human interactions to creating connectors to external web services and transformations for external models to our entity model. In Chapter 7, Defining Your Environment with the Runtime Manager, we will analyze all the technical requirements to implement user interfaces, and in Chapter 10, Integrating KIE Workbench with External Systems, we will review all the relevant details about system-to-system interactions and the mechanisms that we need to know in order to keep everything simple.
Having a clear vision of the components that we need to implement and having a standardized and conceptually coherent way of interaction will make our life easier, and we will end up with simple applications that are easy to maintain.
By the time we finish this stage, our business processes will be executable. This will allow us to test, verify, validate, and simulate the process behavior. For the next iterations, this stage becomes unnecessary, and the only thing that changes between one implementation and another are the external systems' connectors as well as the technologies used to build frontends. All these topics will be covered in the jBPM6 technical sections where the technical details that need to be considered and best practices will be introduced. Also, in this stage, we can start showing the progress of our process definition in playbacks, so all parties involved in the process discovery and formalizing can see that the goals of the business processes are achieved.
At runtime, we will put our business process and assets in a production environment. For the first iterations, it will probably be a production-like environment (that is, a full development environment).
This is the point where we start training users to understand how to interact with the activities of the business processes. For doing so, it's a best practice to use a unified approach to build User Interfaces (UIs), because it simplifies training (the user will not need to adapt to different components). We will see about unified user interfaces in Chapter 6, Human Interactions.
During the first iteration of this stage, the runtime should be restricted to a few simple processes and to a small well known group of users. When we have already tested the processes doing the real work in this situation, we will be ready to handle bigger processes, bigger groups of people, and more critical tasks and business goals.
This stage is when we actually start detecting how our processes behave in the real world. We can measure how the model allows users to have information; if they need extra information, we can see which tasks can have an improved performance and many other things that start providing us with invaluable information for the next iteration. Always take notes of that information, as it will be very important for future process-related improvement.
The first step is the most difficult. After we learn to take them, pretty soon walking becomes a simple thing. The same thing happens when sending our processes to a production environment. The experience you gain from doing so is very useful, and following this book, I hope you can do it with the least amount of problems possible.
Once we have finished the major steps of stage 4 and we have a stable enough runtime, we need to start concerning ourselves with the information that runtime gives us. Process execution can send many events to components that are external to the actual runtime. Those components can be fed to a dashboard-like tool to allow us to monitor process execution and actual performance metrics. This is a stage where the process simulation from stage 2 can be validated, and notes of the actual estimations should be taken for improvement of process simulations in the next iteration.
These dashboards are really important for key people who want to see snapshots of how the company is working in order to make the right decisions. As you can imagine, knowing the number of processes and activities that are completed per hour can be really helpful for planning, accepting new commitments of work from providers and customers, as well as measuring the company's growth rate. This is just one example of the things that you can do if you have the information available.
Monitoring is about real-time information analysis and display, but it should also be about flexibility. You might need to add new sources and types of metrics as fast as possible to measure them within the runtime. A related branch of studies called Business Activity Monitoring (BAM) defines best practices for doing so. Tools for BAM must be flexible enough to show information in such a dynamic manner. For example, a manager might want to see aggregated data from different sources when he asks for something like this:
"The average time of completion of one or a group of processes related to client X accounts in the last month."
We usually display this information in different widgets that are specially designed to show very simple values. These widgets provide an overview about what is happening in the company in just one screen where we can see multiple bar graphs, line graphs, and tables that let us quickly see what percentage of processes are in different stages throughout our process runtime environment.
The important thing about the monitoring stage is the externalization of information. These metrics can provide you with different perspectives to know which are the best places for improvement in your business processes and in your company altogether.
Now, we will bring together all the things that we learn from the other stages and plan accordingly to better our business processes in our next iteration. By the time we reach this stage, we have our process runtime (and business relevant metrics) running to provide us with relevant information about the processes' execution. We might also have learned about exceptional situations in our processes that weren't considered at first and their ad hoc solutions. We now know how to simulate our processes better. We might also have new questions for our business experts from what we learned.
In this stage, we take care of business changes that need to be reflected in our business processes. All their improvements, along with the planning generated from the learned lessons, is used as an input for the next iterations—starting again from stage 1 and completing the BPM cycle.