The word business associated to the process definition can seem ambiguous, but it is mostly due to how abstract the concept of business really is. From a developer-friendly perspective, we can think of business as a particular domain or context, or group of contexts. It could be easily exchanged in this case for domain, field, company, company unit, and any lingo that specifies a defined area of work. Business processes need to be evaluated, analyzed, designed, modeled, and validated by people who understand the domain where those processes belong. Since there are different business roles within a company, each should know a different perspective of the same process. Then, the business process becomes a common point of interaction and a common language between those roles. Activities, decisions, events, and many different components will define the structure of the way a process that is relative to a business needs to be conducted.

It is important to notice that the goal of business processes is tightly associated to the business goals, objectives, and strategies. With all these considerations in mind, we can arrive to the following definition:

Business processes are a sequence of business activities done by business users and business applications (company or third-party systems) to achieve a business goal for the purpose of a specific increase in value from the business' perspective.

We'll analyze this definition in detail by splitting it into its three main concepts and covering them thoroughly.

An activity is a black box piece of work that contributes directly to achieving a business goal. We surely need to understand how the activity is performed. However, as we say, it is a black box, because from the process' perspective, the only things important are the inputs it needs and the outputs produced. The activity performed can be very simple or very complex, depending on the perspective of the process definition.

What is considered an activity in one process perspective can be composed of a group of low-level processes, which are in turn composed of a series of simpler activities. For example, a high-level process in a car factory can have an activity called "Build the Car's Engine", but from a lower-level perspective, activities could be detailed to the point of just telling a robotic arm to activate the weld for a second (one minuscule step in the car's engine preparation). In other words, a business process can be composed of multiple subprocesses.

Once you select a perspective or level for a particular process definition, you should stick to it for describing all activities regarding that specific process. You can call other lower-level processes from activities, so you can always go into more detail later. Besides, high-level strategic processes and low-level more technical processes will surely have different roles and experts behind them. A manager in a car factory might not be interested in having so much detail in their perspective, but an engineer would. Also, managers prefer to see the bigger picture, and tend to simplify their views to be able to cope with everything at once.

Usually, the low-level perspectives end up driving the operative end of the company, performing work such as the following:

In the end, all processes are tied together by high-level perspective processes, which end up providing information for decision making, managing, and coordination. High-level processes usually aggregate information about the general performance, possible improvements, or any sort of relevant information for management. We will see examples of subprocesses, role assignment, and different activity type handling when we cover process writing in Chapter 3, Using BPMN 2.0 to Model Business Scenarios.

Business processes, at least when designed correctly, are not thought to be monolithic systems that cover everything, but coordination software between other systems (the business applications) and people (the business users). There are some differences in the way people and systems behave and also in the way they need to be coordinated.

The most important part of the business process definition is achieving a goal. It's the sole reason for the process' existence; without it, the process has no purpose at all. This should never be forgotten.

The business goal of a process shouldn't be disruptive or impossible to accomplish in any way in the selected business process perspective. Each business process definition must define a clear goal and all the activities must be defined in order to contribute to achieving said goal.

Mixing goals and perspectives is a common mistake that you need to avoid when you model your first business processes. For example, when defining the process of building a car in an automated factory, we might use the same process to define the steps a robotic arm should follow to make a simple weld, overcomplicating the process. When this happens, it's recommended to split the business process definition into multiple well-scoped processes for the sake of understandability and maintainability.

One way of achieving this difficult task is to create a brief textual description of the process' main responsibility, its ownership, and the concrete goal that it was designed to achieve. This brief description can be used to train new people to understand why the process is useful to the company. You will also end up with a self-documented process that can be used by the company to improve quality levels.

Discovery of new processes is started most of the time by business analysts. It involves a certain level of knowledge engineering; a branch of requirement gathering involved in correctly merging different knowledge representation strategies, such as business rules, process definitions, and so on, with the knowledge from domain experts.

This stage has added weight when you're implementing the first iteration of the BPM cycle, which is choosing a starting point to demonstrate the importance of BPM for the business. I recommend choosing a small process to start, with a noncritical objective from the business perspective, since the results of the first iteration will surely teach you a lot of ways to do it better in the next iteration. It is important to take time to evaluate the learned lessons at the end of each of the iterations.

You'll notice that business analysts alone are not enough to perform this stage. Most companies that reach a certain level of maturity on BPM end up having a Process Improvement department, which involves technical people and a project leader solely dedicated to discovering and improving company business processes. They also include or collaborate with business experts, sponsors, and BPM champions (highly ranked people in the company, by title or merit, who are dedicated to encouraging the use of BPM throughout the company). Making BPM an enterprise-wide discipline is extremely important to make it work successfully; therefore, communication and acceptance of all areas of the company involved in BPM becomes a priority.

After identifying a goal to build a process around, this stage consists of performing interviews with business experts, representatives of operation, and anyone who is involved (or that should be involved) in the process. To achieve effective interviews, you have to prepare a questionnaire for each person/role involved in the process. Always target your questions to each role in the company. It is also advisable to have a wide set of questions to ask each interviewee as well as more specific questions, in case complex activities arise.

Some questions that I've found useful for these interviews are as follows:

If they answer that they do paper work, you should get a scanned version of all the forms that have to be filled out. If they are using different systems/screens/applications, get those screenshots.

Always let interviewees know the goal of the process being analyzed and the purpose of improving the processes and how that would help them in their everyday work. This will increase collaboration on their side and reduce the stress associated with the fear of being scrutinized on work performance. You're not there to evaluate them, but to learn from them. Let them know that.

The information the interviews provide will help you understand what is being done to achieve the business goal, and it will result in a list of all the activities that the company executes related to that particular process.

Once you have all the different answers, you will be able to cross-reference them to determine the main path of the business process at hand. An example that could be the outcome of this first stage can be seen in the following figure:

The preceding figure shows a brief graphical description of the relevant steps in approving a loan, but still doesn't adhere to any specific format. This is something to be dealt with in a later stage.

During this stage, you might also find new terms related to the activities in the process. Try to disambiguate those terms as much as possible by creating and updating a business dictionary with all the related terminology of the particular process.

Finally, find hidden activities related to company-wide processes (the most usual case is related to batch processes whose result end up impacting process activities). You'll need some expertise in the domain, which is why you need to check with domain experts to point out vocabulary inconsistencies, missing activities, and ambiguous terms, as well as to identify irrelevant business activities, duplicated activities, and so on.

When the business process, its owner, and the business goal have been identified, we can start working in a formal, unambiguous representation of the business process.

Formalizing processes is done using a predefined language. The purpose of the language is to be able to share the model with other people in a way that can only be understood in one way, as long as other people understand said language. There are many languages that have been designed over the last 20 years for this purpose; most of them are based on different graphical representations to help people quickly understand the activities needed to achieve the business goal.

When picking a language, you should always consider the ones that define the most widely accepted standard to make sure that most people can understand it. In 2014, the most widely accepted standard happens to be BPMN 2.0. We will learn the graphical representation and execution semantics of BPMN 2.0 in depth in Chapter 3, Using BPMN 2.0 to Model Business Scenarios. For the time being, let's just say that it provides a widely accepted formal language to represent processes that is not just implemented by jBPM6, but by many other process engines. So, even if you decide to use another engine, writing your processes in BPMN 2.0 would still be a good idea.

In this stage, business analysts trained in BPMN 2.0 will model the business processes. They should choose the level of accuracy of the process representation, depending on the time and information available for this stage. Since BPM is an iterative discipline, they can improve on that accuracy in later iterations.

This stage is also a very good point to start testing our processes through simulations. By determining the resources assigned to our process activities (people, time, and money), we can predict in a statistical way which activities would consume the most of each resource and plan ahead on assigning appropriate resources based on those simulations.

The resulting artifacts from this stage will be the formalized process with a graphical representation that can be shared and understood by different roles in the company.

In the third stage, the Development team works with the Business Analyst team to add all the technical details that will allow the process to run.

By now, we have a very important asset already implemented, that is, the business process' formal representation. This will act as common ground for exchange of ideas, improvement, formalized contract between areas, and also as documentation of what is being done. For that reason, it is important to keep it safe, versioned, and centralized. For this purpose, we usually set up a knowledge repository to store them.

Also, at this stage, the process definition needs to be enriched with all the information that its activities will handle and manipulate to achieve the business goal. There are different options to achieve this, and we'll discuss them in detail in the following sections.

At runtime, we will put our business process and assets in a production environment. For the first iterations, it will probably be a production-like environment (that is, a full development environment).

This is the point where we start training users to understand how to interact with the activities of the business processes. For doing so, it's a best practice to use a unified approach to build User Interfaces (UIs), because it simplifies training (the user will not need to adapt to different components). We will see about unified user interfaces in Chapter 6, Human Interactions.

During the first iteration of this stage, the runtime should be restricted to a few simple processes and to a small well known group of users. When we have already tested the processes doing the real work in this situation, we will be ready to handle bigger processes, bigger groups of people, and more critical tasks and business goals.

This stage is when we actually start detecting how our processes behave in the real world. We can measure how the model allows users to have information; if they need extra information, we can see which tasks can have an improved performance and many other things that start providing us with invaluable information for the next iteration. Always take notes of that information, as it will be very important for future process-related improvement.

The first step is the most difficult. After we learn to take them, pretty soon walking becomes a simple thing. The same thing happens when sending our processes to a production environment. The experience you gain from doing so is very useful, and following this book, I hope you can do it with the least amount of problems possible.

Once we have finished the major steps of stage 4 and we have a stable enough runtime, we need to start concerning ourselves with the information that runtime gives us. Process execution can send many events to components that are external to the actual runtime. Those components can be fed to a dashboard-like tool to allow us to monitor process execution and actual performance metrics. This is a stage where the process simulation from stage 2 can be validated, and notes of the actual estimations should be taken for improvement of process simulations in the next iteration.

These dashboards are really important for key people who want to see snapshots of how the company is working in order to make the right decisions. As you can imagine, knowing the number of processes and activities that are completed per hour can be really helpful for planning, accepting new commitments of work from providers and customers, as well as measuring the company's growth rate. This is just one example of the things that you can do if you have the information available.

Monitoring is about real-time information analysis and display, but it should also be about flexibility. You might need to add new sources and types of metrics as fast as possible to measure them within the runtime. A related branch of studies called Business Activity Monitoring (BAM) defines best practices for doing so. Tools for BAM must be flexible enough to show information in such a dynamic manner. For example, a manager might want to see aggregated data from different sources when he asks for something like this:

We usually display this information in different widgets that are specially designed to show very simple values. These widgets provide an overview about what is happening in the company in just one screen where we can see multiple bar graphs, line graphs, and tables that let us quickly see what percentage of processes are in different stages throughout our process runtime environment.

The important thing about the monitoring stage is the externalization of information. These metrics can provide you with different perspectives to know which are the best places for improvement in your business processes and in your company altogether.

Now, we will bring together all the things that we learn from the other stages and plan accordingly to better our business processes in our next iteration. By the time we reach this stage, we have our process runtime (and business relevant metrics) running to provide us with relevant information about the processes' execution. We might also have learned about exceptional situations in our processes that weren't considered at first and their ad hoc solutions. We now know how to simulate our processes better. We might also have new questions for our business experts from what we learned.

In this stage, we take care of business changes that need to be reflected in our business processes. All their improvements, along with the planning generated from the learned lessons, is used as an input for the next iterations—starting again from stage 1 and completing the BPM cycle.

A BPM system must have a lot of different features to be a good asset for its users. For each individual stage of the cycle, you should find different tools or projects that you will need to evaluate before deciding to adopt them.

For the Discovery stage, a BPMS should provide you with a way to gather business knowledge. The tools that are most useful for this stage are questionnaire builders, interview recording software, and other means to store and analyze information from interviewees and other sources to understand how things work in the company. Most of the open source projects provide a modeling tool without giving us the appropriate information gathering and analysis tools. This usually confuses new adopters, because they start working without figuring out first what they need to model and solve. You should be aware of this and be prepared to spend some time asking questions and analyzing the results.

The Formalization stage needs tools to model your business processes. The best ones nowadays should all support the BPMN 2.0 language to write these processes. Most of these tools are targeted to business analysts whose only technical requirement is understanding of the BPMN 2.0 language, so be prepared to provide training for the people involved in writing the processes. During the first iterations, the first processes are usually defined by people who already know how to write the processes and the implications of modeling an executable business process, and they can even use that writing time to transmit their knowledge.

Another thing to notice about the quality of a process editor is how it integrates with external configurations and modeling information, such as entity models and specific business-centric activity descriptions. Make sure that you're able to do those things when you evaluate a process editor.

The Implementation and Runtime stages are the ones where the most tools are usually found. They constitute the main component of a BPMS, that is, they provide an execution environment for your processes. From the software point of view, these two stages are well covered by the existing open source projects in the market. It's important to understand that a BPMS should allow the technical people of the company to directly interact with the process engine so that they can customize and extend the provided generic behavior. During these stages, we will see a strong relation of BPM with SOA-based applications. BPM can become a very important component in relation to web services, as a coordinator, and as a consumer; it can even be exposed itself as a web service. In Chapter 10, Integrating KIE Workbench with External Systems, we will see how to expose existing tooling through web services.

In the Monitoring and Improvement stages, you should concentrate on three things; the API provided to extract information about process executions from the runtime, how simple it is to create a new indicator, and the dashboard's visual flexibility.

Most problems arise in the most decision-intensive stages, that is, stages 1, 5, and 6. There is no current standard methodology to discover business processes. Since they are nontechnical tasks, the maturity of the software related to these stages is usually not great. Even if you have automated questionnaire forms, dashboards, and improvement planning software, you still need to analyze the data by people and make decisions. Also, to make those decisions, you need to learn about the company to do those tasks correctly.

From a developer's perspective, most BPMS provides a set of API's to easily plug and integrate your applications with the process engine (irrespective of the technology or language that you use in your company's applications). BPMS also provide a clear description about the information they store by default, and how to extend and customize that information for specific domain analysis. This mechanism is usually designed to be generic and extensible.

The main differences that you can usually find between process engines are as follows:

These items are a good starting point to evaluate and compare different Process Engines' features. After that is done, the next step is to analyze the available tools for each stage.

Tooling usually is heavily evaluated during the first phase of comparison between different projects, especially in case of integration projects. For real-life full implementations, it is always preferable to adapt the tools for a particular usage—after modest to extensive modifications are made to them. Usually, the desired output is to integrate it with existing applications and within company proprietary frameworks.

From the process engine internal functionality perspective, it's easy to build tools and integrate them into your existing applications using the APIs provided. From the BPMS perspective, you should always check that the internal functionality from the process engine is easily exposed through a set of APIs, because the majority of closed source products don't expose the internal APIs to provide extension points, and this can make integration quite difficult to manage. BPM plays an important role in integration with other enterprise applications and providing service coordination. The more importance BPM gains for a company, the more it will be related to controlling different activities performed by many different applications in a company. The easier the said integration can be done, the better and less painful BPM adoption becomes.

To integrate tools, if it is in the interest of your company during evaluation time, you should check the following features:

Remember, you must be ready to change all project generic UIs/tools provided by the BPMS. You must know how they were built. A good measure of the tooling development quality is how easy it is to start extending the tooling projects.

Last, but no less important, is the evaluation of how well the features of the selected BPMS relate to established standards. There are two standards you should check for current applications, the Web Service Human Task (WS-HT) standard and the BPMN 2.0 standard. The WS-HT standard defines two main features:

Also, the BPMN 2.0 standard defines a set of XML tags to describe business processes, their interactions with external systems, and their structures.

The biggest advantage of having a set of standard API's is that the better the different implementations adhere to those standards, the easier it is to decouple the definitions of processes from the engine's internal functionality—this allows for the possibility of migrating to another technology if you ever wish to. We will learn more about BPMN 2.0 and human interactions in Chapter 3, Using BPMN 2.0 to Model Business Scenarios, and Chapter 6, Human Interactions, respectively.