Business Process Management is the natural evolution and convergence of several powerful forces within the fields of software development methodology, enterprise application technology, and management theory. These underlying forces have all matured and converged at the right time for a productive fusion, which we know as business process management.
Evolution of software development methodologies
Traditional software development methodologies owe much to their engineering roots. The waterfall approach to software development was designed with the idea that building a piece of software is like building a bridge: the better your design and blueprint, the sturdier the end result. In reality, this approach falls very far short of perfection.
Developing enterprise application software is about delivering value to a business, and the business expresses that value as a set of business requirements. The problem with the waterfall approach, and the difference from bridge construction, is that unlike the laws of physics and the construction properties of metal and concrete, business requirements are subject to change. Businesses cannot afford to stay still: if they don't adapt to the marketplace then they will not survive. So business requirements are necessarily a shifting target.
Unfortunately, this is not the only problem with the traditional software development methodologies. There is also the problem of business requirements "dissonance". This is where the layers of end users, analysts, and developers create a chain of Chinese whispers, resulting in software that fails to resemble the original requirement. Each link in the chain puts its own interpretation on the requirement, until the end result is horribly different from what the business originally needed. This requirements dissonance can easily be visualized:
In recent years, the traditional waterfall approach to software development has been superseded by other, more adaptable methodologies. These methodologies attempt to break down the requirements dissonance by taking out the middle man as much as possible, and by creating prototypes early on, and then iterating them towards the final version. This allows for an iterative approach to software development, far removed from the "build a bridge" traditional approach:
The
most prominent of these newer development methodologies is Agile. On the right project, there is no doubt that Agile development can deliver valuable software more successfully, and more quickly, than the waterfall approach.
Nevertheless, Agile development and its ilk do have serious drawbacks and limitations. The first and most obvious limitation is that the Agile development methodology does away with the Business Analyst. This is an important drawback, because often the BA's interpretation of the requirements is more logical and more far-sighted than that of the end user who specifies the original requirement. This can mean that the developer can be led up blind alleys by an end user who doesn't have the necessary perspective.
There is also the problem that although we are removing some layers of interpretation, the layer of interpretation that we are leaving in place is the one that causes the most significant dissonance: the developer still has to interpret what the end user means.
This can mean that time is unnecessarily wasted on honing a prototype that starts off a long way from what the business needs. Indeed, some Agile developments have turned into one extremely long prototyping process, with an end result never being reached. This is an expensive way to develop software.
So what is the ideal, and where is Business Process Management in relation to this? For some idealists, the best situation would be one where the business users can build the software tools they need for themselves, without having to rely on developers or analysts. Unfortunately, although programming languages are becoming simpler all the time, we are still light years away from them being abstracted enough for an end user to build their own software. Software development is still hard.
Nevertheless, BPM does go some way towards this ideal, and given the right scenario, it can successfully deliver valuable software in extremely short time scales. In a similar fashion to Agile, BPM relies on cutting out the middle man as much as possible, only this time the emphasis is on a strong partnership between the end user and the BA working on iterations towards the final software:
The reinstatement of the Business Analyst has several advantages:
Firstly, the BA is skilled in the interpretation of requirements, and so their business process models are likely to be close to the original requirement.
Secondly, a BA's models are far easier for an end user to understand than code or even prototype software, allowing for closer collaboration and faster development.
Thirdly, the BA has the long term view and business skills to steer the end user's expression of their requirements in the most beneficial direction.
Last, and by no means least, models can usually be produced much more quickly than working software. As the working software produced by a BPM system is initially generated from the BA's process model, this is an extraordinarily fast method of software development.
Don't be tempted to think that this means developers are no longer required, however. The reality of BPM development is that it makes the working relationship between end user, BA, and developer much more symbiotic and productive, but does not make any of those roles redundant. BPM is a partnership approach to software development; on one hand between the end user and the BA, and on the other between the BA and the developer. The skills of a developer are very much still required to take a BPM system all the way to implementation. Where a business process calls for the integration of other systems, that integration work will almost certainly involve an interface built by a developer. And while the software that is generated by the BPM suite is good, it does still require some development to make it properly fit for purpose.
It would be foolhardy to suggest that the BPM approach is the right one in every software development scenario, but it is a formidable new challenger to other development methodologies. Later on in this chapter, we'll consider some of the scenarios where a BPM approach is the most appropriate.
The emergence of key technologies
Workflow software has been around since the early 90s, if not before. These systems were most often used in document management scenarios, where a document (for example, an insurance claim form) was passed between different departments as work was done on it. This worked well because the workflow system only had to maintain a pointer to the document in order to pass it down the process chain. Where things got more difficult was when the workflow system met other, task-specific systems.
Most mature processes involve the coordination of several systems. For example, we might have one system to record our insurance claim, another to work out what payment is due on the claim, and yet another to make the payment to the end customer. Before the advent of internet technologies, and specifically XML, it would have been a mammoth and fearsomely complicated task to integrate and tie together these task-specific systems and their proprietary programming languages and data formats within the context of a process. This was quite often attempted, however, and the result was usually a development and maintenance nightmare. More code ended up being written to handle the interfaces than was actually needed to process the work.
Thankfully, XML emerged from the internet revolution as a simple way for systems to talk to each other without them having to know about each other's proprietary data formats. Many task-specific systems now implement XML web services, making the task of integrating them into a process relatively simple.
As a result, business process management can certainly be viewed as a repackaging of the workflow software that was available in the 90s, but the reality is that those old tools could never have delivered the same value as BPM, because the technology landscape has fundamentally changed in the interim.
Meanwhile—management theory
The third leg of the tripod that has raised BPM up to its prominent position is the focus on process in management theory since the 1980s.
What is a business process and why do we want to manage it?
What do we mean by "business process"? We typically mean a collection of business activities that takes one or more kinds of input and creates an output that is of value to the business. It is the focus by management theorists on the elements of this definition that has led us to BPM. This quotation from W. Edwards Deming, founder of the quality movement, is illuminating:
"If you can't describe what you are doing as a process, you don't know what you're doing."
Any business process improvement project is an attempt to answer the fundamental question of "How do we organize our activities so that we can minimize inputs, maximize outputs, and maximize value?".
Business process improvement and re-engineering
There
are several strands of management theory that are built around this fundamental question, and there are some striking examples where these theories have been effectively put into practice. Think of Jack Welch, who turned General Electric from a struggling manufacturing company to a highly profitable service-based company. Amongst other initiatives, this successful transformation can be attributed to radical business process re-engineering, and adoption of Six Sigma quality practices. Think too of Michael Dell, whose company of the same name changed the playing field of PC making and retail through a relentless focus on process improvement and ruthless process efficiency.
In business process re-engineering and improvement thinking, processes are viewed as organizational building blocks with as much (if not more) significance as functional areas and geographic territories. Business process re-engineering emerged in the 1980s with the idea that sometimes radical redesign and reorganization of these process building blocks was necessary to lower costs and increase the quality of service, and that IT was the key enabler for that radical change. The trouble with this radical approach is that it is too difficult to achieve in the real world. Mature organizations often simply cannot wipe the slate clean, and re-organize themselves without the instinctive memory of past processes and procedure creeping back in. Ultimately, business process re-engineering initiatives came to be viewed as nothing more than a cover up for downsizing efforts.
Business process improvement initiatives have been more successful, although they have been hampered by the lack of a comprehensive solution. Good-quality process design would be let down by sketchy IT support that couldn't be adapted. A business process would be designed around system constraints rather than systems doing exactly what the process required.
Nevertheless, many of the elements of business process improvement have proven to be useful and have not been discarded. Business process modeling has certainly increased businesses' ability to understand their operations and to make rational decisions about how best to organize their activities. Also, the definition and measurement of process metrics have given concrete, meaningful, and achievable targets for managers to work towards. The business is now more involved than ever before in the specification and delivery of IT programs.
From this convergence, BPM emerges
BPM is the final piece of the puzzle that allows business process initiatives to be fully successful. BPM espouses the incremental approach of business process improvement, but the IT delivery phase is supported by custom-designed tools that reduce the effect of requirements dissonance by allowing the delivery to be driven by the business.
In its simplest form, workflow software is generated from the process maps that are modeled by the Business Analyst. This workflow software is then the end user's "front end" to the process, and it controls the execution of the process in the live environment. Other software is then used to report on the operation of the process within the workflow software, allowing for dashboarding of key performance indicators. These dashboards can in turn be used to drive ongoing process improvement decisions.
Business process management isn't just one piece of software or one analysis technique: it is a suite of software, a framework of analysis techniques, and a defined project lifecycle. The Business Analyst, with their unique perspective on both business and technology, are in the happy position of having the right relationships and the right skill set to drive BPM initiatives in the enterprise.