Process mining – creating passive management systems?

Neil Ward-Dutton

Tuesday, February 21, 2012 by

Although it’s not a formal part of the BPM research programme I set out in advance at the end of last year, in the past few weeks I’ve been drawn into looking in some detail at the emerging process mining space.

Process mining has been an active academic research space for some years (and eminent BPM research leader Prof Wil van der Aalst leads a team at TU/e which has been instrumental in launching commercial projects as well as advancing research in the area).

The Process Mining manifesto, published late last year, is a really good overview of the area and call to action – and one thing it makes clear is that the scope of process mining technology is much broader than process discovery (which is the area that most commentary has focused on so far, with much discussion conflating the two concepts).

According to the manifesto,† there are three applications of process mining – discovery, conformance checking and enhancement:

  • Discovery is about taking event logs and analysing them to produce models of work.
  • Conformance is about comparing existing models of work with evidence from event logs to discover any operational ‘gaps’ between prescribed or recommended practices and actual work in the field.
  • Enhancement is about using data from event logs to enrich the information provided by static models – perhaps by overlaying performance information, for example; or even using historical event information to predict the performance of work currently in progress and suggest ways to optimise it ‘on the fly’.

What’s particularly interesting to me, based on my reading of the manifesto at least, is that the authors (or at least some of them) appear to propose that process mining in its broadest context provides the foundation for a different kind of process management system from the kind many people are familiar with today – one that’s ‘passive’ rather than ‘active’.

This ‘passive system’ is not like today’s BPMSs, which manage processes and the execution of work using those processes through a core co-ordinating application that orchestrates the flow of work between people and systems.

Rather, through ongoing and continuous mining of event logs ‘in the background’, not directly connected to the systems that people use to get work done, such a system would work by detecting the shadows that work casts onto existing IT systems; tracking those shadows in the context of models (discovered or purposely created); and then using that analysis to drive a) management insights into opportunities for improvement and b) operational insights into optimal execution of work.

As the manifesto itself points out, the engineering and research foundations are already in place to make a system like this possible today. Such a system would have the potential to deliver many of the benefits that today’s BPM projects can deliver, but without interposing a new application layer that risks disrupting relationships that people have with their existing working habits and IT systems.

Still, though, I think it’s going to take a few years before such systems gain significant mainstream traction in industry. Why? Because a lot of the practical detail of implementing such a system in industry would require new tools to be built, and the big vendor money is currently being poured into ongoing marketing and improvement of today’s generation of BPMSs; and there are no vendors of any significant size that could release such a platform in the near future without confusing the hell out of its prospect and customer base.

I think we will see systems like this start to be deployed in the coming years, particularly in scenarios where ‘unstructured’ knowledge work is at the heart of the business domain under consideration – but that doesn’t mean process mining is a dead-end: far from it.

I really think we’ll see a lot of real-world deployment of process mining’s discovery application, and quite soon (in the coming months). Why? Because in this context, process mining techniques and technologies help to address an immediate pain point that an established community of industry practitioners have. Specifically, how to quickly discern the actual state of work in a given area of a business to provide a reliable foundation for analysis of improvement opportunities.

I’ll be looking a little more at this ‘discovery’ aspect of process mining in a forthcoming post. In the meantime, I’d love to get your thoughts on this larger question – it’s still an emerging area and I know I am a long way from having all the answers!



Posted in Analytics, Information Management, BPM

11 Responses to Process mining – creating passive management systems?

  1. Hi Neil:

    As far I understood you are narrowing process mining application scope. It’s not necessary to develop an “intelligent” process mining system with lots of out of the box integration. The beauty of process mining is something that is agnostic. Yes. No lock in. If you approach process mining trying to integrate with every possible enterprise system, tomorrow you will have black holes because execution spans today’s assumptions. Process Mining relies on event data and you can’t control where data can be found that show evidence of process execution.

    One of the key challenges of Process Mining is comparing vis-a-vis the current execution patterns and performance (or other perspective)with the starting point. This a ultimate challenge combined with human reasoning can transform the way we want to understand how a company operates.

    For those who are interested in real world process mining application, where is a case study published at BP Trends:

    For those who want to understand how a process mining project is carried:

    For those who want to understand the 7 objection? to process mining:


    Alberto Manuel

    • Neil Ward-Dutton says:

      Maybe we’re talking at cross-purposes here Alberto: I was talking about a “system” in its general sense (a set of interrelated components, procedures and people) rather than the very specific definition that we’ve come to associate with software platforms like BPMSs which are all about close integration of tools and platforms.

  2. Great post, Neil!

    The idea of “passive” systems for process support is intriguing, and has been the subject of a number of research papers even before the Process Mining Manifesto (e.g., see my take on the topic here: In one way or another, researchers always gravitate towards a visionary take on the topic, sketching a “brave new world” scenario where an all-knowing and intelligent AI learns from process observations in the background, and then automatically applies its findings to current operations.

    I think that an “automated learning” approach, i.e. a fully-automated “passive” system, will always have to balance between being overly restrictive on the one hand, and, on the other hand, being eventually useless because its recommendations are mostly common sense. Not that it is not worthwhile to pursue this direction, but that balance is quite hard to strike for the general use case, and is probably best left for university researchers to explore for some time to come.

    I would argue that you can start assembling your very own “passive” system, with tools that are available right now. For process execution, use any system which places no constraints on how processes are executed. To achieve transparency, complement that system with a process mining tool which lets you know how work is executed in detail, on demand.

    The actual change needs to be in the paradigm used, i.e. in the way that process management is understood by stakeholders. Abandon the idea of “controlling” process execution, where constraints and rules are dictated from above to prevent mishaps in execution. Replace it by a “trust and check” model, where knowledge workers enjoy complete freedom. Through periodic process mining analysis, management can spot quality or efficiency problems reliably and early on, and then take appropriate action to prevent it from happening again. This action can take the form of meetings or briefings, to communicate rules and best practices, it can be in the form of explicit rules or constraints implemented in the case management system, or anything else really.

    The current paradigm emphasizes anticipating problems, and preventing them proactively. If you trust in the experience and intelligence of your staff, and in their having the best interests of your company in mind, you can change that paradigm right now, without waiting for other tools to arrive. The actual shift is not a technical one, but is in the mindset of all actors involved, especially management.


    • Neil Ward-Dutton says:

      Interesting stuff Christian!
      Have you got any examples of organisations that have done what you’re outlining here?

      • There are some examples I have seen where organisations are using this approach. However, they were typically coming from a situation where the process was “loosely” supported, and then added monitoring / analysis via process mining to “close the loop” later on.

        I should probably add that my original comment was less of a response to what you suggested in your post, which in my opinion is a very sensible approach. I was more reminded of “pie in the sky” visions of completely-automated BPM lifecycles that I have seen lots of people dream up. My point was that they often try to address a human / organizational problem with a technological solution, which is misguided.

        As I see it, all process support paradigms (from tightly controlled to completely lenient, ACM-style) are useful and have their place. And all of them equally benefit from the analytical insight that process mining (and other analytics) provides.

  3. Hi Neil,

    I completely agree. A key distinction of process mining is that it requires processes simply to be ‘observable’ regardless of how ‘automated’ they are.

    When people mistake ‘observability’ for ‘automation’, then they ask what the benefit is in “simply rediscovering what you have already specified in your BPM system”. They picture an ‘active’ BPM as the required basis for process mining.

    But that’s not the point. For example, there are ERPs where no process rules are configured at all and people can basically do what they want. These processes can still be mined using process mining.

    Even if there are rules, there usually are (have to be) considerable degrees of freedom, which leads to the fact that nobody has a clue what is really going on. Process mining can provide this transparency: By quickly discerning the actual state of work as a foundation for analysis of improvement opportunities it acts as a ‘passive’ BPM approach already today.

  4. Pingback: Observe and Report — Flux Capacitor

  5. I would like to follow up on Anne’s comment. Many ERPs, especially those built in the 1980s, are organised at a functional level. For example; Inventory, MRP, Shop Floor Control Modules.

    But this is not how they are used. ERPs are about supply chains. This is why the SCOR & APQC Frameworks are so useful. They describe how the supply chain processes really work – using different parts of different modules to create an end to end process.

    Process Mining can reveal the actual processes in an ERP.It can show the bottlenecks and the rework. This provides excellent data for those interested in Lean Six Sigma and Constrain Theory. It provides the data for these tools in a timely and efficient manner.

    The Super Project provides the opportunity to deal with heterogeneity & complexity of ERP systems.

    You can create ‘Ontologies’for the various ERPS and use abstractions to make the mined data even easier to study.

    Process Mining can be readily leveraged using other tools and methodologies making it an important part of the BPM toolkit.

    • Neil Ward-Dutton says:

      I absolutely agree about PM being an important part of the BPM toolkit George! I guess the interesting “next question” is to what extent PM can become a continuous and iterative discipline applied across the whole improvement lifecycle…
      Are you involved in PM projects?

      • Process Mining can be applied across the whole improvement lifecycle, when there are event logs available. The interesting link is with ‘Big Data’.

        What’s interesting is that now that ‘Big Data’is becoming more important there will be even more opportunities for process mining.

        Firstly, this kind of data will contain event information (rather than summary data) which will allow for process mining.

        Secondly, all data comes from processes! So, its necessary to understand the processes as well as the data that is created.

  6. Pingback: BPM Quotes of the week « Adam Deane

Leave a comment: