Conformance checking
Conformance checking is a process mining technique that compares an existing process model with an event log of the same process.[1] It can be used to check if reality, as recorded in the log, conforms to the model and vice versa.
For instance, there may be a process model indicating that purchase orders of more than one million Euro require two checks. Analysis of the event log will show whether this rule is followed or not. Another example is the checking of the socalled “four-eyes” principle stating that particular activities should not be executed by one and the same person. By scanning the event log using a model specifying these requirements, one can discover potential cases of fraud. Hence, conformance checking may be used to detect, locate and explain deviations, and to measure the severity of these deviations.
Overview
Unlike process discovery, conformance checking takes both a model and event log as a starting point. While conducting a conformance check the behavior of a process model and the behavior recorded in an event log are compared to find commonalities and discrepancies. Such analysis may result in global conformance measures (e.g., 85% of the cases in the event log can be replayed by the model) and local diagnostics (e.g., activity x was executed 15 times although this was not allowed according to the model).
The interpretation of non-conformance depends on the purpose of the model:
- If the model is intended to be descriptive, discrepancies between model and log indicate that the model needs to be improved to capture reality better.
- If the model is normative, then such discrepancies may be interpreted in two ways: they may expose undesirable deviations (i.e., conformance checking signals the need for a better control of the process). or may reveal desirable deviations (i.e., workers may deviate to serve the customers better or to handle circumstances not foreseen by the process model).
Techniques and Metrics
Most conformance checking techniques are based on the principle of Replay, i.e., the event log is replayed on the process model. For example, while replaying an event log on a Petri net, one can count the number of missing and remaining tokens. Alternatively, one can try to optimize the mapping of traces onto models by introducing costs associated to skipping, ignoring, or swapping events in the log and/or model.
Typical conformance metrics are:
- Fitness. A model with good fitness allows for the behavior seen in the event log. A model has a perfect fitness if all traces in the log can be replayed by the model from beginning to end. There are various ways of defining fitness. It can be defined at the case level, e.g., the fraction of traces in the log that can be fully replayed. It can also be defined at the event level, e.g., the fraction of events in the log that are indeed possible according to the model.
- Simplicity. The simplicity dimension refers to Occam’s Razor. In the context of process mining this means that the simplest process model that can explain the behavior seen in the log, is the best model. The complexity of the model could be defined by the number of nodes and arcs in the underlying graph. Also more sophisticated metrics can be used, e.g., metrics that take the structuredness or entropy of the model into account.
- Fitness and simplicity alone are not adequate as metrics. The process model may be overfitting or underfitting.
References
Further reading
- Aalst, W. van der (2011). Process Mining: Discovery, Conformance and Enhancement of Business Processes. Springer Verlag, Berlin (ISBN 978-3-642-19344-6).
- A. Rozinat and Aalst, W. van der (2008). Conformance Checking of Processes Based on Monitoring Real Behavior. Information Systems, 33(1):64–95.
- A. Adriansyah, B.F. van Dongen, and Aalst, W. van der (2010). Towards Robust Conformance Checking. In J. Su and M. zur Muehlen, editors, BPM 2010 Workshops, Proceedings of the Sixth Workshop on Business Process Intelligence (BPI2010), Lecture Notes in Business Information Processing. Springer-Verlag, Berlin.
- J. Munoz-Gama and J.Carmona (2010). A Fresh Look at Precision in Process Conformance, IN BPM 2010, Lecture Notes in Computer Science, Volume 6336, 211-226.
External links
- Process mining research at Eindhoven University of Technology.