Process Model Realism Measuring Implicit Realism 8/09/2014 dr. Benot Depaire Research Trigger Number of possible execution paths explode with AND-construct with n activities in parallel: X = n! 8/09/2014 dr. Benot Depaire n
X 3 6 4 24 5 120 6
720 7 5040 8 40320 Research Trigger 8/09/2014 A B
C A C B B A C B
C A C A B C B A dr. Benot Depaire
Research Trigger A B C A C B C
A B C B A 8/09/2014 dr. Benot Depaire Research Trigger A
B C A C B C A B
C B A R1: All activities must occur 8/09/2014 dr. Benot Depaire Research Trigger A B
C A C B C A B C
B A R1: All activities must occur R2: A must occur before B, unless we start with C 8/09/2014 dr. Benot Depaire Research Trigger When are we generalizing too much? Context: Capture the true underlying process
Is the discovered model realistic? 8/09/2014 dr. Benot Depaire Process Realism Explicit Realism: All observed behavior should be in the model Implicit Realism: Only the realistic unobserved behavior should be in the model 8/09/2014 dr. Benot Depaire
Assumptions There are no measurement errors in the log There are no infinite loops possible in the process The fitness of a discovered model = 1 (All execution paths are equiprobable) 8/09/2014 dr. Benot Depaire Process Realism Implicit Realism: Only the realistic unobserved behavior should be in the model.
8/09/2014 dr. Benot Depaire Process Realism Implicit Realism: Only the realistic unobserved behavior should be in the model. Implicit Realism Measure: How confident can we be that the unobserved behavior is realistic? 8/09/2014 dr. Benot Depaire
Implicit Realism Measure m = number of paths in process M n = number of paths in log L xi = frequency of path i in log L Pi = Probability of path i occurring in L u = # unobserved paths of M in L TM(L) = statistic to determine u 8/09/2014 dr. Benot Depaire Implicit Realism Measure IR(L,M) = P[TM(L) >= u | M, n] IR(L,M): Probability that a model M would generate a log L with at least u missing paths (given n)
The lower IR(L,M), the less confident we can be that M actually produced L because M contains too much unobserved behavior! (for a given n) 8/09/2014 dr. Benot Depaire Implicit Realism Measure 8/09/2014 dr. Benot Depaire Empirical Illustration
8/09/2014 dr. Benot Depaire Assumptions There are no measurement errors in the log There are no infinite loops possible in the process The fitness of a discovered model = 1 (All execution paths are equiprobable) 8/09/2014 dr. Benot Depaire Conclusions
IR Measure has a very precise and intuitive interpretation Current IR Measure should be used for evaluation, not comparison! 8/09/2014 dr. Benot Depaire Process Model Realism Q&A 8/09/2014 dr. Benot Depaire Implicit Realism
Precision To what extent does the model NOT contain too much behavior (no underfitting) Generalization To what extent does the model NOT contain too little behavior (no overfitting) 8/09/2014 dr. Benot Depaire Implicit Realism Precision To what extent does the model NOT contain too much behavior (no underfitting) To what extent does the model ONLY contain
observed behavior Generalization To what extent does the model NOT contain too little behavior (no overfitting) To what extent does the model contain unobserved behavior 8/09/2014 dr. Benot Depaire