Decision Management (ODM,ADS)

I'm looking to possibly determine what approaches may be available or applicable to provide assurance or measure the effectiveness of a rule or combination of rules within a ruleset in ODM.  For some background, we have an existing bespoke decision management tool that allows our analysts to see what rules matched based on which specific rule criteria and what values exist within that rule criteria matched the values in input document ( i.e. the input BoM ).  This visibility allows our analysts to determine the effectiveness/assurance of a specific rule in determining the correct/desired outcome for which the rule was developed and hence its effectiveness.  

We are looking to replace the bespoke decision management system with an ODM based solution. As such we need to be able to deliver and demonstrate a similar level of visibility of rule execution to determine effectiveness/ assurance to our analysts.  It's our understanding that this type of rule, rule criteria and data matching introspection granularity is not available from within ODM.  As such, I'm interested in what approaches other ODM users have in place to enable Decision/Assurance/Effectiveness Measurement?

Thanks

  Peter

Peter

There are some great tools for doing this in ODM and we regularly use them when building solutions for customers as this kind of transparency and assurance is a common request. There are generally several elements

• Testing using the ODM test framework to make sure that boundary conditions cause expected outcomes
• Debugging specific issues using rule tracing and rule logs
• The ODM simulation reports allow for the assessment of aggregate impact of proposed changes, before they have been deployed
• Interim variables can be captured in the XOM and then saved to create a decision log

We use decision modeling to capture requirements and design the solution. Decision modeling, using the DMN notation, decomposes decisions into a network of sub-decisions, allows logic to be normalized (similarly to how you would normalize data) and ensures the rules are structured the way the business users think of their problem.

With a decision-model based approach, the testing framework can be applied effectively at each level in the decision model, allowing for thorough testing without an explosion of test cases. The model clearly defines the aggregates for the simulation report, so business owners can see how individual rule changes affect each element of their decision-making. And it ensures that the interim variables all have business meaning so the decision log created can be used to explain a decision and assure its accuracy.

Drop me a line or connect on LinkedIn if you'd like to discuss further or see a demo of any of this. We an IBM partner and do a ton of work with ODM, almost all of which has these kind of transparency and assurance requirements. I'd be happy to share what we have learned.

I'm looking to possibly determine what approaches may be available or applicable to provide assurance or measure the effectiveness of a rule or combination of rules within a ruleset in ODM.  For some background, we have an existing bespoke decision management tool that allows our analysts to see what rules matched based on which specific rule criteria and what values exist within that rule criteria matched the values in input document ( i.e. the input BoM ).  This visibility allows our analysts to determine the effectiveness/assurance of a specific rule in determining the correct/desired outcome for which the rule was developed and hence its effectiveness.  

We are looking to replace the bespoke decision management system with an ODM based solution. As such we need to be able to deliver and demonstrate a similar level of visibility of rule execution to determine effectiveness/ assurance to our analysts.  It's our understanding that this type of rule, rule criteria and data matching introspection granularity is not available from within ODM.  As such, I'm interested in what approaches other ODM users have in place to enable Decision/Assurance/Effectiveness Measurement?

Thanks

  Peter

I'm looking to possibly determine what approaches may be available or applicable to provide assurance or measure the effectiveness of a rule or combination of rules within a ruleset in ODM.  For some background, we have an existing bespoke decision management tool that allows our analysts to see what rules matched based on which specific rule criteria and what values exist within that rule criteria matched the values in input document ( i.e. the input BoM ).  This visibility allows our analysts to determine the effectiveness/assurance of a specific rule in determining the correct/desired outcome for which the rule was developed and hence its effectiveness.  

We are looking to replace the bespoke decision management system with an ODM based solution. As such we need to be able to deliver and demonstrate a similar level of visibility of rule execution to determine effectiveness/ assurance to our analysts.  It's our understanding that this type of rule, rule criteria and data matching introspection granularity is not available from within ODM.  As such, I'm interested in what approaches other ODM users have in place to enable Decision/Assurance/Effectiveness Measurement?

Thanks

  Peter

Hi Peter,

From a product management standpoint, getting to the granularity of which condition(s) triggered the execution of a particular rule has not been a common request. 
ODM users tend to concentrate of building testing suite, checking that the expected outcome is met for each scenario, and possibly ensure that a particular rule got executed for a given scenario.
If it's important to capture additional information when a rule is executed, rule authors can also add additional log print as part of the action section of the rule. 
Another option is to use the rule debugger in Rule Designer and step through the execution and variables, when it's difficult to understand why an expected outcome is not met.

I'm looking to possibly determine what approaches may be available or applicable to provide assurance or measure the effectiveness of a rule or combination of rules within a ruleset in ODM.  For some background, we have an existing bespoke decision management tool that allows our analysts to see what rules matched based on which specific rule criteria and what values exist within that rule criteria matched the values in input document ( i.e. the input BoM ).  This visibility allows our analysts to determine the effectiveness/assurance of a specific rule in determining the correct/desired outcome for which the rule was developed and hence its effectiveness.  

We are looking to replace the bespoke decision management system with an ODM based solution. As such we need to be able to deliver and demonstrate a similar level of visibility of rule execution to determine effectiveness/ assurance to our analysts.  It's our understanding that this type of rule, rule criteria and data matching introspection granularity is not available from within ODM.  As such, I'm interested in what approaches other ODM users have in place to enable Decision/Assurance/Effectiveness Measurement?

Thanks

  Peter

I'm looking to possibly determine what approaches may be available or applicable to provide assurance or measure the effectiveness of a rule or combination of rules within a ruleset in ODM.  For some background, we have an existing bespoke decision management tool that allows our analysts to see what rules matched based on which specific rule criteria and what values exist within that rule criteria matched the values in input document ( i.e. the input BoM ).  This visibility allows our analysts to determine the effectiveness/assurance of a specific rule in determining the correct/desired outcome for which the rule was developed and hence its effectiveness.  

We are looking to replace the bespoke decision management system with an ODM based solution. As such we need to be able to deliver and demonstrate a similar level of visibility of rule execution to determine effectiveness/ assurance to our analysts.  It's our understanding that this type of rule, rule criteria and data matching introspection granularity is not available from within ODM.  As such, I'm interested in what approaches other ODM users have in place to enable Decision/Assurance/Effectiveness Measurement?

Thanks

  Peter

------------------------------
Peter Brennan
------------------------------