Watson Studio

A common request I get is how to extract coefficients from regression and logistic regression model nuggets in Modeler. This is usually possible for any model that generates pmml, by exporting the pmml and then using the xml source node to parse the pmml and bring the coefficients back into Modeler as data. I'll first show a simple example of how you can do that with scripting, then i will show an alternative that is now possible with Modeler 17, where you don't have to actually generate the pmml at all.

This is my stream (adapted from one of the shipped demo streams), where I am building multiple logistic regression models:



 

and I would like to work extract the Coefficients generated for each model, but in an automated fashion (ie I don't want to manually copy and paste the values from the tables):



The script editor is accessed by going to Tools...Stream Properties...Execution



The script below locates and executes each of the Logistic build nodes in turn to generate a fresh model nugget, then exports the pmml for each model to a file location specified in the script:

from modeler.api import FileFormat

stream = modeler.script.stream()
session = modeler.script.session()
tr = session.getTaskRunner()

targets = [u"credit1",u"credit2"]

FOLDER = "C:/temp/models/"

def runAndExport(target):
buildnode = stream.findByType("logreg", target)
results = []
buildnode.run(results)
modeloutput = results[0]
filename = FOLDER + target + ".xml"
# print "Exporting", modeloutput, "to", filename
tr.exportModelToFile(modeloutput, filename, FileFormat.XML)
# Good practise to delete the model output from the palette
session.getModelOutputManager().removeModelOutput(modeloutput)

for target in targets:
runAndExport(target)

 

Once i have written or pasted the script into the script editor window, it can be tested by clicking the "Run this Script" button:



The result should be that the pmml files are generated on the file system in the directory specified in the script - but you still then need to parse the pmml to obtain the coefficients.



 

With Modeler 17, we added support for a pmml content model, which means the pmml can be directly manipulated within the python script.  The script below uses this new method.

 

from modeler.api import FileFormat

stream = modeler.script.stream()
session = modeler.script.session()

targets = [u"credit1",u"credit2"]

def runAndExtract(target):
buildnode = stream.findByType("logreg", target)
results = []
buildnode.run(results)
modeloutput = results[0]
cm = modeloutput.getContentModel("PMML")
values = cm.getValuesList("//ParamMatrix/PCell", ["parameterName", "beta"], False)
# Good practise to delete the model output from the palette
session.getModelOutputManager().removeModelOutput(modeloutput)
return values

for target in targets:
print "Coefficients for", target, "=", runAndExtract(target)

 

The result of executing this script is that the required coefficients can now be referenced within the script and used however they are needed - in this case, they are just being printed to the debug window.



 

Some tips for adapting this script:

- If you have a different model type: You would need to change the "logreg" to the model type. If you don't know how to refer to the model type, you could CTRL-C the node, then CTRL-SHIFT-P (described in my previous post) within the script editor to learn how to refer to that model type.

- Changing the name/number of models: Make sure each model is given a unique name (ie don't use Auto naming), and just refer to them in the list of targets, eg targets = [u"credit1",u"credit2", u"credit3", u"modelxyz"]

 

 

For more information on accessing results through these content models, refer to the documentation:

http://www-01.ibm.com/support/knowledgecenter/SS3RA7_17.0.0/clementine/scripting_accessresults.dita

 

Thanks to Julian Clinton who provided the scripts for this post.



#scripting
#SPSSModeler
#WatsonStudio