Thank you, Xavier, Following your replies I was able to get all the log statements with specific messages after running the Docplex decision optimization model into IBM Watson Studio. Appreciate the help.
Just a brief explanation of how I achieved this, hope it helps someone in the community.
To get the log statements with the specified message as Xavier mentioned I have added the following line in the payload.
solve_payload = { client.deployments.DecisionOptimizationMetaNames.INPUT_DATA: [
{
"id":"input_data.csv",
"values" : input_data
}
],client.deployments.DecisionOptimizationMetaNames.SOLVE_PARAMETERS:
{ 'oaas.includeInputData': 'false',
'oaas.logAttachmentName': 'log.txt',
'oaas.logTailEnabled': 'true',
'oaas.resultsFormat': 'JSON'},
client.deployments.DecisionOptimizationMetaNames.OUTPUT_DATA: [
{
"id":".*"
}]
}
The following code is used to create a job and get the log statements into a file:
# Creating a job in WML
job_details = client.deployments.create_job(deployment_uid, solve_payload)
#Once the job is completed use the following code to access the complete log statements used in the model
# To view the complete job details
print(job_details)
# To access and write the log statements into a file
import base64 # Log statements are encoded
decoded_log_message = base64.b64decode(job_details['entity']['decision_optimization']['output_data'][3]['content'])
decoded_log_message_str = decoded_log_message.decode("utf-8") #converting bytes into string
#writing the log statements into a file
with open("IBM Cloud Log.txt", "w") as text_file:
text_file.write(decoded_log_message_str)
------------------------------
Anilkumar Lingaraj Biradar
------------------------------
Original Message:
Sent: Fri October 22, 2021 03:24 AM
From: Xavier Nodet
Subject: To view the print or log statements in the log provided in IBM Cloud Watson Machine Learning
Do you mean that you see the CPLEX log, but you don't see the results of the print statements? That seems very strange to me...
With respect to the 'logging' statements, I can't say much. I'd be happy to look into this, but I would need a complete code that reproduces the issue, rather than a screenshot that doesn't show how exactly the 'logging' object is created.
You mention a local notebook. Do you refer to a notebook on your computer from which you launch a WML job? Then you may be interested in the code for the function 'wait_for_job_end' at https://github.com/nodet/dowml/blob/f49fa812af022a4cdb38fba75d546ff2701b4261/src/dowml/dowmllib.py#L827. This code repeatedly fetches the last activity of the job to print it on the screen. You could include a similar code in your notebook so that you get something to look at while the job runs.
------------------------------
Xavier
Original Message:
Sent: Fri October 22, 2021 01:30 AM
From: Anilkumar Lingaraj Biradar
Subject: To view the print or log statements in the log provided in IBM Cloud Watson Machine Learning
Thank you so much, Xavier. That is helpful to get better logs. Currently, I am interested in printing the log statements defined using logging.info() and print statements as mentioned in the image below. I am not able to see this when I run a job for the deployed model in the IBM Watson machine learning. Could you please help me on how to see these statements either on the IBM job log or on the local notebook? Appreciate the help.
------------------------------
Anilkumar Lingaraj Biradar
Original Message:
Sent: Thu October 21, 2021 02:17 AM
From: Xavier Nodet
Subject: To view the print or log statements in the log provided in IBM Cloud Watson Machine Learning
Hi Anilkumar,
In order for docplex to print the CPLEX log, you need to set the 'log_output' parameter in the Model constructor to 'True'. The documentation is at http://ibmdecisionoptimization.github.io/docplex-doc/mp/docplex.mp.model.html, and here's an example:
mdl = Model(name='my_model', log_output=True)
With that, the CPLEX log should appear and you can see it in the ['decision_optimization']['solve_state']['latest_engine_activity'] value of the WML job details. If the log gets long, only the last part will appear there. In order to save all your job's log as an output of the job, you need to instruct the job processor to save it by including the following in your job payload:
"decision_optimization": { "solve_parameters": { "oaas.logAttachmentName": "log.txt", "oaas.logTailEnabled": "true" },
And then you need to retrieve the content of the file "log.txt", as you would for any other job output. A way to systematically save all the job outputs as inline data is to include the following in your payload:
"decision_optimization": { "output_data": [{'id': '.*'}]
If you would rather save these outputs as data_assets in your deployment space, this would be:
"decision_optimization": "output_data_references": [{ "connection": {}, "id": ".*", "location": { "name": "${job_id}/${attachment_name}" }, "type": "data_asset" }],
You will find at https://github.com/nodet/dowml/blob/f49fa812af022a4cdb38fba75d546ff2701b4261/src/dowml/dowmllib.py#L368 some code to decode (when using inline data) or download (when using assets) these logs.
------------------------------
Xavier
Original Message:
Sent: Wed October 20, 2021 08:20 PM
From: Anilkumar Lingaraj Biradar
Subject: To view the print or log statements in the log provided in IBM Cloud Watson Machine Learning
I have deployed the decision optimization model to the IBM Watson Machine Learning, I have added multiple print and log statements to know the progress as mentioned in the image below. But I am not able to see this in the IBM cloud log, could I know whether there is any way to check the log or print statements. So I would be able to know the progress of the model.
------------------------------
Anilkumar Lingaraj Biradar
------------------------------
#DecisionOptimization