Hi Alain,
I didn't modify any lines of the original file
Lines for modeltar = tarfile.open("model.tar.gz", "w:gz")
tar.add(cplex_file, arcname=cplex_file, filter=reset)
tar.close()
print("Storing model")
model_metadata = {
client.repository.ModelMetaNames.NAME: model_name,
client.repository.ModelMetaNames.DESCRIPTION: model_name,
client.repository.ModelMetaNames.TYPE: "do-cplex_12.10",
client.repository.ModelMetaNames.
SOFTWARE_SPEC_UID: client.software_specifications.get_uid_by_name("do_12.10")
}
model_details = client.repository.store_model(model='./model.tar.gz', meta_props=model_metadata)
model_uid = client.repository.get_model_uid(model_details)
print(model_uid)
-----
Store_model call I use these metadata (same as orginal file)
metadata = {
client.spaces.ConfigurationMetaNames.NAME: space_name,
client.spaces.ConfigurationMetaNames.DESCRIPTION: space_name + ' description',
client.spaces.ConfigurationMetaNames.STORAGE: {
"type": "bmcos_object_storage",
"resource_crn": cos_resource_crn
},
client.spaces.ConfigurationMetaNames.COMPUTE: {
"name": "existing_instance_id",
"crn": instance_crn
}
}
------------------------------
Frederic Lacroze
------------------------------
Original Message:
Sent: Wed November 04, 2020 08:05 AM
From: Alain Chabrier
Subject: docplex python
Good progress, indeed.
At the time the model is created on WML, one should set the software specs, which software to use, etc.
Can be a defautl DO software specs, but you can also customize it.
By default you should have something like:
model_metadata = {
client.repository.ModelMetaNames.NAME: model_name,
client.repository.ModelMetaNames.DESCRIPTION: model_name,
client.repository.ModelMetaNames.TYPE: "do-cplex_12.10",
client.repository.ModelMetaNames.SOFTWARE_SPEC_UID: client.software_specifications.get_uid_by_name(
"do_12.10")
}
model_details = client.repository.store_model(model='./model.tar.gz', meta_props=model_metadata)
Are you sure you havent'e removed the SOFTWARE_SPEC line?
Are you sure you are using these meta data for you store_model call?
------------------------------
Alain Chabrier
Original Message:
Sent: Wed November 04, 2020 07:41 AM
From: Frederic Lacroze
Subject: docplex python
Hi Alain,
Before I have a ML with Lite Plan, now I have a ML with Plan v2 standard
I launch cplexrunonwml2.py and with your help I pass some (big) steps
CPLEX file is diet.lp
Creating WML Client
space_id: xxxxxx
Getting deployment
Creating model
Storing model
2020-11-04 13:22:19,147 - ibm_watson_machine_learning.wml_client_error - WARNING - No "meta_props.software_spec" provided.
Traceback (most recent call last):
File "C:\Users\frede\Documents\CPLEX\cplexrunonwml-master\cplexrunonwmlv2.py", line 227, in <module>
main(sys.argv[1:])
File "C:\Users\frede\Documents\CPLEX\cplexrunonwml-master\cplexrunonwmlv2.py", line 142, in main
model_details = client.repository.store_model(model='./model.tar.gz', meta_props=model_metadata)
File "C:\Python38\lib\site-packages\ibm_watson_machine_learning\repository.py", line 407, in store_model
return self._client._models.store(model, meta_props=meta_props, training_data=training_data, training_target=training_target, pipeline=pipeline, feature_names=feature_names, label_column_names=label_column_names,subtrainingId=subtrainingId)
File "C:\Python38\lib\site-packages\ibm_watson_machine_learning\models.py", line 1456, in store
saved_model = self._publish_from_file(model=model, meta_props=meta_props, training_data=training_data, training_target=training_target,ver=version,artifactid=artifactid)
File "C:\Python38\lib\site-packages\ibm_watson_machine_learning\models.py", line 1011, in _publish_from_file
return self._publish_from_archive(model, meta_props)
File "C:\Python38\lib\site-packages\ibm_watson_machine_learning\models.py", line 1182, in _publish_from_archive
payload = self._create_cloud_model_payload(meta_props)
File "C:\Python38\lib\site-packages\ibm_watson_machine_learning\models.py", line 2159, in _create_cloud_model_payload
self._validate_meta_prop(meta_props, self.ConfigurationMetaNames.SOFTWARE_SPEC_UID, STR_TYPE, True)
File "C:\Python38\lib\site-packages\ibm_watson_machine_learning\wml_resource.py", line 139, in _validate_meta_prop
WMLResource._validate_type(meta_props[name], u'meta_props.' + name, expected_type, mandatory)
File "C:\Python38\lib\site-packages\ibm_watson_machine_learning\wml_resource.py", line 122, in _validate_type
raise MissingValue(el_name)
ibm_watson_machine_learning.wml_client_error.MissingValue: No "meta_props.software_spec" provided.
In my bucket "essaidocplex" I have diet.lp
Thank you
------------------------------
Frederic Lacroze
Original Message:
Sent: Wed November 04, 2020 06:41 AM
From: Alain Chabrier
Subject: docplex python
Hi,
You cannot use v1 or v2 of the script per your convenience, you should use the one corresponding to your WML instance version.
Can you check your instance version? Whihc plan is it? Lite Standard or Enterprise? When was it created?
if the instance is v2 (created after September or migrated since September), then you must use the v2 script, which does not use WMl instance_id. The apikey you provide is your personal apikey and not an WML specific apikey.
You might look st this post for comments on using python client with v2 instances: Migrate your python code for DO in WML v2 instances| Medium | remove preview |
 | | Migrate your python code for DO in WML v2 instances | | A new version of Watson Machine Learning (WML) instances was deployed in early September which includes some enhancements and changes. Please refer to my previous post for details on the REST APIs changes. An easy to use option for Data Scientists to deploy and integrate Decision Optimization (DO) models in WML is to use the WML Python client. | | View this on Medium > |
|
|
Details on where to get the apikey are provided here:
Decision Optimization for Watson Machine Learning V4 final API| Medium | remove preview |
 | | Decision Optimization for Watson Machine Learning V4 final API | | Decision Optimization (DO) has been available in Watson Machine Learning (WML) for almost one year now. As optimization models requires some specific asynchronous usage, during this period, some specific new v4 APIs from WML have been necessary to deploy and use optimization models. | | View this on Medium > |
|
|
Alain
------------------------------
Alain Chabrier
Original Message:
Sent: Wed November 04, 2020 05:47 AM
From: Frederic Lacroze
Subject: docplex python
Thank you Alain for this answer (merci Alain)
I saw how to see the crn of COS
I try now cplexrunonwml.py.
For creating WML Client
client = WatsonMachineLearningAPIClient(wml_credentials)
I need 3 informations : apikey, instance_id, url
for apikey, I use the API key defined in IAM
for url, I use "https://eu-gb.ml.cloud.ibm.com"
but I don't understand howto find instance_id ? (I see service id but instance ID not ...) What is this instance ? Instance of what ?
I try alternatively the ID (ID global unique) of my 2 Services ressources Machine : Learning-1w and Watson Studio-gl, but It failed
I get an error with instruction client = WatsonMachineLearningAPIClient(wml_credentials)
2020-11-04 11:21:21,699 - watson_machine_learning_client.wml_client_error - WARNING - Getting instance details failed. (GET https://eu-gb.ml.cloud.ibm.com/v3/wml_instances/xxxxxxxxxxxxxxxx)
Status code: 403, body: {"trace":"xxxxxxf","errors":[{"code":"forbidden","message":"Authorized user does not have an access to this resource"}]}
I search "instance ID" but no result.
Thank you for you help
------------------------------
Frederic Lacroze
Original Message:
Sent: Wed November 04, 2020 03:08 AM
From: Alain Chabrier
Subject: docplex python
Hi,
thanks for you quesiton and using the cpelxrunonwml2 script.
Some questions:
- are you sure your WML instance is a v2 instance? otherwise use cplexrunonwml.py
- where did you get the COS crn?
- where did you get the service crn?
I think the issue is with the service_instance_crn, which seems to be the same as COS crn in your code.
It should be the crn of the WML service instance.
You can get it on IBM CLoud, by clicking on the service in the dashboard (but not on the name). In the right hand side panel, you can get the CRN.
It should start with something like:
'crn:v1:bluemix:public:pm-20:us-south:a/05b089c618b6d54d15a9116170521ef8:xxxxxxxxxxxxxxx3-3c5b-4dfe-ba54-988321ce6277::'
Note the pm-20 (for WML) instaed of COS.
To create a new deployement space, COS is required for storage of the assets of this space, and WML is required to be associated to the space. Then any compute usage on the space will automatically use this WML instance.
Hope this helps,
Alain
------------------------------
Alain Chabrier
Original Message:
Sent: Mon November 02, 2020 11:43 AM
From: Frederic Lacroze
Subject: docplex python
Hello Community,
I used Dropsolve before and now try to use the cplexrunonwml2.py exemple provided by IBM.
Concerning the storage I am not sure of the values :
- cos_resource_crn
- instance_crn
I have created a COS instance and a compartiment named 'essaidocplex'
cos_resource_crn = "crn:v1:bluemix:public:cloud-object-storage:global:a/8b06565251174ba1a52db031b3229108:xxxxxxxx-fbab-4191-82e1-3459479b830b:bucket:essaidocplex"
instance_crn = "crn:v1:bluemix:public:cloud-object-storage:global:a/8b06565251174ba1a52db031b3229108:xxxxxxxx-fbab-4191-82e1-3459479b830b::"
space = client.spaces.store(meta_props=metadata)
An error occur at this line
ibm_watson_machine_learning.wml_client_error - WARNING - Failure during creating new spaces. (POST https://api.eu-gb.dataplatform.cloud.ibm.com/v2/spaces)
Status code: 400, body: {"errors":[{"code":"SPACES0016E","message":"Provided CRN 'crn:v1:bluemix:public:cloud-object-storage:global:a/8b06565251174ba1a52db031b3229108:xxxxxxxx-fbab-4191-82e1-3459479b830b::' is not valid in context where it is used.
Any idea about this error message ?
I try to check bucket 'essaidocplex' properties by doing
from cos_config.resource_configuration_v1 import ResourceConfigurationV1
api_key = "tDX5XPojFFjyoRAFKV5Jvv0aBqTPnZj3daQN1P8cg04M"
bucket = "essaidocplex"
client = ResourceConfigurationV1(iam_apikey=api_key)
config = client.get_bucket_config(bucket)
print(config)
I get
{
"result": {
"name": "essaidocplex",
"service_instance_id": "xxxxxxxx-fbab-4191-82e1-3459479b830b",
"time_created": "2020-10-27T20:34:59.843Z",
"time_updated": "2020-10-27T20:34:59.843Z",
"object_count": 2,
"bytes_used": 1866,
"crn": "crn:v1:bluemix:public:cloud-object-storage:global:a/8b06565251174ba1a52db031b3229108:xxxxxxxx-fbab-4191-82e1-3459479b830b:bucket:essaidocplex",
"service_instance_crn": "crn:v1:bluemix:public:cloud-object-storage:global:a/8b06565251174ba1a52db031b3229108:xxxxxxxx-fbab-4191-82e1-3459479b830b::"
},
"headers": {
"_store": {
"date": [
"Date",
"Mon, 02 Nov 2020 23:12:38 GMT"
],
"ibm-cos-config-api-ver": [
"ibm-cos-config-api-ver",
"1.0"
],
"accept-ranges": [
"Accept-Ranges",
"bytes"
],
"ibm-cos-request-id": [
"ibm-cos-request-id",
"7be403cc-cf4b-4155-adb1-dd0f7ccabea4"
],
"etag": [
"ETag",
"NO_ETAG"
],
"content-type": [
"Content-Type",
"application/json"
],
"content-length": [
"Content-Length",
"503"
]
}
},
"status_code": 200
}
Thank you for help
------------------------------
Frederic Lacroze
------------------------------
#DecisionOptimization