Cloud Pak for Data

Come for answers. Stay for best practices. All we’re missing is you.

View Only

Back to discussions

Expand all | Collapse all

Using Jupyter Notebooks on CP4D

1. Using Jupyter Notebooks on CP4D

MARK JEYNES

Posted 10/17/21 08:30 PM

I'm following the short tutorial on using Jupyter Notebooks on CP4D, published here

Build models using Jupyter Notebooks in IBM Watson Studio

At the step where I should want to add the Notebook to my deployment space I select 'add to space' but Notebook is not one of the available asset types.

I would like to know what I may have done incorrectly and how to resolve this please

IBM Developer

remove preview

Build models using Jupyter Notebooks in IBM Watson Studio

Predict customer churn using IBM Watson Studio ranging from a semi-automated approach using the Model Builder, a diagrammatic approach using SPSS Modeler Flows to a fully programmed style using Jupyter notebooks.

View this on IBM Developer >

------------------------------
MARK JEYNES
------------------------------

#CloudPakforDataGroup

2. RE: Using Jupyter Notebooks on CP4D

Like
David Castello
Posted 01/13/22 10:46 AM

Reply
I did the same tutorial. I was able to run the cells in the notebook, until the cell with the following code:

published_model = client.repository.store_model(
model=gbc,
meta_props=metadata)

------------------------------
David Castello
------------------------------
3. RE: Using Jupyter Notebooks on CP4D

Like
David Castello
Posted 01/13/22 11:00 AM

Reply
I am doing the same tutorial. I run the cells of the notebook without errors, until the cell with the followinog code:

published_model = client.repository.store_model(
model=gbc,
meta_props=metadata)

The response was:

Traceback (most recent call last):
File "/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/mlrepositoryclient/model_collection.py", line 254, in _save_scikit_pipeline_model
model_artifact = self._create_pipeline_model(artifact, query_param)
File "/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/mlrepositoryclient/model_collection.py", line 514, in _create_pipeline_model
model_artifact = self._create_pipeline_model_v4_cloud(model_artifact, query_param)
File "/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/mlrepositoryclient/model_collection.py", line 1152, in _create_pipeline_model_v4_cloud
model_output = self.repository_api.ml_assets_model_creation_v4_cloud(model_input, query_param,headers)
File "/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/swagger_client/apis/repository_api.py", line 301, in ml_assets_model_creation_v4_cloud
(data) = self.ml_assets_model_creation_v4_cloud_with_http_info(models_input,query_param, headers, **kwargs)
File "/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/swagger_client/apis/repository_api.py", line 384, in ml_assets_model_creation_v4_cloud_with_http_info
_return_http_data_only=params.get('_return_http_data_only'))
File "/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/swagger_client/api_client.py", line 389, in call_api
response_type, auth_settings, callback, _return_http_data_only)
File "/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/swagger_client/api_client.py", line 209, in __call_api
post_params=post_params, body=body)
File "/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/swagger_client/api_client.py", line 425, in request
body=body)
File "/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/swagger_client/rest.py", line 234, in POST
body=body)
File "/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/swagger_client/rest.py", line 202, in request
raise ApiException(http_resp=r)
ibm_watson_machine_learning.libs.repo.swagger_client.rest.ApiException: (500)
Reason: Internal Server Error
HTTP response headers: HTTPHeaderDict({'Date': 'Wed, 12 Jan 2022 02:33:17 GMT', 'Content-Type': 'application/json', 'Content-Length': '455', 'Connection': 'keep-alive', 'server-timing': 'intid;desc=38aef9113ae9644d, intid;desc=38aef9113ae9644d', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'CF-Cache-Status': 'DYNAMIC', 'Expect-CT': 'max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"', 'Server': 'cloudflare', 'CF-RAY': '6cc2faa9aed30b86-DFW'})
HTTP response body: {
"trace": "41cf06b8b7bef33c55735940744bf480",
"errors": [{
"code": "unknown_error",
"message": "An unknown error occurred (Illegal URI reference: Invalid input ' ', expected '/', 'EOI', '#', '?' or pchar (line 1, column 32): /v2/software_specifications/Not Found?space_id=8e990cec-0f27-42a7-9a78-c955cc0652c7\n ^)",
"more_info": "https://cloud.ibm.com/apidocs/machine-learning"
}],
"status_code": "500"
}

---------------------------------------------------------------------------
ApiException Traceback (most recent call last)
/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/models.py in _publish_from_object_cloud(self, model, meta_props, training_data, training_target, pipeline, feature_names, label_column_names, project_id)
2486 saved_model = self._client.repository._ml_repository_client.models.save(model_artifact,
-> 2487 query_param=query_param_for_repo_client)
2488 return self.get_details(u'{}'.format(saved_model.uid))

/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/mlrepositoryclient/model_collection.py in save(self, artifact, query_param)
212 if artifact.meta.prop(MetaNames.TYPE).startswith("scikit-learn"):
--> 213 return self._save_scikit_pipeline_model(artifact, query_param)
214 elif artifact.meta.prop(MetaNames.TYPE).startswith("xgboost"):

/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/mlrepositoryclient/model_collection.py in _save_scikit_pipeline_model(self, artifact, query_param)
265 print(traceback.format_exc())
--> 266 raise e
267

/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/mlrepositoryclient/model_collection.py in _save_scikit_pipeline_model(self, artifact, query_param)
253 if artifact.uid is None:
--> 254 model_artifact = self._create_pipeline_model(artifact, query_param)
255 else:

/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/mlrepositoryclient/model_collection.py in _create_pipeline_model(self, model_artifact, query_param)
513 def _create_pipeline_model(self, model_artifact, query_param=None):
--> 514 model_artifact = self._create_pipeline_model_v4_cloud(model_artifact, query_param)
515 return model_artifact

/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/mlrepositoryclient/model_collection.py in _create_pipeline_model_v4_cloud(self, model_artifact, query_param)
1151
-> 1152 model_output = self.repository_api.ml_assets_model_creation_v4_cloud(model_input, query_param,headers)
1153 if model_output is not None:

/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/swagger_client/apis/repository_api.py in ml_assets_model_creation_v4_cloud(self, models_input, query_param, headers, **kwargs)
300 else:
--> 301 (data) = self.ml_assets_model_creation_v4_cloud_with_http_info(models_input,query_param, headers, **kwargs)
302 return data

/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/swagger_client/apis/repository_api.py in ml_assets_model_creation_v4_cloud_with_http_info(self, models_input, query_param, headers, **kwargs)
383 callback=params.get('callback'),
--> 384 _return_http_data_only=params.get('_return_http_data_only'))
385

/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/swagger_client/api_client.py in call_api(self, resource_path, method, path_params, query_params, header_params, body, post_params, files, response_type, auth_settings, callback, _return_http_data_only)
388 body, post_params, files,
--> 389 response_type, auth_settings, callback, _return_http_data_only)
390 else:

/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/swagger_client/api_client.py in __call_api(self, resource_path, method, path_params, query_params, header_params, body, post_params, files, response_type, auth_settings, callback, _return_http_data_only)
208 headers=header_params,
--> 209 post_params=post_params, body=body)
210

/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/swagger_client/api_client.py in request(self, method, url, query_params, headers, post_params, body)
424 post_params=post_params,
--> 425 body=body)
426 elif method == "PUT":

/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/swagger_client/rest.py in POST(self, url, headers, query_params, post_params, body)
233 post_params=post_params,
--> 234 body=body)
235

/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/libs/repo/swagger_client/rest.py in request(self, method, url, query_params, headers, body, post_params)
201 if r.status not in range(200, 206):
--> 202 raise ApiException(http_resp=r)
203

ApiException: (500)
Reason: Internal Server Error
HTTP response headers: HTTPHeaderDict({'Date': 'Wed, 12 Jan 2022 02:33:17 GMT', 'Content-Type': 'application/json', 'Content-Length': '455', 'Connection': 'keep-alive', 'server-timing': 'intid;desc=38aef9113ae9644d, intid;desc=38aef9113ae9644d', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'CF-Cache-Status': 'DYNAMIC', 'Expect-CT': 'max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"', 'Server': 'cloudflare', 'CF-RAY': '6cc2faa9aed30b86-DFW'})
HTTP response body: {
"trace": "41cf06b8b7bef33c55735940744bf480",
"errors": [{
"code": "unknown_error",
"message": "An unknown error occurred (Illegal URI reference: Invalid input ' ', expected '/', 'EOI', '#', '?' or pchar (line 1, column 32): /v2/software_specifications/Not Found?space_id=8e990cec-0f27-42a7-9a78-c955cc0652c7\n ^)",
"more_info": "https://cloud.ibm.com/apidocs/machine-learning"
}],
"status_code": "500"
}

During handling of the above exception, another exception occurred:

WMLClientError Traceback (most recent call last)
/tmp/wsuser/ipykernel_172/4106139078.py in <module>
10 published_model = client.repository.store_model(
11 model=gbc,
---> 12 meta_props=metadata)

/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/repository.py in store_model(self, model, meta_props, training_data, training_target, pipeline, feature_names, label_column_names, subtrainingId)
408 """
409
--> 410 return self._client._models.store(model, meta_props=meta_props, training_data=training_data, training_target=training_target, pipeline=pipeline, feature_names=feature_names, label_column_names=label_column_names,subtrainingId=subtrainingId)
411
412 @docstring_parameter({'str_type': STR_TYPE_NAME})

/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/models.py in store(self, model, meta_props, training_data, training_target, pipeline, version, artifactid, feature_names, label_column_names, subtrainingId)
1539 training_target=training_target, pipeline=pipeline,
1540 feature_names=feature_names,
-> 1541 label_column_names=label_column_names)
1542 else:
1543 saved_model = self._publish_from_object(model=model, meta_props=meta_props, training_data=training_data, training_target=training_target, pipeline=pipeline, feature_names=feature_names, label_column_names=label_column_names)

/opt/conda/envs/Python-3.7-OpenCE/lib/python3.7/site-packages/ibm_watson_machine_learning/models.py in _publish_from_object_cloud(self, model, meta_props, training_data, training_target, pipeline, feature_names, label_column_names, project_id)
2489
2490 except Exception as e:
-> 2491 raise WMLClientError(u'Publishing model failed.', e)
2492
2493 def _upload_autoai_model_content(self, file, url, qparams):

WMLClientError: Publishing model failed.
Reason: (500)
Reason: Internal Server Error
HTTP response headers: HTTPHeaderDict({'Date': 'Wed, 12 Jan 2022 02:33:17 GMT', 'Content-Type': 'application/json', 'Content-Length': '455', 'Connection': 'keep-alive', 'server-timing': 'intid;desc=38aef9113ae9644d, intid;desc=38aef9113ae9644d', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'CF-Cache-Status': 'DYNAMIC', 'Expect-CT': 'max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"', 'Server': 'cloudflare', 'CF-RAY': '6cc2faa9aed30b86-DFW'})
HTTP response body: {
"trace": "41cf06b8b7bef33c55735940744bf480",
"errors": [{
"code": "unknown_error",
"message": "An unknown error occurred (Illegal URI reference: Invalid input ' ', expected '/', 'EOI', '#', '?' or pchar (line 1, column 32): /v2/software_specifications/Not Found?space_id=8e990cec-0f27-42a7-9a78-c955cc0652c7\n ^)",
"more_info": "https://cloud.ibm.com/apidocs/machine-learning"
}],
"status_code": "500"
}

------------------------------
David Castello
------------------------------
4. RE: Using Jupyter Notebooks on CP4D

Like
Joel Patterson
Posted 01/14/22 07:25 AM
Edited by System Admin 01/20/23 04:18 PM

Reply
Are you using CP4D on IBM Cloud or another install (i.e. the Federal CPD instance).   The two will differ when if comes to publishing to a deployment space.

OK, I see this is IBM Cloud.   Let me go through this.

Update:
(1) Fortunately the example shows location as us-south which is where most WML instances are created.   I can't find a way to get this value from the GUI console....
(2) Change the default_py3.7 value to default_py3.8 -- that got the line to work for me.

------------------------------
Joel Patterson
------------------------------

Original Message
5. RE: Using Jupyter Notebooks on CP4D

Like
Lutalo Henry
Posted 11/11/22 09:25 AM

Reply
hello david, did you figure out how to get past this?

------------------------------
Lutalo Henry
------------------------------

Original Message
6. RE: Using Jupyter Notebooks on CP4D

Like
Praveen Kumar K
Posted 11/08/22 09:31 AM

Reply
can anyone help me with this, i get will will deploying model in jupiter notebook

WMLClientError: Publishing model failed. Reason: (400) Reason: Bad Request HTTP response headers: HTTPHeaderDict({'Date': 'Tue, 08 Nov 2022 12:24:29 GMT', 'Content-Type': 'application/json', 'Content-Length': '292', 'Connection': 'keep-alive', 'server-timing': 'intid;desc=215bfcc61d99dbf0, intid;desc=215bfcc61d99dbf0', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'CF-Cache-Status': 'DYNAMIC', 'Server': 'cloudflare', 'CF-RAY': '766e492bff336780-DFW'}) HTTP response body: { "trace": "70117ce132f5a5c390d86a49a1a68730", "errors": [{ "code": "invalid_request_entity", "message": "Invalid request entity: Unsupported software specification 'default_py3.6'.", "more_info": "https://cloud.ibm.com/apidocs/machine-learning" }], "status_code": "400" }

------------------------------
Praveen Kumar K
------------------------------
7. RE: Using Jupyter Notebooks on CP4D

Like
Elif Sile
Posted 01/02/24 01:50 PM

Reply
I think problems like this may occur due to version incompatibility. Please check here to matching versions.
https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/pm_service_supported_frameworks.html?context=cpdaas&locale=en&audience=wdp

------------------------------
Elif Sile
------------------------------

Original Message

Cloud Pak for Data

Cloud Pak for Data

Using Jupyter Notebooks on CP4D

MARK JEYNES10/17/21 08:30 PM

David Castello01/13/22 10:46 AM

David Castello01/13/22 11:00 AM

Joel Patterson01/14/22 07:25 AM

Lutalo Henry11/11/22 09:25 AM

Praveen Kumar K11/08/22 09:31 AM

Elif Sile01/02/24 01:50 PM

1. Using Jupyter Notebooks on CP4D

2. RE: Using Jupyter Notebooks on CP4D

3. RE: Using Jupyter Notebooks on CP4D

4. RE: Using Jupyter Notebooks on CP4D

5. RE: Using Jupyter Notebooks on CP4D

6. RE: Using Jupyter Notebooks on CP4D

7. RE: Using Jupyter Notebooks on CP4D

Office

Community Links

IBM Links

Cloud Pak for Data

Cloud Pak for Data

Using Jupyter Notebooks on CP4D

MARK JEYNES10/17/21 08:30 PM

David Castello01/13/22 10:46 AM

David Castello01/13/22 11:00 AM

Joel Patterson01/14/22 07:25 AM

Lutalo Henry11/11/22 09:25 AM

Praveen Kumar K11/08/22 09:31 AM

Elif Sile01/02/24 01:50 PM

1. Using Jupyter Notebooks on CP4D

2. RE: Using Jupyter Notebooks on CP4D

3. RE: Using Jupyter Notebooks on CP4D

4. RE: Using Jupyter Notebooks on CP4D

5. RE: Using Jupyter Notebooks on CP4D

6. RE: Using Jupyter Notebooks on CP4D

7. RE: Using Jupyter Notebooks on CP4D

Related Content

Build a custom image for Watson Studio in Cloud Pak for Data 4

How to login to the ICPD 7-day trial env?

Improving Code Reusability, Part 2: Passing an Operator as a Parameter

Extending Streams Functionality with Native Functions

How to setup different environments in IBM Cloud pak like Dev, UAT and Production env?

Office

Community Links

IBM Links