Cloud Pak for Data

IBM Open Data for Industries APIs in IBM Cloud Pak for Data 3.5: Search APIs

IBM Open Data for Industries in IBM Cloud Pak for Data 3.5 is an enterprise-grade open source-based platform that is backed by security and world-class service levels. IBM Open Data for Industries delivers the only end-to-end solution on the OSDU data foundation that supports infrastructure-agnostic, cloud-agnostic, and vendor-neutral technologies to reduce complexity in building an integrated energy data platform.

Users can learn the architecture and the various components of Open Data for Industries in IBM Cloud Pak for Data 3.5 in this blog: https://community.ibm.com/community/user/cloudpakfordata/blogs/harris-yang1/2021/04/29/odi-apis-cpd35-schema-apis

Open Data for Industries provides the following APIs (components) to govern the Oil & Gas data resides on IBM Cloud Pak for Data, and users leverage these RESTful APIs to collaborate with this service.
Delivery API: Deliver data through a general-purpose mechanism by using Delivery API.
Entitlements API: Authenticate and authorize access to data using the Entitlements API.
File API: Fetch records or request file location data.
Indexer API: Index records for efficient search.
Legal API: Remain compliant in the different stages of the data lifecycle by using the Legal API.
Schema API: Manage schemas to define and enforce sets of attributes on functional data by using Schema API.
Search API: Query the data that is stored on storage layer using the Search API.
Storage API: Store data on storage layer by using the Storage API.
Workflow API: Define, manage and run process definitions by using the Workflow API.

This blog will introduce the Search API with sample code in a python notebook to show the steps to access Open Data for Industries instance in IBM Cloud Pak for Data.

Open Data for Industries Search API
The Search API provides a mechanism for searching indexed documents that contain structured data. You can search, index, organize, and present search results. Documents and indexes are saved in a separate persistent store that is optimized for search operations. The Search API can index any number of documents.

The following example shows the steps to search well data and wellbore data with the search APIs in a Python notebook in IBM Cloud Pak for Data. A well is the origin of a set of wellbores and a well has one or more wellbores. A wellbore is a hole in the ground that extends from a point at the earth's surface to the maximum point of penetration.

1. Create a new Python 3.7 notebook in Cloud Pak for Data with Open Data for Industries enabled.

2. Get the access bear token to the Open Data for Industries service instance.
Users can refer the same step in the previous blog on Schema APIs in https://community.ibm.com/community/user/cloudpakfordata/blogs/harris-yang1/2021/04/29/odi-apis-cpd35-schema-apis

3. Search well data from the Open Data for Industries service instance.

# Search well data
well_schema = "opendes:osdu:well-master:0.2.1"

headers = {
        # Using default. Need to be updated based on real instance partition setup
        'data-partition-id': ODI_DATA_PARTITION, 
        'authorization': "Bearer " + BEARER_TOKEN,
        "Accept": "application/json",
        "Content-type": "application/json",
}

search_data = {
    "kind": well_schema,
    "query": "id:*",
    "returnedFields": [
        "data.Data.IndividualTypeProperties.FacilityName", 
        "data.ResourceID", 
        "data.GeoLocation"
    ]
}

svc_url = "https://osdu-cpd-osdu." + ODI_INSTANCE + "/osdu-search/api/search/v2/query"
response = requests.request("POST", svc_url, json=search_data, headers=headers)
print(json.dumps(response.json(), indent=2))​

4. Search wellbore data from the Open Data for Industries service instance.

# Search wellbore data
well_id = "srn:master-data/Well:2785:"

wellbore_schema = "opendes:osdu:wellbore-master:0.2.1"

search_data = {
    "kind": wellbore_schema,
    "query": "data.Data.IndividualTypeProperties.WellID: \""+ well_id +"\"", 
    "returnedFields": [
        "data.Data.IndividualTypeProperties.FacilityName", 
        "data.ResourceID"
    ]
}

svc_url = "https://osdu-cpd-osdu." + ODI_INSTANCE + "/osdu-search/api/search/v2/query"
response = requests.request("POST", svc_url, json=search_data, headers=headers)
print(json.dumps(response.json(), indent=2))​

5. Search welllog data from the Open Data for Industries service instance.

# Search welllog data
wellbore_id = "srn:master-data/Wellbore:2785:"

welllog_schema = "opendes:osdu:welllog-wpc:0.2.1"

search_data = {
    "kind": welllog_schema,
    "query1": "data.Data.IndividualTypeProperties.WellboreID: \""+ wellbore_id +"\"", 
    "returnedFields": [
        "data.ResourceID",
        "data.Data.IndividualTypeProperties.WellboreID",
        "data.Data.GroupTypeProperties.Files", 
        "data.Curves.Mnemonic",
    ]
}

svc_url = "https://osdu-cpd-osdu." + ODI_INSTANCE + "/osdu-search/api/search/v2/query"
response = requests.request("POST", svc_url, json=search_data, headers=headers)
print(json.dumps(response.json(), indent=2))​

The above steps and sample code show the basice procedure to search data with Open Data for Industries RESTful APIs in IBM Cloud Pak for Data. Users can get the detail API information from IBM Cloud Pak for Data 3.5 documentation https://www.ibm.com/docs/en/cloud-paks/cp-data/3.5.0?topic=resources-open-data-industries-apis
#CloudPakforDataGroup