Cloud Pak for Data

IBM Open Data for Industries APIs in IBM Cloud Pak for Data 3.5: Schema APIs

IBM Open Data for Industries in IBM Cloud Pak for Data 3.5 is an enterprise-grade open source-based platform that is backed by security and world-class service levels. IBM Open Data for Industries delivers the only end-to-end solution on the OSDU data foundation that supports infrastructure-agnostic, cloud-agnostic, and vendor-neutral technologies to reduce complexity in building an integrated energy data platform.

The following diagram shows the various components that form the Open Data for Industries solution:


Open Data for Industries provides the following APIs (components) to govern the Oil & Gas data resides on IBM Cloud Pak for Data, and users leverage these RESTful APIs to collaborate with this service.
Delivery API: Deliver data through a general-purpose mechanism by using Delivery API.
Entitlements API: Authenticate and authorize access to data using the Entitlements API.
File API: Fetch records or request file location data.
Indexer API: Index records for efficient search.
Legal API: Remain compliant in the different stages of the data lifecycle by using the Legal API.
Schema API: Manage schemas to define and enforce sets of attributes on functional data by using Schema API.
Search API: Query the data that is stored on storage layer using the Search API.
Storage API: Store data on storage layer by using the Storage API.
Workflow API: Define, manage and run process definitions by using the Workflow API.

This blog will introduce the Schema API with sample code in a python notebook to show the steps to access Open Data for Industries instance in IBM Cloud Pak for Data.

Open Data for Industries Schema API
The Schema API provides mechanisms to define and enforce structure to the data that is managed by Open Data for Industries. A schema is a structure, which is defined in JSON, which provides data type information for the data record fields. Users usually look into the data schemas to get the overview of the raw data.

The Schema API supports several subsurface data types. These data types are categorized in the following table:
Wellbore: A hole in the ground that extends from a point at the earth's surface to the maximum point of penetration.
Well: The origin of a set of wellbores. A well has one or more wellbores.
Seismic: Subsurface structures and energy waves.

User can call the Schema API to retrieve the schemas of Open Data for Industries in IBM Cloud Pak for Data with the following steps and code.
1. Create a new Python 3.7 notebook in Cloud Pak for Data with Open Data for Industries enabled.

2. Get the access bear token to the Open Data for Industries service instance.
In order to get the access token, user need provide keycloak URL of the Open Data for Industries service instance, client id and client secret, username and password.

# Get ODI access token
import requests, json

ODI_INSTANCE = <your ODI isntance fullname>
kc_url = "https://keycloak-osdu-keycloak." + ODI_INSTANCE + "/auth/realms/OSDU/protocol/openid-connect/token"

headers = {
    'content-type': "application/x-www-form-urlencoded"
}

body = {
    "grant_type": "password", 
    "scope": "openid profile", 
    "client_id": <your client id>, 
    "client_secret": <your client secret>, 
    "username": <your username>, 
    "password": <your password>
}

response = requests.post(kc_url, data=body, headers=headers, verify=True)
BEARER_TOKEN = response.json()["access_token"]
BEARER_TOKEN​

3. Call the Schema service endpoint to retreivew schemas of the Open Data for Industries data.

# Call Schema API to retreive schema data
ODI_DATA_PARTITION="opendes"
headers = {
        # Using default. Need to be updated based on real instance partition setup
        'data-partition-id': ODI_DATA_PARTITION, 
        'authorization': "Bearer " + BEARER_TOKEN,
        "Accept": "application/json",
        "Content-type": "application/json",
}
svc_url = "https://osdu-cpd-osdu." + ODI_INSTANCE + "/osdu-schema/api/schema/v1/schema"
response = requests.request("GET", svc_url, headers=headers)
print(json.dumps(response.json(), indent=2))​

The above steps and sample code show the basice procedure to retreive schema data with Open Data for Industries RESTful APIs in IBM Cloud Pak for Data. Users can get the detail API information from IBM Cloud Pak for Data 3.5 documentation https://www.ibm.com/docs/en/cloud-paks/cp-data/3.5.0?topic=resources-open-data-industries-apis
#CloudPakforDataGroup