Global Data Lifecycle - Integration and Governance

By Pat O’Sullivan and Lena Woolf

In this post, you will learn how to define custom asset types and catalog custom assets.
The assets governed as part of a Watson Knowledge catalog environment do not exist in an isolated bubble within the enterprise. These assets support and address specific issues or topics that are relevant to lines of businesses. A key success factor for many Data Governance activities is to determine how to represent business interest. This tight connection to the business can help the end users to know which assets are relevant to them and their colleagues. This business angle also is an important consideration to the Data Stewards and other administration staff.

Within Watson Knowledge Catalog (WKC) there is a wide array of options to show areas of business interest of artifacts or assets. For example, it is possible to use tags or secondary categories to state business interest on artifacts such as Terms , Policies and Rules. Or you can use Tags to state similar business interest for Data Assets.

Yet, in some cases an organization may wish to have a more explicit way of representing how these assets interact with the business. For example, organization may wish to connect different Business Functions with governed assets.

Custom assets related to terms and data assets.

The diagram above shows an example of how such Business Functions (Wealth Management , Lending) and Business Processes (Administer Portfolio, Administer Trade, etc) might interact with objects in Watson Knowledge Catalog such as Business Terms, Policies and Data Assets.

When determining to represent objects such as Business Processes, it is important to remember that Watson Knowledge Catalog is not a Business Process Modelling tool. So the objective is not to represent and maintain all possible aspects of Business Processes and associated Business Activities. Instead, the focus should be on what is the appropriate level of representation for such objects. So that the required connection between the Governance and Data Assets and associated Business objects can be maintained.

In this earlier posts you have learned how to define custom attributes for existing asset types. The process of defining new types is very similar.
You can define custom asset types via API to describe asset type with its custom properties. As a part of type definition, you can provide user friendly labels for type name and property names to display in Catalog Asset Details UI.
Lets define custom asset type Business Function asset type by giving it a technical name “acme_business_function”. You can use PUT /v2/asset_types/{type_name} API to create and update asset types. Please specify the “acme_business_function” asset type name as the URL path parameter so API call would look like this PUT https://api.dataplatform.cloud.ibm.com/v2/asset_types/acme_attributes?catalog_id=replace_with_catalog_id.

Here is the API request body for “acme_business_function” asset type.

{
   "description":"Business Function attributes",
   "attribute_only": false,
   "fields": [
     {
        "key":"criticality",
        "type":"string",
        "facet":false,
        "is_array":false,
        "is_searchable_across_types":false
     }
  ],
  "localized_metadata_attributes": {
   "name": {
     "default":"Business Function",
     "en": "Business Function"
   }
 },
   "relationships":[
   ],
  "properties": {
       "criticality": {
         "type": "string",
         "description": "A classification of how critical this function is",
         "placeholder": "Please include  criticality classification (e/g/ High, Medium)",
         "required": true,
         "readonly": false,
         "max_length": 100,
         "label": {
           "default": "Criticality",
           "en": "Criticality"
         }
       }
     }
}

Note the difference in request body comparing to earlier post. We are not including “decorates” JSON property in the body of the asset type request, because we are making new asset type, instead of adding attributes to existing type. We also including “attribute_only”: false JSON property to inform WKC that this asset type is meant for making assets only.
For more information on how to run this API, see https://medium.com/ibm-data-ai/tailor-your-data-catalog-with-custom-asset-attributes-a2b468535a48.

Once the “acme_business_function” asset type is defined, we can create new Wealth Management catalog assets for by calling POST /v2/assets API with the following request body:

{
  "metadata": {
     "name": "Wealth Management",
     "description": "Wealth management is an investment advisory service that combines other financial services to address the needs of affluent clients",
     "tags": [
       "services"
     ],
     "asset_type": "acme_business_function",
     "origin_country": "us",
     "rov": {
       "mode": 0
     },
     "asset_category": "USER"
  },
  "entity": {
   "acme_business_function": {
        "criticality": "High"
    }
  }
}

I used this simple shell script to issue a cURL request, providing appropriate catalog id and bearer token.

token=replace_with_your-user_bearer_token_here
addr=api.dataplatform.cloud.ibm.com
catalog_id=32ddaccd-da8d-4e48-a2e3-6e1817f46b5acurl -v -k POST \
--header "Content-Type: application/json" \
--header "Accept: application/json" \
--header "Authorization: Bearer ${token}" \
-d @acme_business_function_asset.json\
  "https://${addr}/v2/assets?catalog_id=${catalog_id}"

Upon successful execution, you should see new asset appearing in the Catalog UI “Recently Added” section.

All custom attributes of newly created will be visible on the Asset Overview screen UI under “Details” section where they can be populated or updated.

You can associate your custom Wealth Management asset with Governance business terms.

You can relate custom assets with other assets in Catalog using APIs or “Related Assets” section in Asset Overview screen.

Custom Assets can appear in Global Search. Search an an excellent starting point for business users, where they can search for business process by name and navigate to other assets.

In this post, you have learned how to extend Watson Knowledge Catalog to incorporate business-specific assets. This blog has also provided the steps to integrate these with the broader set of catalog assets.