Currently, an API gateway is required as a proxy to connect to the Iceberg REST Catalog in watsonx.data, as it is only accessible within the data plane. The Iceberg REST Catalog endpoint is exposed via a nodeport, running on port 32671 instead of the standard port 443. Since Snowflake restricts connections to port 443, the API gateway is required as a proxy to enable connectivity. While any API gateway can be used, this blog will focus on using AWS API Gateway as a proxy to connect IBM watsonx.data with Snowflake.

Configuring the AWS API Gateway

Complete the following steps to fetch the watsonx.data Metadata Service (MDS) REST endpoint exposed via nodeport and connect to It from AWS API Gateway instead of directly connecting from Snowflake:

1.        In watsonx.data admin console, go to Infrastructure manager and click on the Iceberg catalog for catalog details.

2.        Navigate to the Catalog details page and get the details of the MDS Rest endpoint to use it in AWS HTTP API gateway integration.

Image 1: Catalog details with MDS REST endpoint details

3.        In AWS API gateway, create a route with ANY method using {proxy+} in HTTP API gateway.

NOTE: You must use HTTP proxy integrations for HTTP APIs. Do not use AWS Lambda function integration.

Image 2: HTTP proxy configuration in AWS API gateway

4.        Create HTTP integration to add MDS endpoint. Append {proxy} at the end of wxd rest endpoint. For more information, see Create HTTP proxy integrations for HTTP APIs.

Image 3: HTTP integration endpoint in AWS API gateway

5.        Review the details and submit. The API is now ready to be used.

6.        Copy the invoke URL and use it to integrate watsonx.data Iceberg Rest Catalog with snowflake.

IMPORTANT: Ensure to append /mds/iceberg at the end of invoke url in Snowflake.