App Connect

 View Only

Using IBM App Connect to interact with Google Cloud BigQuery

By Sharvari Gokahle posted Mon September 28, 2020 10:04 AM


Google Cloud BigQuery is an analytical data warehousing solution that enables large enterprises to analyze data across clouds at high speeds. It is a data storage service that can replicate data, provide automatic backup / snapshots of your data and enable you to easily restore and compare data because it can store seven days of change history.

Using Google Cloud BigQuery with IBM® App Connect

You can use App Connect to perform actions on the following objects:

  • Datasets
  • Tables
  • Views
  • Jobs

To use App Connect to integrate Google Cloud BigQuery with your other applications, you need to connect App Connect to your Google Cloud BigQuery account by providing the following information:

  • Project ID: The ID of your Google Cloud BigQuery project.
  • Dataset ID: The ID of the dataset from which you want to load tables.
  • Client email: The email ID of the service account associated with the Google Cloud BigQuery project.
  • Private key: A key used to establish the identity of the service account. 
For more information about other supported connectors, see Connectors.

Scenario: Retrieve documents from IBM® Cloudant® and insert document data into a Google Cloud BigQuery table

As an example, consider this scenario where you use App Connect to create a flow for an API that, when called, retrieves documents from an IBM Cloudant database and inserts data from these documents into a Google Cloud BigQuery table.

In this scenario, the Request node retrieves documents from IBM Cloudant through the Retrieve documents action. You need to use a For each node from the Toolbox to process each document so that you can configure the flow to retrieve data from the documents.

Data from these documents are inserted into Google Cloud BigQuery through the Insert records action. Data is mapped to the appropriate fields in a Google Cloud BigQuery table.

The Response node contains an example of the output mapping of the documents.

Figure 1: Flow for an API to retrieve documents from IBM Cloudant and insert document data into Google Cloud BigQuery table.