App Connect

 View Only

Using IBM® App Connect to interact with Google Cloud Storage

By Shahmini Arumugam posted Thu June 03, 2021 06:28 AM

  

Google Cloud Storage is an object storage service for storing and accessing live or archived data.
The objects are stored in containers that are called buckets. It is a flexible cloud storage product with several options to manage and store your data.

Using Google Cloud Storage with App Connect

You can use App Connect to perform actions on the following objects:

  • Bucket ACL
  • Bucket CORS configuration
  • Bucket lifecycle configuration
  • Bucket location
  • Bucket logging
  • Bucket tags
  • Bucket versioning
  • Bucket website
  • Buckets
  • Object ACL
  • Object versioning
  • Objects

Building a flow in App Connect with Google Cloud Storage

You can now use IBM App Connect to build flows that integrate with Google Cloud Storage and other applications. The Google Cloud Storage connector is displayed as “Google Cloud Storage” on the IBM App Connect User Interface (UI).

Provide the following information to connect App Connect to your Google Cloud Storage account.

Required credentials

 Description
Secret
The secret for your Google Cloud Storage user account. Ensure you set a default project for interoperable access for your user account. For more information, see https://cloud.google.com/storage/docs/authentication/hmackeys.
Access key
The access key for your Google Cloud Storage user account. Ensure you set a default project for interoperable access for your user account. For more information, see https://cloud.google.com/storage/docs/authentication/hmackeys.
Bucket name (optional)
The bucket name from your Google Cloud Storage account if you have access to only specific buckets in your Google Cloud Storage account.

Scenario: Data Ingestion to Google Cloud Storage from Dropbox

Consider this scenario where you use App Connect to create a scheduler-based flow that retrieves any type of file from the specified Dropbox path, downloads the content, and uploads the metadata and the file to a Google Cloud Storage bucket.

 For this scenario:

  • You run a scheduler-based flow that retrieves Dropbox folder items.

The scheduler runs the flow daily on an hourly basis.

  • The Dropbox retrieve operation fetches the folder items from the specified Dropbox path.
  • The flow includes an If and Else conditional statement node to process each folder item.
    1. For each folder item that is tagged as a file, Dropbox downloads the file content.
    2. The Google Cloud Storage upsert operation creates and uploads the metadata of the file to the specified Google Cloud Storage bucket.
    3. The next Google Cloud Storage upsert operation creates and uploads the file to the specified Google Cloud Storage bucket.
Based on the scheduler, the flow checks for any new or updated activities in the specified Dropbox path. For all new files, Google Cloud Storage creates and uploads the files’ metadata and the files to the specified buckets in Google Cloud Storage. For existing files, the files’ metadata and the files are updated with the latest changes and is uploaded to the specified Google Cloud Storage buckets.  

  • Else, a Slack message is created to notify you that the folder item is not tagged as a file.


You can opt to add a response payload from Dropbox to a dedicated Slack channel to list all the retrieved folder items.

 



IBM® App Connect supports a wide range of data formats, application infrastructures, and integration styles. For more information, see IBM® App Connect connectors.

0 comments
54 views

Permalink