Introduction

As organizations continue to adopt cloud-native applications and object-based storage architectures, the demand for cost-effective long-term object archival continues to grow. IBM Storage Protect provides an Object Agent service that exposes an S3-compatible interface, enabling standard S3 clients such as AWS CLI, MinIO Client, and other S3-enabled applications to archive and retrieve data while leveraging IBM Storage Protect's enterprise data protection capabilities.

By utilizing the Glacier storage class, object data can be staged in a Cold Data Cache Pool and automatically migrated to tape storage, providing an economical archival solution without requiring application changes.

This article demonstrates the complete configuration of Object Client archival using IBM Storage Protect, including object ingestion and retrieval operations using both MinIO Client and AWS CLI. In addition, two practical case studies are presented to highlight important considerations when designing Object Client archival solutions.

Solution Architecture

The following workflow illustrates how Object Client data is processed within IBM Storage Protect:

S3 Client (AWS CLI / MinIO Client)
                |
                v
          Object Agent
                |
                v
      Cold Data Cache Pool
                |
                v
            Tape Pool

Workflow Overview

1.     The S3 client connects to the IBM Storage Protect Object Agent.

2.     Data is archived using the requested storage class.

3.     Glacier-class data is initially stored in the Cold Data Cache Pool.

4.     IBM Storage Protect automatically migrates the data to the configured tape pool for long-term retention.

5.     During retrieval operations, data is staged back into the Cold Data Cache Pool before being presented to the requesting S3 client.

Prerequisites

Before starting the configuration, ensure the following prerequisites are met:

·       IBM Storage Protect Server Version 8.2.x or later

·       Configured tape library and tape device class

·       Existing tape storage pool for long-term retention

·       Dedicated filesystem for Cold Data Cache Pool

·       Object Agent service enabled

·       AWS CLI or MinIO Client installed

·       Network connectivity between S3 clients and Object Agent endpoint

·       Administrative access to IBM Storage Protect server

IBM Storage Protect Configuration

Step 1 – Create a Tape Storage Pool

The first step is to define a sequential tape storage pool that will act as the final archival destination for Glacier-class object data.

Once data is migrated from the Cold Data Cache Pool, it will be retained within this tape pool for long-term storage.

Define stgpool <poolname> <devclass> maxscr=value

Step 2 – Create the Cold Data Cache Directory

Create a dedicated directory on the IBM Storage Protect server that will be used to stage incoming Glacier-class objects.

The Cold Data Cache Pool serves as temporary storage before objects are migrated to tape.

Step 3 – Define the Cold Data Cache Pool

Create a Cold Data Cache Pool and configure the tape pool as the NEXTSTGPOOL destination.

This configuration enables automatic migration of Glacier-class data to tape once the data has been successfully written.

Define stgpool <poolname> stgtype=colddatacache directory=<cold data cache directory> nextstgpool=tapepool

Step 4 – Define the Object Domain

Create an Object Domain and configure storage class mappings:

Storage Class	Destination Pool
STANDARD	Standard Pool
GLACIER	Cold Data Cache Pool

This mapping determines where data is stored based on the storage class requested by the S3 client.

Define objectdomain <domain name> coldpool=<cold data cache pool> standardpool=<Directory or cloud container pool>

Step 5 – Configure the Object Agent

Enable the Object Agent service.

In this example:

·       LLA Port: 9000

·       HLA: IP address or fqdn (In lab, we have used IBM SP local host ip address)

The Object Agent acts as an S3-compatible endpoint through which clients can perform archival and retrieval operations.

Define server <server name> hla=<IP address or fqdn> lla=9000 objectagent=yes

The default value for the object agent port is 9000.

Step 6 – Install the Object Agent Service

After enabling the Object Agent, IBM Storage Protect provides the service installation command.

Execute the generated command to create and register the Object Agent service on the operating system.

Step 7 – Register the Object Client Node

Register a node using:

TYPE=OBJECTCLIENT

After registration, note the generated:

·       Access Key

·       Secret Access Key

These credentials will be used by S3-compatible clients for authentication.

Register node <nodename> type=objectclient domain=<objectdomain name>

At this stage, the IBM Storage Protect server-side configuration is complete.

Archive Using MinIO Client

MinIO Client (mcli) provides a simple method to interact with S3-compatible storage.

Step 8 – Install MinIO Client

Download and install the MinIO Client package.

Official Site: - https://dl.min.io/client/mc/release/

verify the installation and confirm the version.

Step 9 – Configure the MinIO Alias

Create an alias using:

·       Endpoint URL

·       Access Key

·       Secret Access Key

This alias provides a convenient way to interact with the Object Agent endpoint.

mcli alias set <alias name> <endpoint url with port> access_key secret_access_key --insecure

If trust certificate has been added, then no need --insecure option

Step 10 – Verify Existing Buckets

Use IBM Storage Protect administrative commands to verify existing buckets associated with the Object Client node.

Within IBM Storage Protect, buckets are represented as filespaces.

Step 11 – Create Bucket1

Create a bucket named:

bucket1

using the MinIO Client.

mcli mb <alias name>/bucket1 --insecure

Step 12 – Verify Bucket Creation

Verify that the newly created bucket is visible from IBM Storage Protect.

Step 13 – Upload Data Using Glacier Storage Class

Upload objects into bucket1 using the GLACIER storage class.

This causes the data to be written to the Cold Data Cache Pool.

mcli cp <filename> <alias name>/bucket1 --storage-class GLACIER --insecure

Step 14 – Verify Uploaded Objects

Confirm that objects are successfully stored within bucket1.

mcli ls --recursive <alias name>/bucket1 --insecure

Step 15 – Verify Cold Data Cache Pool Usage

Verify that the uploaded objects are present within the Cold Data Cache Pool.

This confirms successful ingestion through the Object Agent.

Step 16 – Verify Occupancy

Use QUERY OCCUPANCY to validate storage consumption for the Object Client node.

Step 17 – Verify Automatic Migration

IBM Storage Protect automatically migrates Glacier-class data from the Cold Data Cache Pool to the configured tape pool.

Monitor migration activity to confirm successful archival.

Archive Using AWS CLI

AWS CLI provides another S3-compatible method for interacting with IBM Storage Protect Object Agent.

Step 18 – Install AWS CLI

Download and install AWS CLI.

Note: - We have validated the archive and retrieval process in our lab, and it is functioning correctly with AWS CLI version 2.0.30.

Example:

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64-2.0.30.zip" -o "awscliv2.zip"
unzip awscliv2.zip

cd aws

Install as per the screenshot below

Step 19 – Configure AWS CLI

Configure AWS CLI using the Access Key and Secret Access Key generated earlier.

Step 20 – Verify Existing Buckets

Confirm bucket visibility through query filespace command.

Step 21 – Create Bucket2

Create a bucket named: bucket2

Note:

For lab environments, certificate warnings may be ignored.

Production environments should use trusted certificates and the appropriate CA bundle configuration.

aws s3 mb s3://bucket2 --endpoint-url=<https://object agent hla:lla> --no-verify-ssl

Step 22 – Verify Bucket Creation

Verify that bucket2 appears within IBM Storage Protect.

Step 23 – Upload Data Using AWS CLI

Upload data into bucket2 using the Glacier storage class.

This completes archival validation using AWS CLI.

aws s3 cp <filename> s3://bucket2 --endpoint-url=<https://object agent hla:lla> --no-verify-ssl

Object Retrieval Procedure

Step 24 – Submit Retrieval Request

Initiate a retrieval request using AWS CLI or MINIO mcli.

IBM Storage Protect processes the request and begins staging data from tape.

aws s3api restore-object --bucket <bucket name> --key "filename" --restore-request '{"Days":7,"GlacierJobParameters":{"Tier":"Standard"}}' --endpoint-url=<https://object agent hla:lla> --no-verify-ssl

·        This means the retrieved object will be available in S3 for 7 days before it automatically expires.

·        After this period, the temporary copy is removed, but the original archived object remains in Glacier.

Please use an alternative MINIO mcli command to retrieve the object and display the progress simultaneously.

mcli ilm restore <alias name>/bucket1/<filename> --days 7 --insecure

Run below command to verify the ongoing retrieval activity has been completed or not.

True means still in progress and false means completed.

aws s3api head-object --bucket <bucket name> --key "filename" --endpoint-url <https://object agent hla:lla> --no-verify-ssl

Step 25 – Monitor Copy to Cold Data Cache

After the retrieval request is submitted, IBM Storage Protect automatically starts a:

COPY TO COLDDATACACHE

process.

This retrieves the requested object from tape and stages it within the Cold Data Cache Pool.

Step 26 – Verify Retrieved Objects

Confirm the ongoing retrieval request has been completed or not using below AWS command

True means still in progress and false means completed.

aws s3api head-object --bucket <bucket name> --key "filename" --endpoint-url <https://object agent hla:lla> --no-verify-ssl

Verify that the retrieved object is once again visible within bucket2.

aws s3 ls s3://bucket2 --endpoint-url=<https://object agent hla:lla> --no-verify-ssl

--summarize --human-readable –recursive

Step 27 – Download Retrieved Object

Retrieve the object from bucket2 to the local server.

This completes retrieval validation.

aws s3 cp s3://bucket2/filename <destination path> --endpoint-url=<https://object agent hla:lla> --no-verify-ssl

Case Study – Retrieving Data After MOVE OBJECT CLIENT node data from one tapepool to another tapepool

Scenario

Organizations frequently perform storage migrations due to hardware refreshes, media replacement, or storage optimization initiatives.

A common question is whether Object Client retrieval continues to function after node data has been moved between tape pools.

Test Procedure

1.     Move Object Client data from one tape pool to another using MOVE NODEDATA.

2.     Submit a retrieval request.

3.     Monitor retrieve activity.

Result

IBM Storage Protect successfully retrieves the object from the new tape pool and stages it back into the Cold Data Cache Pool.

The retrieval process remains fully functional after MOVE NODEDATA operations.

Initiate a retrieval request via aws cli

Verify the Copy data process to COLDDATACACHE pool from new tapepool

Restrictions: 

1.     If you do not specify the STANDARDPOOL parameter, the object domain cannot receive requests from the S3 Standard storage class.

2.     If you do not specify the COLDPOOL parameter, the object domain cannot receive requests from the Amazon S3 Glacier Flexible Retrieval (formerly Amazon S3 Glacier) storage class.

3.     Ensure that object client data is not tiered from a directory-container storage pool or a cloud-container storage pool to tape storage. S3 data in these types of pools cannot be tiered to tape.

4.     The object agent does not support presigned URL for object uploading.

5.     The following restrictions apply to the tape storage pool:

·       You cannot replicate object client data to or from the tape storage pool.

·       The tape storage pool cannot be deduplicated.

·       A next storage pool cannot be specified for the tape storage pool.

Best Practices

·       Protect Access Keys and Secret Keys.

·       Use trusted certificates in production environments.

·       Monitor migration processes regularly.

·       Size the Cold Data Cache Pool appropriately.

·       Test retrieval operations periodically.

·       Maintain sufficient tape capacity for long-term retention.

·       Monitor Object Agent service health.

·       Refer link https://www.ibm.com/docs/en/storage-protect/8.2.1?topic=protect-s3-api-operations for S3 API Operations

Conclusion

IBM Storage Protect Object Agent provides a robust S3-compatible interface that enables organizations to integrate modern object-based workloads with enterprise tape storage infrastructure.

By combining Object Client functionality, Cold Data Cache Pools, and tape-backed storage pools, organizations can achieve cost-effective long-term retention while maintaining compatibility with standard S3 APIs.

The examples in this article demonstrated archival and retrieval operations using both MinIO Client and AWS CLI, along with two practical case studies that highlight important operational considerations when designing Object Client environments.

Contributors: Praveen Ariga

Acknowledgment: Special thanks to Ramchandra Sonawane for reviewing this blog.