A Game Changing Series Part 10 – Cloud Data Access (CDA)

Immutable Objects

If you haven’t seen the rest of the Game Changing Series Blog posts you can check them out here to get a high-level explanation of CDA and the other enhancements!

The DFSMSdfp CDA team has been working to round out our support for object stores. The most recent enhancement, delivered by OA67781, revolves around utilizing the Object Versioning and Object Locking functionality that exists in object stores. This gives you the ability to create immutable objects to the limit of what cloud object stores provide.

“What do you mean by ‘immutable objects’, Andrew?” Well, I’m glad you asked, Reader. When I hear the phrase, “Object Immutability”, my first thought is that an object you put in the object store cannot be 1) modified, 2) replaced, or 3) deleted, and when I get that object, I will receive the exact one I expected. However, the reality from object storage is slightly different:

It is true that everyday objects cannot be modified. There is no way to truncate or modify the data within the object; So, that’s one done. P

What about keeping objects from being replaced? The solution for that is to have buckets with object versioning enabled. Each time you put an object into that bucket, it will get a unique version ID; If you put another object with the same name, the object store will give it a different version ID. P

This means you will need to remember the version ID to be sure you retrieve the expected version.

Below is a screenshot of object versions in action. The screenshot shows the web interface to the IBM Cloud for a bucket that has object lock (and object versioning) enabled.

What about keeping objects from being deleted? With just object versioning, the current version or a particular version (if you specify the version ID) can still be deleted; Thus, enter Object Lock. A bucket with object lock enabled gets object versioning and keeps objects from being deleted. This is great, but it means that someone can still upload an object to the bucket, making a new ‘current version’. So, you still need to keep track of the version ID of the one you uploaded and want to keep as your immutable copy. 

The screenshot below shows the error I received when trying to delete one of the versions in the object lock bucket

Since there has been so much interest in immutable objects, this new enhancement seeks to give GDKUTIL users (and callers of the CDA APIs) the ability to work with objects in buckets with object lock and/or versioning enabled. 

• We thought you would want to know the version ID of an object after an upload. 
• We enhanced LIST to be able to report more information about an object, like its version ID. 
• We added the ability to calculate the MD5 sum on data being sent so you can upload to an object lock bucket that requires the Content-MD5 HTTP header on PUTs. 
• We added a bunch of new operations to the sample provider files so that you can do things like:

o   Enabling or suspending object versioning in a bucket.

o   Creating a bucket object lock enabled (includes object versioning).

o   Enabling object lock for a bucket with versioning enabled.

o   Setting the retention policy for objects that are locked.

o   Turn on a legal hold for an object.

• We also enhanced the multi-object delete functionality to allow deletion of all versions for object names. (It really helps clean up a bucket after doing a bunch of testing.) Note it may be a two-step process.

1 Why Object Versioning?

The ability to have versions of the object with the same name came out of a requirement for supporting object lock for objects. But by itself, I can see some interesting use-cases for it. Those familiar with the z/OS concept of Generation Data Groups (GDG) have some insight here. GDGs allow you to create multiple data sets with the same name, where each time you create another data set with the name, you get a new version. This allows for some repeatable scripts (JCL) to upload to the same name each time, thus building a history of the data over time. 

For example, if you put your data into a GDS (Generation Data Set), your batch processing could upload the current (0) version of the data set to the same object name in the object store in a bucket that has object versioning enabled, thus mirroring the GDG structure. 

1.1 What's My Version ID?

If you upload an object to a bucket that has object lock or object versioning enabled, we expected that you would want to know what version ID was assigned to the object once the request was successful. So, we enhanced GDKUTIL to write out a message containing the object version ID. There is a new GDKU0131I message that specifies the object name and the VERSIONID. Also, as a treat, we figured you might want to know the ETag that the object store assigned to the object, so we added GDKU0130I to share that Entity Tag (ETag).

Note that object versioning by itself does not protect from deletion of objects. You can still delete the current version with a simple DeleteObject request or delete previous versions by specifying the version ID with the DeleteObject request.

1.2 Target a Specific Object Version

Now that you know the version ID for a particular object, how do you use it? This support adds a new VERSIONID() keyword. It is recognized on the DOWNLOAD, DELETE, LIST and OPERATION commands so that you can perform that command on the specific version of the object. UPLOAD doesn’t recognize it because the version ID is generated by the object store and doesn’t let us tell them what the version ID should be.

I recommend using the OBJNAMEX DD to specify your full object name. The VERSIONID keyword also allows the version ID to be spread across multiple lines of SYSIN.

1.3 Using LISTOBJECTVERSIONS

When you do a normal LIST command via GDKUTIL to list the objects in a bucket, the returned output will only display the object names without showing any version information. If you want to see what version IDs there are for the objects in a bucket, you should use the new LISTOPERATION() keyword. The new keyword allows you to specify the name of an operation that understands how to parse the returned data so it can be displayed.

The LISTOBJECTVERSIONS operation understands how to recognize the version ID for a particular object name, as well as the storage class and ETag for the object. We also introduced the ability to request specific columns in the LIST output with the LISTCOLS() keyword. The LISTCOLS keyword expects a DD name for data that is a comma separated list of column names from:

·       NAME – Object name

·       MTIME – Modification/Creation time

·       CTIME – Creation time if applicable

·       SIZE – Object size

·       VERSIONID – Version ID of the object

·       ETAG – Entity Tag value for the object

·       STORCLASS – Storage class assigned to the object.

What’s more, the order of names specified will be used as the order of the columns found in the output. Note that the LISTDATEFMT() keyword is used to change the default format of the displayed date of each object.

1.3.1 Bonus: Delete All Versions

Now, say you want to get rid of object versions. You can use the GDKUTIL multi-object operation for DELETE to delete all versions for objects that match the criteria. The example below uses the PREFIX keyword to only find objects that start with “Sales-“, as well as the REGEX keyword to specify a regular expression that means ‘ends with .csv’. At this time, we don’t have a way to specify other selection criteria, such as ‘more than 30 days old’. Maybe someone will open a z/OS Idea requirement?? (Hint, hint)

1.4 Using LISTDELETEOBJECTVERSION

Note that the list of objects returned does not display any objects that have been ‘soft deleted’. Objects are ‘soft deleted’ when a DELETE request is made to just the object name. (No version ID is specified.) It seems like the object store just creates a new ‘current version’ of the object with size zero. This is important to know because the bucket is not counted as empty until all the versions, including the 'soft deleted’ versions are gone.

1.4.1 Bonus: Get rid of the 'soft delete' versions

Maybe you want to remove all of the size zero versions that are soft deletes. The multi-object support with the LISTOPERATION() keyword also allows you to find objects to delete using the LISTDELETEOBJECTVERSIONS operation.

2 Object Lock

When you have an object (version) that has Object Lock enabled for it, that means that only special users can delete specific object versions. I went into it thinking that it was a way to prevent users from deleting objects, which is true, but it doesn’t prevent new versions of objects with the same name being created, making the current version something different from what you’d expect. The AWS documentation has some good information on the subject here.

You can enable a default object lock configuration for a bucket, and you can set a configuration for an individual object (version).

When you enable an object lock configuration on a bucket by the PutObjectLockConfiguration operation, you can set up the default Retention period, as well as the object lock mode. At the end of the retention period, the object (version) becomes unlocked and can be deleted. The mode can determine whether a root user for the account can delete the object anyway. (We found that the IBM Cloud only allowed the COMPLIANCE mode, which means nobody can delete it until the retention period is up.) The new operation in the sample provider files that equates to this is PUTBUCKETOBJLOCK.

You can change the Retention settings for a locked object via the PutObjectRetention operation. This allows you to change the retention period and maybe the mode. The new operation in the sample provider files that equates to this is PUTOBJECTRETENTION.

2.1 Retention of Objects / Lock Until

We wanted to give the z/OS administrators the tools to manage buckets as well as the objects within them. So, the sample provider files have a CREATEOBJLOCKBUCKET operation. This is easily done with the OPERATION() command.

Once a bucket is created with object-lock (and versioning) enabled, you may want to change the default retention settings. You can use the PUTBUCKETOBJLOCK operation to accomplish this. If you examine the PUTBUCKETOBJLOCK operation, you can see a newly added functionality to set default values to CDA variables that are later used in the request. GDK_MODE is set to COMPLIANCE, and GDK_DAYS is set to 1. The “default” definition is used by CDA to have default values for things so that the end user doesn’t have to set them when using a program like GDKUTIL.

Looking in the MESSAGE_BODY JSON object in the provider file, you can see that it uses a schema to build an XML document that is sent to the object store as the body of the request. Within the schema definition, you can see that it will build a ‘Mode’ object with a value of whatever is in GDK_MODE, and additionally a ‘Days’ object with a value of whatever is in GDK_DAYS.

So, now you may be asking yourself, “How do I override the default value for mode and days?” The answer to that is that we’ve added the ability to specify CDA variables and values when the OPERATION command is specified.

GDKUTIL will now look for a DD statement named, OPERVARS. The data should contain a comma separated list of key:value pairs, where the key is a CDA variable name, and the value is whatever string you want to be used when that CDA variable is used.

Later, after you have uploaded an object, the boss says you need to keep that object locked for an even longer time. In this event, knowing its version ID, you can give that object a new retain until date setting. You would do this via the PUTOBJECTRETENTION operation. Note the specification of the VERSIONID() keyword. It allows you to specify a particular VERSIONID that you want.

Of course, if we have operations that let you set the bucket default, or the object lock settings, you expect to be able to find out what the settings are currently. So, we have a GETOBJLOCKCONFIG operation that lets you see what the defaults are for a bucket.

And we have a GETOBJECTRETENTION operation that returns the settings for a particular object.

2.2 Uploading to a bucket with Object Lock - Content MD5

Additionally, we found that we needed to add the ability to calculate the MD5 sum of the data sent so that it could be placed on the Content-MD5 header and the request accepted for objects placed in a bucket with object lock enabled.

Being cognizant that not everyone would want to spend the additional CPU it takes to calculate the MD5 sum for everything sent to the object store, we needed a way to make it optional in the provider file. The idea is that you could turn it on for all uploads using that provider file. To that end, we added a new key to the provider file, “contentMD5”. If the value is “true”, then CDA will calculate the MD5 for each HTTP request body sent by that operation and will store the value in the GDK_BODY_MD5 variable so it could be used in a Content-MD5 header definition. We also needed a way to let the provider file build headers optionally. So, there is a new OPTHEADER type of object in the requestParameters. If the header can be built successfully (i.e. no errors resolving a CDA variable), then it will be used. Otherwise, processing continues.

So, if you are planning to upload data to buckets with object lock enabled, and your object store requires the Content-MD5 header when sending data to it, change the contentMD5 key’s value to “true”. As always, the WEBTOOLKITLOGU keyword lets you see all the logging from the z/OS Client for Web Enablement Toolkit as it talks to the object store.

Check out our Cloud Data Access content solution page to learn more and get started!

We would love to hear your thoughts on CDA and these new enhancements! Leave a comment on this blog post, start a discussion on the DFSMS Community Page, or join Mainframe Data Management LinkedIn group to discuss there!

Authors:

Andrew Wilt

Burramsetty Sushma Sri

Editor:

Alexis Kapica