If you haven’t seen the rest of the Game Changing Series Blog posts you can check them out here to get a high-level explanation of CDA and the other enhancements!

We in z/OS DFSMS understand the benefits and importance of supporting Cloud computing and the necessity of being able to access your data in cloud environments. To learn some basics about how the z/OS base component Cloud Data Access (CDA) provides a simple way to store any z/OS data directly onto cloud object storage, check out Part 1 of this blog series here.

We are excited to share that CDA is continuing to get even better! We’ve been busy and made some awesome enhancements to the Cloud Data Access component of z/OS.

New CDA functionality

The GDKUTIL utility can now create and delete buckets from z/OS. The new OPERATION command can be used to create a bucket using the CREATEBUCKET operation. Check out the updated sample provider files to get more details about this operation; The IBMCOS.json provider file even has a new CREATEOBJLOCKBUCKET operation that can be used to create S3 buckets with object lock enabled.

Additionally, the GDKUTIL utility has been updated to support new PREFIX and DELIMITER keywords on the LIST command. The PREFIX keyword lets you narrow the list results to only those ones beginning with the specified prefix. (Originally, only the first 1000 object names would be returned. Now, all object names in the bucket will be returned in the list output.)

Even better, GDKUTIL can now process multiple objects in one DELETE command. If you are doing cleanup of objects in a bucket, where some objects are no longer needed, you can use the new BUCKET DD statement to identify the bucket where objects should be deleted from. The PREFIX keyword can be used to narrow the objects processed, and a new REGEX keyword allows specification of a regular expression to further tailor the objects to be deleted. We also added a TEST keyword that does all the processing except for the actual deletion so you can see what objects would be deleted!

We believe that these enhancements will simplify your management of the cloud objects from z/OS.

But wait, there’s more!

We wanted to make it easier for you to bring groups of objects down to z/OS. The DOWNLOAD command of GDKUTIL has been enhanced to recognize the new BUCKET DD statement, along with the LOCNAME DD containing a z/OS UNIX target directory. The combination of these two tells GDKUTIL that it should download multiple objects from the bucket into the z/OS UNIX directory. The PREFIX keyword is also recognized and allows the list of objects (pending processing) to be narrowed to only the ones that begin with that prefix; The new REGEX keyword allows that list to be further tailored using a regular expression to choose which objects to download. The TEST keyword can be used to see what objects would match and what the final z/OS UNIX path name would be. Lastly, if you use the forward slash character (/) in the object name as a pseudo-directory separator, GDKUTIL will recreate that directory structure under the LOCNAME directory. If you use a cloud application to generate data into multiple cloud objects, the new GDKUTIL multi-object download support can bring that group of objects down to z/OS in a single command.

Finally, if you have a z/OS application that creates groups of z/OS UNIX files, you can UPLOAD the entire directory tree to cloud object store to share with other z/OS systems and cloud applications. The UPLOAD command has been enhanced to recognize the BUCKET DD statement as the target of the multi-object upload, and the LOCNAME DD statement containing a z/OS UNIX directory name as the relative root of the files to upload. GDKUTIL will traverse the directory sub-tree and upload each regular file it finds. The PREFIX keyword can be used to specify a prefix that each object name should start with. The rest of the object name will be made up of the relative path and the file name using the forward slash character (/) as a pseudo-directory separator. The REGEX keyword can be used to specify a regular expression that is applied to each file name when choosing whether to upload it or not. The TEST keyword allows you to see what files would be chosen without being uploaded.

Summary of new or updated Operations, Keywords, and Statements:

•        BUCKET  DD

•        UPLOAD z/OS UNIX directory tree

•        PREFIX(<object_name_prefix>)

•        REGEX(<regular_expression>)

•        DOWNLOAD objects to z/OS UNIX directory

•        PREFIX(<object_name_prefix>)

•        REGEX(<regular_expression>)

•        DELETE multiple objects

•        PREFIX(<object_name_prefix>)

•        REGEX(<regular_expression>)

•        TEST keyword for multi operations

•        OPERATION(<name>)

•        OPERATION(CREATEBUCKET)

•        OPERATION(DELETEBUCKET)

•        OPERATION(CREATEOBJLOCKBUCKET)

•        LIST

•        PREFIX(<prefix>)

•        DELIMITER(<delimiter>)

New APIs

If you are an application programmer that uses the CDA APIs, this support also introduces five! new APIs:

·       GDKINIT – Start a CDA session. The HTTPS connection is kept open for the entire session. The provider file, config file, and key file are only read once.

·       GDKTERM – End a CDA session. Clean up and terminate the HTTPS connection

·       GDKGEN – Perform an operation named in the provider file. i.e. CREATEBUCKET

·       GDKVALD – Validate provider file operation and key existence, and optionally return values from the provider file. (If your application requires the CREATEBUCKET operation to exist, use GDLVALD to ensure it is in the provider file.)

·       GDKQUERY – Query the current CDA code for what APIs and optional parms it supports.

One last note: Your provider files need to be updated to support much of the new functions. Make sure you compare your existing provider file with the updated samples in /usr/lpp/dfsms/gdk/samples/providers/ and move over the added sections.

These enhancements are introduced with the PTFs for OA65224. The official publications are being updated, but until then, you can visit the following link to get all the juicy details:

https://public.dhe.ibm.com/eserver/zseries/zos/DFSMS/CDA/OA65224

Check out our Cloud Data Access content solution page to learn more and get started!

We would love to hear your thoughts on CDA and these new enhancements! Leave a comment on this blog post, start a discussion on the DFSMS Community Page, or join Mainframe Data Management LinkedIn group to discuss there!

Stay tuned for ‘A Game Changing Series Part 3’!

Author: Andrew Wilt - Cloud Data Access Product Owner

Editors: Alexis Ricci - z/OS DFSMS Product Manager, Barbara McDonald - z/OS DFSMS Product Manager