If you haven’t seen the rest of the Game Changing Series Blog posts you can check them out here to get a high-level explanation of CDA and the other enhancements!

As promised, this plethora of enhancements did not leave our API users out!

CDA API Enhancements

We have made some improvements to the Cloud Data Access API calls in the OA65990 continuous delivery APAR and we are hoping these enhancements make your programming life easier.

Logging Output Override

It may not always be possible to have stderr or a SYSOUT DD for the CDA logging. It gets even messier if you are using the CDA Session support along with parallel tasks with all the output going into the same funnel.

We added a new optional parameter to all the CDA APIs called “logOutput” that expects a text string as its value. This text string indicates any valid thing that the fopen() function can open to write to. So, if you had an alternate output log file, you would pass in the absolute path to the file. (CDA will create it for you if it doesn’t exist.) If you have a data set that you want the output to go to, you should use the fopen() syntax for a data set name: //’<DATASET_NAME>’ . If the data set does not exist, it will be created with default attributes, which should be LRECL=1024, RECFM=VB. Additionally, if you have allocated a DD statement, you can pass that name as: DD:<mydd>, where <mydd> is the DD statement name.

As always, check out the DEBUG log to see what the optional parameters you passed were:

Skip Key File and Provider File Disk Read

One of the concerns I’ve heard was about CDA reading the key file and provider file from disk on every API call. This concern can be alleviated by using the CDA session support with GDKINIT/GDKTERM. In a CDA session, the key file and provider file is only read one time. However, for IMS and other applications that need the best performance, using a CDA session may not be feasible.

We added new optional parameters to the CDA APIs to allow passing in of the key file contents as well as the provider file contents. The application may read the data in a non-performance sensitive flow, and pass that data as needed to the CDA API.

·       providerFile – The value for this key consists of the provider file contents. CDA will bypass reading the provider file from the disk when this optional parameter is passed. CDA will make a copy of the passed data and use the JSON parser on that for the CDA operations.

·       keyFile – The value for this key consists of the gdkkeyf.json contents for the current user. CDA will bypass reading the user’s gdkkeyf.json file, instead making a copy of the value and using the JSON parser on that for the CDA operations. Note that CDA APIs such as GDKKEYAD, and GDKKEYDL, which modify the gdkkeyf.json file will not use any passed in keyfile. Additionally, it is important that the passed in key file contents are those for the current user, because CDA will use the ICSF keylabel associated with the current user to decrypt the information. If the contents are from a different user, decrypting those contents using the incorrect encryption key will result in garbage being used as the credentials.

GDK_SECONDARY CDA Symbol

When the OA64874 continuous delivery support for metadata on objects was done, one of the z/OS data set metadata fields that was not included was the secondary allocation amount for a data set. With this support, the &GDK_SECONDARY symbol is now recognized and resolved for z/OS data sets. This value may also be passed as part of the metadata string passed on the “metadata” optional parameter.

The resolved values will be in the format: <nnn>-<unit>, where nnn is the number or amount, and unit is the allocation unit. The allocation unit can be CYL for cylinders, TRK for tracks, BLK for blocks, REC for records, MB for megabytes, KB for kilobytes, and B for bytes.

Text Conversion Override

When CDA was young, doing conversion of text from EBCDIC (IBM-1047) to ASCII (ISO8859-1) was all it could do. Fortunately for us, people like CDA, they really do. This means that I have started being asked what the reason was that we didn’t support other Character Code Set IDs (code pages) when doing the text conversion. It was a good question, and one I didn’t really have a good answer for. So, I threw it to the top of the list and included it in this set of enhancements!

CDA has been modified to have the concept of a local code page and a remote code page. A local code page means that text data on the z/OS system is expected to be in that code page. The remote code page means that data in the remote cloud server is expected to be in that code page. So, when doing text conversion on a GDKWRITE call, the data will be converted from local to remote. When doing text conversion on a GDKGET call, the data will be converted from remote to local. This only applies to user data being converted as part of a GDKGET or GDKWRITE call. (The rest of CDA logging and messages is still done using IBM-1047 text, as well as responses from the cloud server being converted to IBM-1047 from ASCII.)

CDA will continue to have a local code page default of IBM-1047, and a remote code page default of ISO8859-1. That default can be overridden by key/value pairs in the provider file. The “localCodePage” key can be used to specify the default local code set for all GDKGET/GDKWRITE calls. The “remoteCodePage” key can be used to specify the remote code set for all GDKGET/GDKWRITE calls.

Additionally, new optional parameters, “localCodePage” and “remoteCodePage” can be passed on a GDKWRITE or GDKGET call with a value specifying a code set to be used if doing text conversion.

Examples of code set values accepted by the iconv() function can be found at:

Table 2: https://www.ibm.com/docs/en/zos/3.1.0?topic=cscu-code-set-converters-supplied

PKCS#11 HMAC sha256 calculation Enhancement

While in the midst of some heavy multi-threading using pthreads to perform multiple CDA API calls in parallel, we came across some issues in our usage of the PKCS#11 C APIs to do SHA-256 HMAC calculations when calculating the authentication signature for AWS4 or Azure communication. To address those multi-threading problems, we implemented direct usage of the ICSF APIs; CSFPTRC, CSFPHMG, and CSFPTRD to calculate the HMAC signatures.

If the callers of the CDA APIs want to use the new method to calculate the signatures, a new optional parameter was introduced called, “multi_thread”. When the value is “true”, then CDA will use the new method.

AMODE31 C DLL

We wanted to help our XLC callers with an easier, streamlined path, so we’ve introduced an AMODE31 DLL that XLC callers can call directly. This support delivers a new Dynamic Load Library (DLL) into SYS1.SIEALNKE, with an associated side-file or side-deck delivered into SYS1.SIEASID.

The benefits of this new DLL mainly apply to users of CDA Sessions. With GDKCSS, every CDA API call will cause a fetch and release of the GDKAPI program object. With the GDKDLL31, the gdkinit() call will fetch the GDKAPI program object, and gdkterm() will release it. All other API calls done through the DLL for that CDA session will use a saved pointer for the GDKAPI program object entry point.

If you have XLC code that you have been creating programs by binding with SYS1.CSSLIB(GDKCSS), you can simply change the IEWL Binder JCL to include the GDKDLL31 side-file from SYS1.SIEASID, to start using the direct XLC functionality.

Below you can find details on how the INCLUDE SYSLIB(GDKCSS) statement can be changed to INCLUDE SIEASID(GDKDLL31) to change your program to use the AMODE31 DLL.

OLD:

NEW:

AMODE31 JSON Credential Management C APIs

The GDKDLL31 Dynamic Load Library also includes some new APIs related to credential management. JSON is becoming a default simplified method for communicating between programs, especially when the implementation languages are different. The existing credential management APIs; gdkkeyad(), gdkkeydl(), gdkkeysr(), and gdkkeygr() can be used, but figuring out all the API parameters, and whether you are passing the address of a pointer field that points to a buffer to hold a secret key can be frustrating and difficult.

The new JSON APIs take a JSON string as input and return a JSON string as output. Each API may have different expectations as to the key/value pairs it is expecting, but in general, all require a provider name to be passed in. Often, an optional resource name may be passed in. (See the GDKUTIL credentials ADD section, 1.3.1, here for a discussion about what a ‘resource’ is.)

These APIs will take the input JSON string and break it into the equivalent parameters to pass to the original equivalent CDA credential management API.

If you are a user of the Google Cloud Platform (GCP), you are probably familiar with generating Service Credentials to access the GCP object storage. Their preferred format is to give you the credentials information in a JSON file. If you wanted to use the gdkkeyadJ() API to tell CDA to store the credentials found in that file, you could pass the following JSON string on the gdkkeyadJ() call:

Upon return from the CDA API, it will build a return JSON containing the return code from processing, the provider name that the operation was performed against and the resource name. Additional key/value pairs may be returned as part of the API, such as gdkkeysr(). Additionally, any messages that were received or created as part of processing are also returned in the JSON.

The idea is that many new uses of C APIs on z/OS is coming from Java, Python, or Go, and those languages can handle JSON well. Additionally, with the usage of z/OSMF and Ansible, the return data is displayed or formatted in some way, and JSON return is well-suited to that.

AMODE64 JSON Credential Management C APIs

With the expanding usage of Python, Java, and others on z/OS, most run AMODE64 with POSIX(ON). The new GDKDLL64 Dynamic Load Library shipped into SYS1.SIEALNKE with side-file shipped into SYS1.SIEASID is intended to make usage of the CDA Credential Management APIs easier from these new high-level languages.

The 64-bit versions of the APIs simply have “64” in the name and do the work to invoke the Language Environment Compiler-Writer Interfaces (CWI) to switch to a 31-bit Language Environment and call the equivalent AMODE31 JSON API.

Across all these interfaces, it is expected that the caller is responsible for freeing the returned JSON buffer once they are done using it.

Check out the publications update document for all the details on how to call and use the new gdkkeyad64J(), gdkkeydl64J(), gdkkeysr64J(), and gdkkeygr64J() APIs. https://public.dhe.ibm.com/eserver/zseries/zos/DFSMS/CDA/OA65990/OA65990_Publication_Updates.pdf

All of these enhancements are being made available on z/OS v2r5 and above by the PTFs for OA65990. Publication updates (there’s a lot) can be found at: https://public.dhe.ibm.com/eserver/zseries/zos/DFSMS/CDA/OA65990/OA65990_Publication_Updates.pdf

Check out our Cloud Data Access content solution page to learn more and get started!

We would love to hear your thoughts on CDA and these new enhancements! Leave a comment on this blog post, start a discussion on the DFSMS Community Page, or join Mainframe Data Management LinkedIn group to discuss there!

Author:

Andrew Wilt

Editor:

Alexis Kapica (Ricci)