zPET - IBM Z and z/OS Platform Evaluation and Test - Group home

Using DSFS with Multiple Data Set Qualifiers for the HLQ Directory Level

  

Using DSFS with Multiple Data Set Qualifiers for the HLQ Directory Level

Data Set File System (DSFS) is a new type of physical file system that renders traditional z/OS data sets accessible by programs, shell scripts, and end users of z/OS UNIX System Services.

For more basic introduction about DSFS, please refer to blog DSFS introduction.

Currently, DSFS assumes that the first qualifier of a data set name is the High Level Qualifier (HLQ). For HLQs that have a very large number of data sets, this can lead to a lot of wasted CPU cycles during catalog processing when an HLQ is first populated, and whenever it is resynced due to the directory refresh timeout interval.

Now with PTF UJ92199 (which was made available on March 2, 2023) for APAR OA63218, users can specify multiple qualifiers to be represented as a single directory in DSFS which is intended to limit the amount of time spent processing the catalog.

For example, for data set name USER.A.B.C.D, currently it can only be accessed in a given path directory as user/a.b.c.d. With this new support, it can be accessed as user/a.b.c.d, user.a/b.c.d, user.a.b/c.d, or user.a.b.c/d. 

Here’s an example from our zPET environment, for data set name DEMI.PARMLIB.BACKUP.D230207

(1) It can be accessed as demi/parmlib.backup.d230207

135:/dsfs/txt $ cd demi

136:/dsfs/txt/demi $ ls

parmlib3.backup.d221021    spftemp1.list.        jobs.plex                           spftemp2.list

parmlib.backup.d230206     spf1.list               

parmlib.backup.d230207     spflog1.list             ispf.profile            

parmlib3.backup.d221010 

(2) It can also be accessed as demi.parmlib/backup.d230207

138:/dsfs/txt $ cd demi.parmlib

139:/dsfs/txt/demi.parmlib $ ls

backup.d230206  backup.d230207

(3) It can also be accessed as demi.parmlib.backup/d230207

141:/dsfs/txt $ cd demi.parmlib.backup

142:/dsfs/txt/demi.parmlib.backup $ ls

d230206  d230207

Each of these HLQ directories will be considered as different paths, which means an attempt to access the same file in an exclusive mode, such as edit, rename or remove, in both directories will be considered a conflict and cause a failure. This is the same as if the same data set was accessed in both the /dsfs/bin and /dsfs/txt directory today.

For example, in our zPET environment, let’s see what happens when we access the same file from different paths with different HLQs.

The same file

demi.parmlib.backup.d221216(adyset00)

Different paths

/dsfs/txt vs /dsfs/rec

Different HLQs

demi vs demi.parmlib

On session 1, we edit the file from path /dsfs/txt with HLQ demi

135:/u/demi $ cd /dsfs/txt/demi 

136:/dsfs/txt/demi $ cd parmlib.backup.d221216                

137:/dsfs/txt/demi/parmlib.backup.d221216 $ oedit adyset00 

On session 2, all attempts to access the same file encounters error EDB9623E no matter browse / view /edit the same file from path /dsfs/rec with HLQ demi.parmlib

84:/dsfs/rec/demi.parmlib/backup.d221216 $ obrowse adyset00
/dsfs/rec/demi.parmlib/backup.d221216/adyset00                                                                   
ISRU800 Unexpected error from call to BPX1OPN in ISRUNXFV; Error: EBUSY (X'0072') Reason: EDB9623E (X'EDB9623E')  

85:/dsfs/rec/demi.parmlib/backup.d221216 $ oview adyset00
/dsfs/rec/demi.parmlib/backup.d221216/adyset00                               
ISRU800 Unexpected error from call to BPX1OPN in ISRUNXFV; Error: EBUSY (X'0072') Reason: EDB9623E (X'EDB9623E')  

86:/dsfs/rec/demi.parmlib/backup.d221216 $ oedit adyset00
/dsfs/rec/demi.parmlib/backup.d221216/adyset00                               
ISRU800 Unexpected error from call to BPX1OPN in ISRUNXFV; Error: EBUSY (X'0072') Reason: EDB9623E (X'EDB9623E')  

87:/u/demi $ bpxmtext EDB9623E
DSFS Fri Dec 16 20:33:36 2022
Description: DSFS cannot enqueue a data set member because the member is
already enqueued in a conflicting mode in another path. 
Action: Wait for the member enqueue to be released. Retry the operation. If
the problem persists, contact the service representative. 

Data sets can be excluded from being accessed by DSFS by use of the HLQ_LIST configuration option. This currently applies only for single qualifier HLQs. With the addition of this support, HLQs with multiple qualifiers can also be specified in the HLQ_LIST.

For example, in our zPET environment, specifying SYS1.PARMLIB for HLQ_LIST means SYS1.PARMLIB along with SYS1.PARMLIB.A, SYS1.PARMLIB.A.B cannot be used as HLQ directories, but SYS1 can still be used as a HLQ directory. In other words, data sets starting with SYS1 are not allowed to access from /dsfs/txt/sys1.parmlib or /dsfs/txt/sys1.parmlib.a (take /dsfs/txt path as an example), but they are allowed to access from /dsfs/txt/sys1.

Here we specify SYS1.PARMLIB for our HLQ_LIST, we can see we encounter ‘Operation not permitted’ when trying to access data sets from /dsfs/txt/sys1.parmlib, and /dsfs/txt/sys1.parmlib.backup, but are able to access data sets from /dsfs/txt/sys1.

172:/dsfs/txt $ dsadm config -hlq_list_add sys1.parmlib
IDFS00326I Successfully added the specified high-level qualifier directory names to the HLQ list.
 
173:/dsfs/txt $ dsadm configquery -hlq_list
The value for -hlq_list is SYS1.PARMLIB.
 
174:/dsfs/txt $ cd sys1.parmlib
cd: sys1.parmlib: EDC5139I Operation not permitted.
 
175:/dsfs/txt $ cd sys1.parmlib.backup
cd: sys1.parmlib.backup: EDC5139I Operation not permitted.
 
176:/dsfs/txt $ cd sys1
...
drwxrwxrwx    2 bpxroot  sys1           0 Sep  6 01:00 msgenu
drwxrwxrwx    2 bpxroot  sys1           0 Jun  8 01:00 msgjpn
drwxrwxrwx    2 bpxroot  sys1           0 Dec  1 00:00 parmlib2
drwxrwxrwx    2 bpxroot  sys1           0 Dec  1 00:00 pet.clist
drwxrwxrwx    2 bpxroot  sys1           0 Dec  1 00:00 pet.iefjobs
drwxrwxrwx    2 bpxroot  sys1           0 Dec 13  2021 pet.jcl
...

Furthermore, if SYS1 is specified for HLQ_LIST, then data sets starting with SYS1 are not allowed to access from HLQ directories that have SYS1 or SYS1.A or SYS1.A.B as the first qualifiers.

Here we specify SYS1 for HLQ_LIST, we can see we encounter ‘Operation not permitted’ when trying to access data sets from /dsfs/rec/sys1 and /dsfs/rec/sys1.parmlib

161:/dsfs/rec/sys1 $ dsadm config -hlq_list_add sys1
IDFS00326I Successfully added the specified high-level qualifier directory names to the HLQ list.
 
162:/dsfs/rec/sys1 $ dsadm configquery -hlq_list
The value for -hlq_list is SYS1.
 
164:/dsfs/rec $ cd sys1
cd: sys1: EDC5139I Operation not permitted.
 
165:/dsfs/rec $ cd sys1.parmlib
cd: sys1.parmlib: EDC5139I Operation not permitted. 

Single HLQ directories can have their own creation parameters specified. With this new support, each multiple qualifier HLQ directory can have its own creation parameters. If none exist for an HLQ directory that a user is trying to create a data set in, there will be no search for other HLQs that might apply to the same data set. It will simply fail as it does today. In other words, multiple qualifier directories have no association with other multiple HLQ directories. When a data set is created under an HLQ, it will be added to the other existing HLQ directories.

For example, in our zPET environment, there’s creation parameters for HLQ demi, so you’re allowed to create files or directories under demi.

202:/u/demi $ dsadm fileinfo -path /dsfs/txt/demi
   path: /dsfs/txt/demi
   fid                    11,1          anode                  160527,3036     
   length                 8192          format                 BLOCKED         
   1K blocks              8             dir tree status        VALID           
   PDS model anode        31,73         PS model anode         32,56           
   object type            DIR           object linkcount       4               
   object genvalue        0x00000000    dir version            1               
   dir name count         16            data set name type     HLQ DIR         
   recfm                  na            lrecl                  na              
   data mode              TEXT          data set status        RETRIEVED       
   data set name          DEMI                                     
   ENQ held               NO                                     
   direct blocks          0x0005F68A    0xFFFFFFFF    0xFFFFFFFF    0xFFFFFFFF 
                          0xFFFFFFFF    0xFFFFFFFF    0xFFFFFFFF    0xFFFFFFFF 
   indirect blocks        0xFFFFFFFF    0xFFFFFFFF    0xFFFFFFFF   
   mtime       Nov  8 03:39:50 2022     atime        Dec  1 03:20:11 2022
   ctime       Nov  8 03:39:50 2022     create date  Jun 29 2022         
   not encrypted                        not compressed                   
   PDS model              cyl dsorg(po) lrecl(180) recfm(fb) space(5,1) blksize(0) dsntype(library)
   PS model               cyl dsorg(ps) lrecl(300) recfm(vb) space(5,1) blksize(0)
   vnode,vntok          0x00000050,,0x072003C0      0x02D30180,,0x00000000
   opens                oi=0          rd=0          wr=0
   file segments        na         file unscheduled        na          
   meta buffers         0          dirty meta buffers      1 
 
203:/u/demi $ cd /dsfs/txt/demi
204:/dsfs/txt/demi $ touch test1202.txt
205:/dsfs/txt/demi $ ls test*
test1202.txt

But no creation parameters for HLQ demi.jobs, so you’re not able to create files or directories under demi.jobs.

207:/dsfs/txt $ dsadm fileinfo /dsfs/txt/demi.jobs
   path: /dsfs/txt/demi.jobs
   fid                    108,9605      anode                  163074,3288     
   length                 8192          format                 BLOCKED         
   1K blocks              8             dir tree status        VALID           
   PDS model anode        0,0           PS model anode         0,0             
   object type            DIR           object linkcount       3               
   object genvalue        0x00000000    dir version            1               
   dir name count         3             data set name type     HLQ DIR         
   recfm                  na            lrecl                  na              
   data mode              TEXT          data set status        RETRIEVED       
   data set name          DEMI.JOBS                                
   ENQ held               NO                                     
   direct blocks          0x0005F68C    0xFFFFFFFF    0xFFFFFFFF    0xFFFFFFFF 
                          0xFFFFFFFF    0xFFFFFFFF    0xFFFFFFFF    0xFFFFFFFF 
   indirect blocks        0xFFFFFFFF    0xFFFFFFFF    0xFFFFFFFF   
   mtime       Dec  1 03:20:24 2022     atime        Dec  1 03:20:26 2022
   ctime       Dec  1 03:20:24 2022     create date  Dec  1 2022         
   not encrypted                        not compressed                   
   PDS model              na           
   PS model               na           
   vnode,vntok          0x00000050,,0x072011D0      0x015542A0,,0x00000000
   opens                oi=0          rd=0          wr=0
   file segments        na         file unscheduled        na          
   meta buffers         0          dirty meta buffers      1 
 
208:/dsfs/txt $ cd demi.jobs
209:/dsfs/txt/demi.jobs $ touch test1202
touch: file "test1202": EDC5121I Invalid argument. 

After specifying creation parameters for HLQ demi.jobs, creating files or directories are allowed.

210:/dsfs/txt/demi.jobs $ dsadm createparm -path /dsfs/txt/demi.jobs -pdsmodel "cyl dsorg(po) lrecl(180) recfm(fb) space(5,1) blksize(0) dsntype(library)"
 
211:/dsfs/txt/demi.jobs $ dsadm createparm -path /dsfs/txt/demi.jobs -psmodel "cyl dsorg(ps) lrecl(300) recfm(vb) space(5,1) blksize(0)"
 
213:/dsfs/txt/demi.jobs $ cd ..
214:/dsfs/txt $ dsadm fileinfo -path demi.jobs
   path: /dsfs/txt/demi.jobs
   fid                    108,9605      anode                  163074,3288     
   length                 8192          format                 BLOCKED         
   1K blocks              8             dir tree status        VALID           
   PDS model anode        35,73         PS model anode         36,56           
   object type            DIR           object linkcount       3               
   object genvalue        0x00000000    dir version            1               
   dir name count         3             data set name type     HLQ DIR         
   recfm                  na            lrecl                  na              
   data mode              TEXT          data set status        RETRIEVED       
   data set name          DEMI.JOBS                                
   ENQ held               NO                                     
   direct blocks          0x0005F68C    0xFFFFFFFF    0xFFFFFFFF    0xFFFFFFFF 
                          0xFFFFFFFF    0xFFFFFFFF    0xFFFFFFFF    0xFFFFFFFF 
   indirect blocks        0xFFFFFFFF    0xFFFFFFFF    0xFFFFFFFF   
   mtime       Dec  1 03:20:24 2022     atime        Dec  1 03:20:26 2022
   ctime       Dec  1 03:20:24 2022     create date  Dec  1 2022         
   not encrypted                        not compressed                   
   PDS model              cyl dsorg(po) lrecl(180) recfm(fb) space(5,1) blksize(0) dsntype(library)
   PS model               cyl dsorg(ps) lrecl(300) recfm(vb) space(5,1) blksize(0)
   vnode,vntok          0x00000050,,0x072011D0      0x015542A0,,0x00000000
   opens                oi=0          rd=0          wr=0
   file segments        na         file unscheduled        na          
   meta buffers         0          dirty meta buffers      1 
 
215:/dsfs/txt $ cd demi.jobs
216:/dsfs/txt/demi.jobs $ touch testjcl
217:/dsfs/txt/demi.jobs $ mkdir jcl
226:/dsfs/txt/demi.jobs $ ls -al
total 48
drwxrwxrwx   4 bpxroot  sys1        8192 Dec  2 02:48 .
drwxrwxrwx   7 bpxroot  sys1        8192 Dec  1 04:57 ..
drwxrwxrwx   2 bpxroot  sys1           0 Dec  2 02:48 jcl
drwxrwxrwx   2 bpxroot  sys1        8192 Dec  2 00:00 plex
-rwxrwxrwx   1 bpxroot  sys1           0 Dec  2 00:00 testjcl 

Once a HLQ directory has been created, it can only be deleted if the HLQ directory name is excluded by the HLQ list.

For example, in our zPET environment, try to delete the HLQ directory name sys1.parmlib.backup.d082517.

164:/dsfs/txt $ ls
demi                         demi.parmlib.backup          demi.parmlib2                dsfsauto                   sys1.parmlib.backup.d082517  demi.jobs                    demi.parmlib.backup.demi     demi.parmlib3              sys1                         sys1.parmlib2                demi.parmlib                 demi.parmlib1              demitest                     sys1.parmlib 
 
167:/dsfs/txt $ dsadm config -hlq_list_add sys1.parmlib.backup.d082517
IDFS00326I Successfully added the specified high-level qualifier directory names to the HLQ list.
 
168:/dsfs/txt $ dsadm configquery -hlq_list
The value for -hlq_list is SYS1.PARMLIB.BACKUP.D082517.
 
169:/dsfs/txt $ ls
demi                      demi.parmlib.backup       demi.parmlib2             dsfsauto                 sys1.parmlib2             demi.jobs                 demi.parmlib.backup.demi  demi.parmlib3            

sys1                      demi.parmlib              demi.parmlib1             demitest                 sys1.parmlib 

We can see the HLQ directory sys1.parmlib.backup.d082517 has been successfully deleted.

OK, that’s all our introduction for multiple HLQs support. We welcome any feedback, suggestion, or comments.

For more information about DSFS, please refer to the Administration guide

Author:

Yu Mei, Dai (dyubj@cn.ibm.com)

Alfred Lease (alease@us.ibm.com)

Kieron D Hinds (kdhinds@us.ibm.com)