Using DSFS with Multiple Data Set Qualifiers for the HLQ Directory Level
Data Set File System (DSFS) is a new type of physical file system that renders traditional z/OS data sets accessible by programs, shell scripts, and end users of z/OS UNIX System Services.
For more basic introduction about DSFS, please refer to blog DSFS introduction.
Currently, DSFS assumes that the first qualifier of a data set name is the High Level Qualifier (HLQ). For HLQs that have a very large number of data sets, this can lead to a lot of wasted CPU cycles during catalog processing when an HLQ is first populated, and whenever it is resynced due to the directory refresh timeout interval.
Now with PTF UJ92199 (which was made available on March 2, 2023) for APAR OA63218, users can specify multiple qualifiers to be represented as a single directory in DSFS which is intended to limit the amount of time spent processing the catalog.
For example, for data set name USER.A.B.C.D, currently it can only be accessed in a given path directory as user/a.b.c.d. With this new support, it can be accessed as user/a.b.c.d, user.a/b.c.d, user.a.b/c.d, or user.a.b.c/d.
Here’s an example from our zPET environment, for data set name DEMI.PARMLIB.BACKUP.D230207
(1) It can be accessed as demi/parmlib.backup.d230207
135:/dsfs/txt $ cd demi
136:/dsfs/txt/demi $ ls
parmlib3.backup.d221021 spftemp1.list. jobs.plex spftemp2.list
parmlib.backup.d230206 spf1.list
parmlib.backup.d230207 spflog1.list ispf.profile
parmlib3.backup.d221010
|
(2) It can also be accessed as demi.parmlib/backup.d230207
138:/dsfs/txt $ cd demi.parmlib
139:/dsfs/txt/demi.parmlib $ ls
backup.d230206 backup.d230207
|
(3) It can also be accessed as demi.parmlib.backup/d230207
141:/dsfs/txt $ cd demi.parmlib.backup
142:/dsfs/txt/demi.parmlib.backup $ ls
d230206 d230207
|
Each of these HLQ directories will be considered as different paths, which means an attempt to access the same file in an exclusive mode, such as edit, rename or remove, in both directories will be considered a conflict and cause a failure. This is the same as if the same data set was accessed in both the /dsfs/bin and /dsfs/txt directory today.
For example, in our zPET environment, let’s see what happens when we access the same file from different paths with different HLQs.
The same file
|
demi.parmlib.backup.d221216(adyset00)
|
Different paths
|
/dsfs/txt vs /dsfs/rec
|
Different HLQs
|
demi vs demi.parmlib
|
On session 1, we edit the file from path /dsfs/txt with HLQ demi
135:/u/demi $ cd /dsfs/txt/demi
136:/dsfs/txt/demi $ cd parmlib.backup.d221216
137:/dsfs/txt/demi/parmlib.backup.d221216 $ oedit adyset00
|
On session 2, all attempts to access the same file encounters error EDB9623E no matter browse / view /edit the same file from path /dsfs/rec with HLQ demi.parmlib
84:/dsfs/rec/demi.parmlib/backup.d221216 $ obrowse adyset00 /dsfs/rec/demi.parmlib/backup.d221216/adyset00 ISRU800 Unexpected error from call to BPX1OPN in ISRUNXFV; Error: EBUSY (X'0072') Reason: EDB9623E (X'EDB9623E')
85:/dsfs/rec/demi.parmlib/backup.d221216 $ oview adyset00 /dsfs/rec/demi.parmlib/backup.d221216/adyset00 ISRU800 Unexpected error from call to BPX1OPN in ISRUNXFV; Error: EBUSY (X'0072') Reason: EDB9623E (X'EDB9623E')
86:/dsfs/rec/demi.parmlib/backup.d221216 $ oedit adyset00 /dsfs/rec/demi.parmlib/backup.d221216/adyset00 ISRU800 Unexpected error from call to BPX1OPN in ISRUNXFV; Error: EBUSY (X'0072') Reason: EDB9623E (X'EDB9623E')
87:/u/demi $ bpxmtext EDB9623E DSFS Fri Dec 16 20:33:36 2022 Description: DSFS cannot enqueue a data set member because the member is already enqueued in a conflicting mode in another path. Action: Wait for the member enqueue to be released. Retry the operation. If the problem persists, contact the service representative.
|
Data sets can be excluded from being accessed by DSFS by use of the HLQ_LIST configuration option. This currently applies only for single qualifier HLQs. With the addition of this support, HLQs with multiple qualifiers can also be specified in the HLQ_LIST.
For example, in our zPET environment, specifying SYS1.PARMLIB for HLQ_LIST means SYS1.PARMLIB along with SYS1.PARMLIB.A, SYS1.PARMLIB.A.B cannot be used as HLQ directories, but SYS1 can still be used as a HLQ directory. In other words, data sets starting with SYS1 are not allowed to access from /dsfs/txt/sys1.parmlib or /dsfs/txt/sys1.parmlib.a (take /dsfs/txt path as an example), but they are allowed to access from /dsfs/txt/sys1.
Here we specify SYS1.PARMLIB for our HLQ_LIST, we can see we encounter ‘Operation not permitted’ when trying to access data sets from /dsfs/txt/sys1.parmlib, and /dsfs/txt/sys1.parmlib.backup, but are able to access data sets from /dsfs/txt/sys1.
172:/dsfs/txt $ dsadm config -hlq_list_add sys1.parmlib IDFS00326I Successfully added the specified high-level qualifier directory names to the HLQ list. 173:/dsfs/txt $ dsadm configquery -hlq_list The value for -hlq_list is SYS1.PARMLIB. 174:/dsfs/txt $ cd sys1.parmlib cd: sys1.parmlib: EDC5139I Operation not permitted. 175:/dsfs/txt $ cd sys1.parmlib.backup cd: sys1.parmlib.backup: EDC5139I Operation not permitted. 176:/dsfs/txt $ cd sys1 ... drwxrwxrwx 2 bpxroot sys1 0 Sep 6 01:00 msgenu drwxrwxrwx 2 bpxroot sys1 0 Jun 8 01:00 msgjpn drwxrwxrwx 2 bpxroot sys1 0 Dec 1 00:00 parmlib2 drwxrwxrwx 2 bpxroot sys1 0 Dec 1 00:00 pet.clist drwxrwxrwx 2 bpxroot sys1 0 Dec 1 00:00 pet.iefjobs drwxrwxrwx 2 bpxroot sys1 0 Dec 13 2021 pet.jcl ... |
Furthermore, if SYS1 is specified for HLQ_LIST, then data sets starting with SYS1 are not allowed to access from HLQ directories that have SYS1 or SYS1.A or SYS1.A.B as the first qualifiers.
Here we specify SYS1 for HLQ_LIST, we can see we encounter ‘Operation not permitted’ when trying to access data sets from /dsfs/rec/sys1 and /dsfs/rec/sys1.parmlib
161:/dsfs/rec/sys1 $ dsadm config -hlq_list_add sys1 IDFS00326I Successfully added the specified high-level qualifier directory names to the HLQ list. 162:/dsfs/rec/sys1 $ dsadm configquery -hlq_list The value for -hlq_list is SYS1. 164:/dsfs/rec $ cd sys1 cd: sys1: EDC5139I Operation not permitted. 165:/dsfs/rec $ cd sys1.parmlib cd: sys1.parmlib: EDC5139I Operation not permitted. |
Single HLQ directories can have their own creation parameters specified. With this new support, each multiple qualifier HLQ directory can have its own creation parameters. If none exist for an HLQ directory that a user is trying to create a data set in, there will be no search for other HLQs that might apply to the same data set. It will simply fail as it does today. In other words, multiple qualifier directories have no association with other multiple HLQ directories. When a data set is created under an HLQ, it will be added to the other existing HLQ directories.
For example, in our zPET environment, there’s creation parameters for HLQ demi, so you’re allowed to create files or directories under demi.
202:/u/demi $ dsadm fileinfo -path /dsfs/txt/demi path: /dsfs/txt/demi fid 11,1 anode 160527,3036 length 8192 format BLOCKED 1K blocks 8 dir tree status VALID PDS model anode 31,73 PS model anode 32,56 object type DIR object linkcount 4 object genvalue 0x00000000 dir version 1 dir name count 16 data set name type HLQ DIR recfm na lrecl na data mode TEXT data set status RETRIEVED data set name DEMI ENQ held NO direct blocks 0x0005F68A 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF indirect blocks 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF mtime Nov 8 03:39:50 2022 atime Dec 1 03:20:11 2022 ctime Nov 8 03:39:50 2022 create date Jun 29 2022 not encrypted not compressed PDS model cyl dsorg(po) lrecl(180) recfm(fb) space(5,1) blksize(0) dsntype(library) PS model cyl dsorg(ps) lrecl(300) recfm(vb) space(5,1) blksize(0) vnode,vntok 0x00000050,,0x072003C0 0x02D30180,,0x00000000 opens oi=0 rd=0 wr=0 file segments na file unscheduled na meta buffers 0 dirty meta buffers 1 203:/u/demi $ cd /dsfs/txt/demi 204:/dsfs/txt/demi $ touch test1202.txt 205:/dsfs/txt/demi $ ls test* test1202.txt |
But no creation parameters for HLQ demi.jobs, so you’re not able to create files or directories under demi.jobs.
207:/dsfs/txt $ dsadm fileinfo /dsfs/txt/demi.jobs path: /dsfs/txt/demi.jobs fid 108,9605 anode 163074,3288 length 8192 format BLOCKED 1K blocks 8 dir tree status VALID PDS model anode 0,0 PS model anode 0,0 object type DIR object linkcount 3 object genvalue 0x00000000 dir version 1 dir name count 3 data set name type HLQ DIR recfm na lrecl na data mode TEXT data set status RETRIEVED data set name DEMI.JOBS ENQ held NO direct blocks 0x0005F68C 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF indirect blocks 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF mtime Dec 1 03:20:24 2022 atime Dec 1 03:20:26 2022 ctime Dec 1 03:20:24 2022 create date Dec 1 2022 not encrypted not compressed PDS model na PS model na vnode,vntok 0x00000050,,0x072011D0 0x015542A0,,0x00000000 opens oi=0 rd=0 wr=0 file segments na file unscheduled na meta buffers 0 dirty meta buffers 1 208:/dsfs/txt $ cd demi.jobs 209:/dsfs/txt/demi.jobs $ touch test1202 touch: file "test1202": EDC5121I Invalid argument. |
After specifying creation parameters for HLQ demi.jobs, creating files or directories are allowed.
210:/dsfs/txt/demi.jobs $ dsadm createparm -path /dsfs/txt/demi.jobs -pdsmodel "cyl dsorg(po) lrecl(180) recfm(fb) space(5,1) blksize(0) dsntype(library)" 211:/dsfs/txt/demi.jobs $ dsadm createparm -path /dsfs/txt/demi.jobs -psmodel "cyl dsorg(ps) lrecl(300) recfm(vb) space(5,1) blksize(0)" 213:/dsfs/txt/demi.jobs $ cd .. 214:/dsfs/txt $ dsadm fileinfo -path demi.jobs path: /dsfs/txt/demi.jobs fid 108,9605 anode 163074,3288 length 8192 format BLOCKED 1K blocks 8 dir tree status VALID PDS model anode 35,73 PS model anode 36,56 object type DIR object linkcount 3 object genvalue 0x00000000 dir version 1 dir name count 3 data set name type HLQ DIR recfm na lrecl na data mode TEXT data set status RETRIEVED data set name DEMI.JOBS ENQ held NO direct blocks 0x0005F68C 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF indirect blocks 0xFFFFFFFF 0xFFFFFFFF 0xFFFFFFFF mtime Dec 1 03:20:24 2022 atime Dec 1 03:20:26 2022 ctime Dec 1 03:20:24 2022 create date Dec 1 2022 not encrypted not compressed PDS model cyl dsorg(po) lrecl(180) recfm(fb) space(5,1) blksize(0) dsntype(library) PS model cyl dsorg(ps) lrecl(300) recfm(vb) space(5,1) blksize(0) vnode,vntok 0x00000050,,0x072011D0 0x015542A0,,0x00000000 opens oi=0 rd=0 wr=0 file segments na file unscheduled na meta buffers 0 dirty meta buffers 1 215:/dsfs/txt $ cd demi.jobs 216:/dsfs/txt/demi.jobs $ touch testjcl 217:/dsfs/txt/demi.jobs $ mkdir jcl 226:/dsfs/txt/demi.jobs $ ls -al total 48 drwxrwxrwx 4 bpxroot sys1 8192 Dec 2 02:48 . drwxrwxrwx 7 bpxroot sys1 8192 Dec 1 04:57 .. drwxrwxrwx 2 bpxroot sys1 0 Dec 2 02:48 jcl drwxrwxrwx 2 bpxroot sys1 8192 Dec 2 00:00 plex -rwxrwxrwx 1 bpxroot sys1 0 Dec 2 00:00 testjcl |
Once a HLQ directory has been created, it can only be deleted if the HLQ directory name is excluded by the HLQ list.
For example, in our zPET environment, try to delete the HLQ directory name sys1.parmlib.backup.d082517.
164:/dsfs/txt $ ls demi demi.parmlib.backup demi.parmlib2 dsfsauto sys1.parmlib.backup.d082517 demi.jobs demi.parmlib.backup.demi demi.parmlib3 sys1 sys1.parmlib2 demi.parmlib demi.parmlib1 demitest sys1.parmlib 167:/dsfs/txt $ dsadm config -hlq_list_add sys1.parmlib.backup.d082517 IDFS00326I Successfully added the specified high-level qualifier directory names to the HLQ list. 168:/dsfs/txt $ dsadm configquery -hlq_list The value for -hlq_list is SYS1.PARMLIB.BACKUP.D082517. 169:/dsfs/txt $ ls demi demi.parmlib.backup demi.parmlib2 dsfsauto sys1.parmlib2 demi.jobs demi.parmlib.backup.demi demi.parmlib3
sys1 demi.parmlib demi.parmlib1 demitest sys1.parmlib
|
We can see the HLQ directory sys1.parmlib.backup.d082517 has been successfully deleted.
OK, that’s all our introduction for multiple HLQs support. We welcome any feedback, suggestion, or comments.
For more information about DSFS, please refer to the Administration guide
Author:
Yu Mei, Dai (dyubj@cn.ibm.com)
Alfred Lease (alease@us.ibm.com)
Kieron D Hinds (kdhinds@us.ibm.com)