IBM Z and LinuxONE - IBM Z - Group home

Using grep with z/OS Data Sets

By Lionel Dyck posted Sat July 24, 2021 12:19 PM

  
How many times have you needed to search a z/OS data set for a string of text? For me, that happens several times a week.  There are several options available to do this, but I want to focus on how to do this using grep from within an application written in REXX.

The easy way to do this, if you have access to TSO PIPEs, would be something as simple as this:

   'pipe < data.set.name | locate /string/ | stem results.'

This would find the string and each record with that string would be placed into the results. stem variable. [for those with z/VM you have CMS PIPEs available with z/VM but for the z/OS installation you need to buy IBM's SmartBatch - sometimes referred to as BatchPipes). However before leaving the topic of PIPEs I want to encourage you to check out the RFE requesting IBM to detach PIPEs from the Batch product and include it as part of the z/OS base - it can be found at https://tinyurl.com/yafnhmrb as RFE 47699 and is THE TOP voted RFE for all RFE's.

So, let's look at using grep since you probably do not have PIPEs available.

The first thing to know is that grep is an OMVS command and to use it from z/OS using REXX requires using the BPXWUNIX function. Also know that the grep command has no awareness of z/OS data sets.

Here is a sample REXX program that uses a generalized subroutine that I've written to invoke grep.
/* rexx */ 
arg file string
x = searchstring(file,string)
if x = 0 then do
say 'No results found for:' string
exit 4
end
do i = 1 to results.0
say results.i
end
exit


The third statement in this REXX program will invoke the searchstring subroutine and is found later in the same REXX program. The reason the searchstring must be a subroutine within the same REXX program is because REXX does not support global variables and we need the resulting stem available in the mainline code.

The searchstring subroutine is this set of code:
 /* -------------------------------------------------------- * 
| Search String subroutine. (copy into main REXX) |
| |
| x = SearchString(dsname,string) |
| |
| NOTEs: |
| 1. dsname will use the users TSO Profile Prefix |
| if not qualified in quotes |
| 2. Results will be returned in the results. stem |
| |
| String may be: string |
| string1|string2 (or) |
| sgring1.*string2 (and) |
| |
| Uses OMVS cat to extract the data and pipes to OMVS |
| grep to do the case insensitive search. |
| |
| The $ is escaped if found in the dataset name. |
* -------------------------------------------------------- */
SearchString: Procedure Expose Results.
file = arg(1)
string = arg(2)

x = listdsi(file)
if x > 0 then do
say "'"file"'" sysdsn(file)
exit 16
end

if pos('(',file) > 0 then do
parse value file with '('mem')'
sysdsname = sysdsname'('mem')'
end

if pos('$',sysdsname) > 0 then
do forever
if pos('\$',sysdsname) > 0 | pos('$',sysdsname) = 0 then leave
p = pos('$',sysdsname)
ld = left(sysdsname,p-1)
rd = substr(sysdsname,p+1)
sysdsname = ld'\$'rd
end

if pos('$',string) > 0 then
do forever
if pos('\$',string) > 0 | pos('$',string) = 0 then leave
p = pos('$',string)
ld = left(string,p-1)
rd = substr(string,p+1)
string = ld'\$'rd
end


dsn = '"//'||"'"sysdsname"'"||'"'

drop results.
x = bpxwunix('cat' dsn "| grep -E -i '"string"'",,results.,e.)


if e.0 > 0 then do
say ' '
say 'errors:'
do i = 1 to e.0;say e.i;end
end
return results.0


There are two interesting parts to this code.  The first interesting part if the section that checks for a $ in the data set name and then in the search string. If a $ symbol is found, then it must be escaped so that the BPXWUNIX processing will not attempt to use it as a shell variable.

The second interesting part of the code, and the one that does the work, is this command:

  x = bpxwunix('cat' dsn "| grep -E -i '"string"'",,results.,e.)


This invokes the BPXWUNIX function, which invokes the OMVS shell to process OMVS commands. The 1st command is cat, which is used to extract the z/OS data set and then pipe it using the shell pipe symbol of |, into the grep command. The grep command uses the -E to function in egrep mode (that is a discussion for another time), and the -i option to instruct grep to ignore the case (upper/lower/mixed) of the string while searching.

The BPXWUNIX returns the results in the stem results. with any error messages in the stem e..

The sample subroutine checks both stems and then if there are errors those will be displayed using the REXX say command. The subroutine exits with a return code containing the number of records found (results.0).

I hope y'all find this useful.

3 comments
82 views

Comments

Sat July 31, 2021 08:26 AM

Just for info, Netview also has Pipes and although not as complete as the CMS version, you can do a lot with them such as searching in datasets. Admittedly not everyone has Netview either but if you have access to systems built with the ADCD then that has Netview installed, just needs to be configured. Also somewhere, I have a grep program which is slightly different to the OMVS one which can run in batch or foreground. I'll have to dig it out...

Sebastian.

Mon July 26, 2021 01:16 PM

@Scott Fagen - I agree the RFE probably isn't going to happen at the same time let's keep nudging IBM along. As for recreating PIPEs - I'll leave that to others :)​

Mon July 26, 2021 12:44 PM

Lionel,

A few thoughts:

1.  It is said that necessity is the mother of invention, congratulations on your inventiveness!
2. You talk about RFE 47699.  As this request is already over seven years old and only has medium priority, I would suggest that it ain't coming soon.
3. If the goal is to have 'Pipes-like' syntax, since you've proven that you can achieve the same results with some lines of REXX (or, if more desirable to cover more environments, assembler or Metal C), why don't you create an equivalent interface and you pop it onto the CBT site to see if you can get others to contribute additional capability as the need arises.

-- Scott Fagen

(I had to edit this because the numbered list feature of the editor doesn't work)