We are playing with JSON generation from zSecure to feed into Python and wonder about the structure of the data that's produced.  This works very well and straight-foward, esp. after applying UJ98240 (thank you!).

First we wrote some newlists without a name:

fileoption dd=json fileformat=json  
print dd=json  
 
newlist type=run  
  sortlist datetime
 
newlist type=racf 
  s class=user cggrpnm=sys1
  sortlist class profile name instdata 
 
newlist type=racf 
  s class=group mask=sys1  
  sortlist class profile userid instdata

This produces a list of dicts, which is fair enough:

{"JSON":[
  {
   "DATETIME":"20Nov25 05:55:57"
  }
 ,
  { 
   "CLASS":"USER"  
  ,"PROFILE":"CEA" 
  ,"NAME":"COMMON EVENT ADAPTER"  
  } 
 ,{ 
   "CLASS":"USER"  
  ,"PROFILE":"IBMUSER"
  ,"NAME":"DON'T EVER USE"  
  ,"INSTDATA":"TOP OF GROUP TREE" 
  } 
 ,  
  { 
   "CLASS":"GROUP" 
  ,"PROFILE":"SYS1"
  ,"USERID":[
    "IBMUSER"
   ,"CEA" 
   ]
  } 
]} 

So we could write things like

for line in data["JSON"]:
  if line.get("CLASS"," ")=="USER":
    print(f'{line["PROFILE"]}') 

However, with more newlists added, the python code could be better structured by adding newlist names in the data:

fileoption dd=json fileformat=json  
print dd=json  
 
newlist type=run  
  sortlist datetime
 
newlist type=racf name=users
  s class=user cggrpnm=sys1
  sortlist class profile name instdata 
 
newlist type=racf name=groups
  s class=group mask=sys1  
  sortlist class profile userid instdata   

but this changed the data structure in an unexpected way. Instead of creating a dict of lists, the structure has a list of dicts of dict.  Note, I omitted the name of the first newlist to illustrate the asymmetry:

{"JSON":[  
  {  
   "DATETIME":"20Nov25 05:55:57"
  }  
 ,
  {"USERS":
   { 
    "CLASS":"USER"  
   ,"PROFILE":"CEA" 
   ,"NAME":"COMMON EVENT ADAPTER"  
   } 
  }  
 ,{"USERS":
   { 
    "CLASS":"USER"  
   ,"PROFILE":"IBMUSER"
   ,"NAME":"DON'T EVER USE"  
   ,"INSTDATA":"TOP OF GROUP TREE" 
   } 
  }  
 ,
  {"GROUPS":  
   { 
    "CLASS":"GROUP" 
   ,"PROFILE":"SYS1"
   ,"USERID":[
     "IBMUSER"
    ,"CEA" 
    ]
   } 
  }  
]}

To process this you would have to concern yourself with dicts of str vs. dict of dicts, and write

for line in data["JSON"]:
  if isinstance(line,dict) and "USERS" in line and isinstance(line["USERS"],dict):
    print(f'{line["USERS"]["PROFILE"]}')

or, if you like to write code that you won't understand in a week from now:

for line in data["JSON"]:
  if isinstance(line.get("USERS",None),dict):
    print(f'{line["USERS"]["PROFILE"]}')

but it would seem to be more pythonic to produce a dict of lists, where the dict index is the newlist name:

{"JSON":{
 "":[
  { 
  "DATETIME":"20Nov25 05:55:57"
  }
 ]
,
 "USERS":[
  {
   "CLASS":"USER" 
  ,"PROFILE":"CEA"
  ,"NAME":"COMMON EVENT ADAPTER" 
  }
 ,
  {
   "CLASS":"USER" 
  ,"PROFILE":"IBMUSER"
  ,"NAME":"DON'T EVER USE"
  ,"INSTDATA":"TOP OF GROUP TREE"
  }
 ]
,
 "GROUPS":[
  {
   "CLASS":"GROUP"
  ,"PROFILE":"SYS1"
  ,"USERID":[
   "IBMUSER"
   ,"CEA"
   ]
  },
 ]
}}

so you could address the structure as a simple dict:

users = data["JSON"].get("USERS",[])
for u in users:
  print(f'{u["PROFILE"]}')

You can, of course, create the users list from the existing list of dicts of dict structure using comprehension

users = [l["USERS"] for l in data["JSON"] if "USERS" in l]

Can anyone shed light on the reason why the dict of dicts structure was chosen?

------------------------------
Rob van Hoboken
------------------------------

Hello Rob,

That's interesting idea, what you're trying to achieve - thanks for sharing. I tried to run this code and for newlist type=run I only find parameter datetime not rundate. Did you make definition for rundate somewhere?

It worked for me to run with datetime well.

and next question - when I ran the code, the results, there results must contain [, ], { and }, but instead they have some special characters - do you use in your code some encoding parameters?

Thank you for catching that, I had to juggle to copy code + data between my 2 laptops and copied the old CARLa code (which did not run).  I fixed the CARLa in the original post.

In the fileoption command you can also specify encoding=utf-8 to generate ascii-like data which you can file-transfer in binary mode.  See the manual.  If you run without UJ98240, a bug in formatting repeated fields (like CONGRPNM) writes non-printable data instead of a comma between list elements.  The following code catches the JSON exception,  though.

try: 
  with open(jsonfile,'r') as input:
    data = json.load(input)
except UnicodeDecodeError:
  # APAR OA68561, PTF UJ98240 should fix this  
  try:  
    # Read the file as bytes 
    with open(jsonfile, 'rb') as f:
        byte_data = f.read() 
    # Replace x'c222' with x'2c22' 
    modified_bytes = byte_data.replace(b'\x20\xc2\x22', b'\x20\x2c\x22')
    # Convert bytes to string (using UTF-8, replace errors with ?)  
    modified_str = modified_bytes.decode('utf-8', errors='replace') 
    data = json.loads(modified_str)
  except Exception as e:  
    sys.stderr.write(f"fix for repeated field did not work: {e}\n") 
    data = None  
except Exception as e: 
  sys.stderr.write(f"json load did not work: {e}\n") 
  data = None                                   

------------------------------
Rob van Hoboken
------------------------------

Thanks Rob