Nice, I'm glad I was able to help you.
Original Message:
Sent: Tue May 07, 2024 09:02 AM
From: John Bourdeau
Subject: parsing JSON
Thanks! This is exactly what I need. I really appreciate the help.
John
------------------------------
John Bourdeau
Original Message:
Sent: Mon May 06, 2024 09:21 AM
From: Angelo Alves
Subject: parsing JSON
You can use regular expression, follow the code.
defVar --name json --type StringdefVar --name vDT --type DataTabledefVar --name generated_text --type StringdefVar --name generated_token_count --type NumericdefVar --name input_token_count --type StringdefVar --name stop_reason --type StringdefVar --name model_id --type StringdefVar --name model_version --type StringdefVar --name created_at --type StringdefVar --name loop --type NumericdefVar --name policy --type StringdefVar --name person --type StringdefVar --name dob --type StringdefVar --name dl --type StringdefVar --name action --type StringdefVar --name effective --type StringsetVar --name "${json}" --value " {\n \"model_id\": \"ibm/granite-13b-chat-v2\",\n \"model_version\": \"2.1.0\",\n \"created_at\": \"2024-04-24T19:42:33.682Z\",\n \"results\": [\n {\n \"generated_text\": \" Named Entities: Policy Number: 72336374\\nPerson: German Martinez, DoB: 07/09/1978, DL: M985-863-98-165-8\\nAction: Add, Effective Date: 06/06/2022\",\n \"generated_token_count\": 55,\n \"input_token_count\": 2175,\n \"stop_reason\": \"eos_token\"\n }\n ]\n }"mapJson --handleError --json "${json}" --mappings "{\"model_id\":\"${model_id}\",\"model_version\":\"${model_version}\",\"created_at\":\"${created_at}\"}"jsonToTable --json "${json}" --jsonPath "$.results" vDT=valuefor --variable ${loop} --from 1 --to ${vDT.Rows} --step 1 mapTableRow --dataTable ${vDT} --row ${loop} --mappings "[{\"Name\":\"generated_text\",\"Number\":null,\"Output\":\"${generated_text}\"},{\"Name\":\"generated_token_count\",\"Number\":null,\"Output\":\"${generated_token_count}\"},{\"Name\":\"input_token_count\",\"Number\":null,\"Output\":\"${input_token_count}\"},{\"Name\":\"stop_reason\",\"Number\":null,\"Output\":\"${stop_reason}\"}]" getRegex --text "${generated_text}" --regexPattern "Policy Number: (?<policy>\\d+).+Person: (?<person>.+?)\\, DoB: (?<dob>.+?), DL: (?<dl>.+?)\\nAction: (?<action>.+?), Effective Date: (?<effective>.+)" --regexOptions "Singleline" --groupname policy policy=value getRegex --text "${generated_text}" --regexPattern "Policy Number: (?<policy>\\d+).+Person: (?<person>.+?)\\, DoB: (?<dob>.+?), DL: (?<dl>.+?)\\nAction: (?<action>.+?), Effective Date: (?<effective>.+)" --regexOptions "Singleline" --groupname person person=value getRegex --text "${generated_text}" --regexPattern "Policy Number: (?<policy>\\d+).+Person: (?<person>.+?)\\, DoB: (?<dob>.+?), DL: (?<dl>.+?)\\nAction: (?<action>.+?), Effective Date: (?<effective>.+)" --regexOptions "Singleline" --groupname dob dob=value getRegex --text "${generated_text}" --regexPattern "Policy Number: (?<policy>\\d+).+Person: (?<person>.+?)\\, DoB: (?<dob>.+?), DL: (?<dl>.+?)\\nAction: (?<action>.+?), Effective Date: (?<effective>.+)" --regexOptions "Singleline" --groupname dl dl=value getRegex --text "${generated_text}" --regexPattern "Policy Number: (?<policy>\\d+).+Person: (?<person>.+?)\\, DoB: (?<dob>.+?), DL: (?<dl>.+?)\\nAction: (?<action>.+?), Effective Date: (?<effective>.+)" --regexOptions "Singleline" --groupname action action=value getRegex --text "${generated_text}" --regexPattern "Policy Number: (?<policy>\\d+).+Person: (?<person>.+?)\\, DoB: (?<dob>.+?), DL: (?<dl>.+?)\\nAction: (?<action>.+?), Effective Date: (?<effective>.+)" --regexOptions "Singleline" --groupname effective effective=value logMessage --message "\r\nPolicy Number: ${policy}\r\nPerson: ${person}\r\nDoB: ${dob}\r\nDL: ${dl}\r\nAction: ${action}\r\nEffective Date: ${effective}" --type "Info"nextreturn
------------------------------
Angelo Alves
Original Message:
Sent: Fri May 03, 2024 05:16 PM
From: John Bourdeau
Subject: parsing JSON
it helped a lot.
How would I get the values within the "generated_text" row borken into their named pairs. For example, Número da política: 72336374 as a single variable?
Thanks again for the help!
------------------------------
John Bourdeau
Original Message:
Sent: Thu May 02, 2024 01:14 PM
From: Angelo Alves
Subject: parsing JSON
Hi John
I solved your problem by following the steps.
- First I used mapJson to map the JSON root values.
- Then I used jsonToTable to create a table with the items in the Result list.
- Finally, I used a loop to map each row of the table with mapTable.
defVar --name json --type StringdefVar --name vDT --type DataTabledefVar --name generated_text --type StringdefVar --name generated_token_count --type NumericdefVar --name input_token_count --type StringdefVar --name stop_reason --type StringdefVar --name model_id --type StringdefVar --name model_version --type StringdefVar --name created_at --type StringdefVar --name loop --type NumericsetVar --name "${json}" --value " {\n \"model_id\": \"ibm/granite-13b-chat-v2\",\n \"model_version\": \"2.1.0\",\n \"created_at\": \"2024-04-24T19:42:33.682Z\",\n \"results\": [\n {\n \"generated_text\": \" Entidades nomeadas: Número da política: 72336374\\nPessoa: German Martinez, DoB: 07/09/1978, DL: M985-863-98-165-8\\nAção: Adicionar, Data de Vigência: 06/06/2022\",\n \"generated_token_count\": 55,\n \"input_token_count\": 2175,\n \"stop_reason\": \"eos_token\"\n }\n ]\n }"mapJson --handleError --json "${json}" --mappings "{\"model_id\":\"${model_id}\",\"model_version\":\"${model_version}\",\"created_at\":\"${created_at}\"}"jsonToTable --json "${json}" --jsonPath "$.results" vDT=valuefor --variable ${loop} --from 1 --to ${vDT.Rows} --step 1 mapTableRow --dataTable ${vDT} --row ${loop} --mappings "[{\"Name\":\"generated_text\",\"Number\":null,\"Output\":\"${generated_text}\"},{\"Name\":\"generated_token_count\",\"Number\":null,\"Output\":\"${generated_token_count}\"},{\"Name\":\"input_token_count\",\"Number\":null,\"Output\":\"${input_token_count}\"},{\"Name\":\"stop_reason\",\"Number\":null,\"Output\":\"${stop_reason}\"}]"next
Hope this helps.
------------------------------
Angelo Alves
Original Message:
Sent: Wed May 01, 2024 04:13 PM
From: John Bourdeau
Subject: parsing JSON
I'm trying to parse a JSON string.
This is the JSON
{"model_id":"ibm/granite-13b-chat-v2","model_version":"2.1.0","created_at":"2024-04-24T19:42:33.682Z","results":[{"generated_text":" Named Entities: Policy Number: 72336374\nPerson: German Martinez, DoB: 07/09/1978, DL: M985-863-98-165-8\nAction: Add, Effective Date: 06/06/2022","generated_token_count":55,"input_token_count":2175,"stop_reason":"eos_token"}]}
First step I convert the JSON to table
jsonToTable --json "{\n \"model_id\": \"ibm/granite-13b-chat-v2\",\n \"model_version\": \"2.1.0\",\n \"created_at\": \"2024-04-24T19:42:33.682Z\",\n \"results\": [\n {\n \"generated_text\": \" Named Entities: Policy Number: 72336374\\nPerson: German Martinez, DoB: 07/09/1978, DL: M985-863-98-165-8\\nAction: Add, Effective Date: 06/06/2022\",\n \"generated_token_count\": 55,\n \"input_token_count\": 2175,\n \"stop_reason\": \"eos_token\"\n }\n ]\n}" --jsonPath "$" vDT=value tableRows=rows tableColumns=columns
Next I try and extract a cell using "findTableCell"
findTableCell --dataTable ${vDT} --value "[Policy Number]" --search "AllCells" --direction "Left2RightTop2Bottom" --occurrencetype "First" searchSuccess=success rowNumber=row columnsNumber=column columnName=columnName cellContent=value occurrencesQuantity=count
When I list the data table created as vDT this is what I get:
Table vDT: ibm/granite-13b-chat-v2, 2.1.0, 4/24/2024 7:42:33 PM, [
{
"generated_text": " Named Entities: Policy Number: 72336374\nPerson: German Martinez, DoB: 07/09/1978, DL: M985-863-98-165-8\nAction: Add, Effective Date: 06/06/2022",
"generated_token_count": 55,
"input_token_count": 2175,
"stop_reason": "eos_token"
}
]
Rows: 1
Columns: 4
No matter what I do the value is blank. What I need are the values for "Policy Number", "Person", "DoB", etc.
Any ideas?
------------------------------
John Bourdeau
------------------------------