Originally posted by: rarw

Hi,

I am processing a cobol flat file and need to selectively pick out some of the data. I am using the LEFT function to do this, but it is not working when the ETHNICITY field is composed of spaces instead of alpha data. There are other fields, the race codes, in the record that are filled with spaces and there does not seem to be a problem with these.

The incorrect output that is produced has an extra blank character, this can be seen below in the example output. For visibility, in the description below, I have replaced blank characters (spaces) with periods.

This seems like an easy operation to apply and why it is adding an extra blank char, or not taking a blank char out of the output, is a mystery to me.

In the description below the field name is followed by the size of the field. All fields are text and padded with spaces.

Thank you for your time and help!
Becky
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Example Input:

PRIMARY_LANGUAGE_3 = EN. record 1, = EN. record 2
ETHNICITY_2 = .. record 1, = NH record 2
RACE1_2 = WH record 1, = AA record 2
RACE2_2 = .. record 1, = .. record 2
RACE3_2 = .. record 1, = .. record 2
RACE4_2 = .. record 1, = .. record 2
RACE5_2 = .. record 1, = .. record 2

Records:
record 1: EN...WH........
record 2: EN.NHAA........

Total record length = 15
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Example Output:

PRIMARY_LANGUAGE_3
ETHNICITY_1 LEFT(ETHNICITY_2,1)
RACE1_1 LEFT(RACE1_2,1)
RACE2_1 LEFT(RACE2_2,1)
RACE3_1 LEFT(RACE3_2,1)
RACE4_1 LEFT(RACE4_2,1)
RACE5_1 LEFT(RACE5_2,1)

Records:
Record 1: EN...W.... (incorrect output) Want: EN..W....
Record 2: EN.NA.... (correct output)

Expected total record length = 9
#DataExchange
#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender

Originally posted by: janhess

Has this to do with the way you have padding defined in your type tree?
Text fields are usually padded with spaces for Cobol layouts so if the data is all spaces it will be treated as empty so LEFT(x,1) will give nothing. Output padding depends on whether it is applied to Any Context or Fixed Group.
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: rarw

Thank you for your quick reply.

For all of the fields above the settings are as follows:

Min: 0
Max: X (see sizes above per field)
Pad: Yes
Value: <SP>
Padded To: Fixed Size
Length: X
Justify: Left
Apply Pad: Any Context

Is "Any Context" correct for fields that are sometimes filled with spaces and a LEFT function must be applied? Or, do we need to use "Fixed group"? This is one subject that is difficult to relate to in the docs!

Thanks- Becky
#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender
#DataExchange

Originally posted by: janhess

On output the AnyContext should pad the field to fixed size. You may need to do an if(present(field),left(field,1)). What version are you using?
#DataExchange
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender

Originally posted by: rarw

Version 8.1
Build 114

Thanks, I will try your suggestion!
Becky
#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender
#DataExchange

Originally posted by: janhess

I did a test with the attached map and it seemed to work ok with LEFT. The only record it didn't produce was when both input fields were empty.
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: janhess

Sorry, forgot to save map before I zipped it.
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: rarw

Yeah!!!

This works for ETHNICITY when it is blank (going from 2 spaces to one space):

=IF(PRESENT(R509_ETHNICITY Field:In1), LEFT(R509_ETHNICITY Field:In1, 1), " ")

But I do not understand why this must be used with ETHNICITY but no problem arises with the race codes. Race codes 2 thru 5 are also spaces and also have the Left(field,1) applied to a two char field - going to one char.

Does this make any sense?

Thanks for your help!
Becky

#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: janhess

It's either a bug or you have the dields defined differently in the type tree.
Can you post the tree, map and sample data?
#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender
#DataExchange

Originally posted by: rarw

Attached is a zip file with the type tree, map, input file and two output files. If anything else is needed, please let me know.

These maps also relate to my additional question pertaining to the "Component Range not outputing correct record length". The incorrect output file is ArchFile.txt. The desired group is (24:24) and TEXT_BLOB_36 for both Elig group and Auth group.

The output file relating to this post question (Left function) is file MAR509_output.txt. (Ethnicity and the 5 race codes)

There is one input file (BDCH01S_test_forum.txt) for both outputs.

Thanks for your help in looking into this.
Becky
#DataExchange
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender

Originally posted by: rarw

Additional Notes:
(1)
The race codes have the rule '=IF(PRESENT(R509_RACE_1 Field:In1), LEFT(R509_RACE_1 Field:In1, 1))' , I had meant the rule to be '=IF(PRESENT(R509_RACE_1 Field:In1), LEFT(R509_RACE_1 Field:In1, 1), " ")' - with "else" logic in rule

(2)
If you take the space (" ") out of the ethnicity rule "else" logic '=IF(PRESENT(R509_ETHNICITY Field:In1), LEFT(R509_ETHNICITY Field:In1, 1), " ")', the record length output is incorrect.

Becky
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: janhess

Looks like a bug. If you specify the text_blob_36 as min 36 max 36 pad = no then it handles the field ok otherwise it seems to treat it as 34 characters according to debug.
#DataExchange
#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender

Originally posted by: rarw

Thanks for all of your help janhess. I implemented your suggestions and the file output for MAP 1 is now correct for both the ethnicity issue and the text_blob_36 issue.

Relative to the ethnicity field, I have found some quirky memory issues with the second map (Map 2) that runs off of the output of the first map (Map 1). Map 2 uses the same type tree and fields as the MAP 1, plus some additional fields. I am seeing that the ethnicity problem strangely persists in Map 2 if it is blank, even though the input and output field size is one (the ethnicity field is output incorrectly as two spaces if it is empty). To, get around this, I have defined logic to set it to "N" if it is empty (=IF(PRESENT(ETHNICITY Field:In1), LEFT(ETHNICITY Field:In1, 1), "N")). (Repeating the logic in Map 1 of =IF(PRESENT(ETHNICITY Field:In1), LEFT(ETHNICITY Field:In1, 1), " ") did not work.) I would like to understand why it is happening between map runs when the input data for MAP 2 is not 2 char but one!

Maybe we are incorrect in our technique of having more than one map utilize the same type tree, using some of the same field names, even though the maps are run separately. Is the memory for the variables released per map run? Oddly, it looks like it is not. Would it be better to utilize a different type tree for each map, or is our implementation common practice?

Without knowing the data structure and memory allocation behind the WTx implementation it is difficult to surmise what the issue is, but to me it seems like there is a memory issue here. However the data structure is stored, whether it is by a list, or array, or something else, the data storage appears to keep the original 2 char ethnicity if it is empty. This is all "arm waving", but it seems that if the problems are carried over from one map to the next (separate runs using the same type tree) that there is a memory issue (sharing) since the input of the second map has an ethnicity of one char and not two. But, when it is empty it is being set to two char even though the output field size is also one!

I also have a filler issue, where only 7 spaces are output instead of the 11 expected (min=0, max=11, pad=yes, value=<SP>, padded to=Fixed Size, length=11, justify=left, apply pad=any context). This seems like simple output to create; I have tried various settings to no avail.

Thanks for all of your help,
Becky

#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender
#DataExchange

Originally posted by: janhess

Have you raised this with IBM support? I think it is a bug. They probably have a fix available as it must be affecting a number of sites.
#IBM-Websphere-Transformation-Extender
#DataExchange
#IBMSterlingTransformationExtender