Originally posted by: ashw_r

Hi All,

I have situation where in I am creating a typetree for third party data, but the problem is it has a variable length field with no terminator. For example below is the structure of one segment(record) in flat string.

Below table depicts record properties.

Field-----------------Length------------Comments
SegmentID-------------2Bytes------------"CC" (constant value, which can be used as intiator in WTX)
SegmentLength---------2Bytes------------Indicates length of whole subsegment (including indicator: CC).
SegmentData-----------1 to 40Bytes------Variable length field without a terminator

Sample Data for this segment: CC14SAMPLEDATA
(SegmentID= CC, SegmentLen= 14, Data=SAMPLEDATA (10Bytes), total 14bytes including all segments)

Also This segment is preceeded and succeeeded by other type of segments. And even fun part is, this segment can be repeated.

In this situation, Here I cant use SIZED Component attributes on SegmentLength, because SIZED attribute applys to the succeeding componets. Here in my case segmentLength 14 byte including preceeding field SegmentID.

I need to create the type tree for this type of data organization.

Please anyone who faced similar situation or who know how to handle this situation, give me your valuable input.

Note:At this situation
No Pading Rule
No Terminator

Thanks and Regards
Ashwin
#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender
#DataExchange

Originally posted by: TarunB

You could create a pre-process map to read in the data up to the SegmentLength field and then ignore everything after it.

So in your example below, read in SegmentID and SegmentLength (both Fixed length 2 I presume) and treat the rest of the record as a blob.

CC14{...blob...}

In this pre-process map you then subtract 4 from the SegmentLength value, but don't edit any other data in the file. So you end up with

CC10{...blob...}

Output the resulting file (which is basically the same file with only the SegmentLength values reduced by 4), and use this file to trigger the actual map. In the next map you would now be able to used the SIZED attribute.

It's a lot simpler than this message makes it sound.
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: NancyR

If your first two fields are both fixed at a length of 2 bytes each, use a component rule on the third field that tests it for a length of the value of the second field minus 4. Using a component rule on the first field of $="CC" should take care of the repetition. You might also want to apply the Identifier attribute to that first field.

Or you could define "CC" as the initiator of the group and then use a rule on the third field that tests it for a length of the value of the second field minus 2.
#DataExchange
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender

Originally posted by: NancyR

Did you try my suggestion? In your example, when WTX sees the KK the engine will understand it is no longer looking at CC segments. So it will not try to validate the KK12A1B2C3D4 part of the string as a CC segment. If your data will have all the CC segments followed by all the KK segments, your type tree should have a group that has CC_segment(s) followed by KK_segment(s) both with a range of 0:s.

If the CC and KK segments could be in any order you'll need to define the containing group as unorderd.
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: ashw_r

@Tarun and @Nancy,

Thanks for your suggestion.

@Tarun
You are correct, when we have only one such segment or record is present.

@Nancy:

what i understand from component rule is that: these rules will be applied as validation once the data sits on TypeTree virtually.

In my case there could be any number segments after CC segment, and more over different type of segment altogether.

For Example
CC Segment : SegmentID =CC, SegmentLenth=14 SegmentDate="ABCDEFGHIJ" Total 14 Bytes of segment.
CC14ABCDEFGHIJ

KK Segment: SegmentID =KK, SegmentLenth=12 SegmentDate="A1B2C3D4" Total 12 Bytes of segment.
KK12A1B2C3D4

Now if the Flat String looks like, CC + KK segments: CC14ABCDEFGHXYKK12A1B2C3D4

Even I thought of preprocessing Map, but it doesnt work when have data like above.

Is there any other in built function or feature in WTX, where in I can programaticaly control how data sits on TypeTree virtually?
#IBM-Websphere-Transformation-Extender
#DataExchange
#IBMSterlingTransformationExtender

Originally posted by: TarunB

If you know the segment initiators you could use FIND in the pre-processing map to locate the starting position of the segment and then subtract 4 as detailed in my previous response.

Or you could try contacting the source data owner and asking them to change the data!
#IBM-Websphere-Transformation-Extender
#DataExchange
#IBMSterlingTransformationExtender

Originally posted by: PradeepV.B.

Hi,

I have attached a sample example that contains a map which handles variable length field in a fixed file. What you can do is that, create an item with size = 1 with range as 0 to s that will act as the variable length field. Add a component rule on this field that will check the count of this field against the lenght of the segment-(sum of size of fixed fields). For example

COUNT($)<=(SegmentLen Field-SIZE(Name Field)-SIZE(Age Field)-4)

Where $ is 1 byte field.

When you read this you have to use Seriestotext function to combine all the bytes.

Hope this solves the problem.

Thanks,
Pradeep
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange