Originally posted by: paul.brett

When using the IBM WebSphere Transformation Extender (WTX) engine to validate EDI data that is known to be good, you can sometimes get the input marked as invalid for no immediately human visible reason. In other words, when a human looks at the data, it looks 'valid'; however, the WTX engine flags the data as invalid.

Why does this happen? In some cases the human brain makes assumptions about the syntax boundaries of the data that the WTX engine will not make. Fortunately WTX is designed to handle the modelling of any structured complex data so the resolution to such problems requires just modifications to the WTX Typetree (metadata representations) to be more specific to allow the parser to correctly determine the delimiters in the data.

In the following description we will assume that the segment terminator (which in the real world is variably determined) has been established to be a tilde "~". In this case, analysis of the trace shows that a RESTART is attempted. Replacing some of the values within the EDI allows the data to pass validation. The specifics of the data that causes this issue are:

- Any string that ends with GE and then a "~". (Example GEORGE~)
- Any string that ends with ISA and then a "~". (Example LISA~)

This has become known as the GEORGE/LISA issue.

In one example case, both components of the GE Segment were optional, so when we see 'GE' followed by the terminator for this type WTX finds that to be a valid instance of that type. That gets us out of our RESTART, but we're not in an appropriate place in the data for our supposed place in the Typetree, and we find the data to be invalid.

The use case can be resolved by tightening the definition of the Typetree properties, by forcing some of the components to be mandatory instead of them all being optional.

This is not a product limitation - it is a side effect of having such a robust and powerful data modelling tool. The GEORGE and LISA 'problem' is not a product problem at all....it's an opportunity to use the flexible tool set provided by WTX - with unintended side effects if the Typetree is not specific enough to help the parser properly find all syntax..

Thank-you

Paul   (with lots of help from Stephanie Fetzer and Bob Connolly)

Follow me on Twitter