Originally posted by: wcartmill

We have a requirement to receive UTF-8 from a customer, but submit ISO-8859-1 to the application. Since UTF-8 has many more characters than ISO-8859-1, the app wants us to convert the "extras" to a defined ASCII equivalent, for example ć => c and Đ => Th or D, depending on the customer. Has anyone been able to resolve this issue?

thanks
--Wendy
#DataExchange
#IBM-Websphere-Transformation-Extender
#IBMSterlingTransformationExtender

Originally posted by: SystemAdmin

I don't think there's going to be an easy answer for this - doing a direct convert will mean that you lose the characters that can't be converted (iconv behaves in the same way).

As a work-around I would read the data in as a UTF-8 blob initially and then use the substitute function to replace characters:

=SUBSTITUTE(utf8blob,
"ć",
"c",
"Đ",
"D",
"ű",
"u")

#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: wcartmill

We've been looking at that possibility, and feel it will work for up to a few dozen entries. Our current list is over 300, so we're not sure it's feasible. Anyone seen non-WTX tools that can do this? Thanks for your input.
#IBM-Websphere-Transformation-Extender
#DataExchange
#IBMSterlingTransformationExtender

Originally posted by: SystemAdmin

If I understand your issue correctly, you might want to use an exclusive restriction list by characters, with a replacement string for each char that you might want to transcode on your output typetree. Do that change to the supertype of your items and use the propagate feature to apply to all items.
#IBMSterlingTransformationExtender
#DataExchange
#IBM-Websphere-Transformation-Extender

Originally posted by: wcartmill

Thanks a lot! That's exactly what we need.
#DataExchange
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender

Originally posted by: SystemAdmin

You are welcome, don't hesitate to post again if you have problem implementing that solution
#IBMSterlingTransformationExtender
#IBM-Websphere-Transformation-Extender
#DataExchange

Originally posted by: SystemAdmin

Hi Wendy,
I've done some more investigation and the solution at http://www.ahinea.com/en/tech/accented-translate.html should work for you.
(searching for "convert diacritic characters base" throws up more pages too).

Basically you'll need to call something external, e.g. Perl or Java, to convert the character into a Unicode decomposable format - you then strip off the diacritics.

Olly.
#IBMSterlingTransformationExtender
#DataExchange
#IBM-Websphere-Transformation-Extender