MQ

MQ

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Back to discussions
Expand all | Collapse all

Code conversion on MQ using UTF-16, UTF-8, ISO8859-1.....

  • 1.  Code conversion on MQ using UTF-16, UTF-8, ISO8859-1.....

    Posted Sun July 29, 2018 05:46 PM
    Hi,
    We have an article data management process.
    In that process we have a PIM system(IBM's Infosphere master Data management) which runs default on UTF-16. PIM puts a message on MQ which runs default on UTF-8.  Messages are get from MQ by the IBM i(AS400) which runs on UTF-16 and some GET services use CCSID 37. 
    Messages are also get from the queues by datapower Xi52 which inserts the message in a MS SQL Server database that database default runs on ISO8859-1 (latin1 ). The messages which it concerns  are XML messages and have this in the header:  
    <?xml version="1.0" encoding="UTF-8"?>

    Since a while we sometimes see that SQL Server has incomplete messages some characters are replaced with "?" in MS SQL Server. 
    Our requirment is we must be able to process all messages coming from all European Countrys.
    I think the minimal what we need then is UTF-16.  So ISO8859-1 does not fit.
    Our MQ runs default  UTF-8 (1208).  UTF-8 is multibyte. 1,2,3 or 4 bytes that depends on the first 2 bits.  SQL database systems in general don't support UTF8 also IBM i is having a hard time with UTF-8.

    Ok all systems we use(IBM i, MS SQL Server) supports UTF-16.   Also MQ can support UTF-16.
    Our MQ default is at this moment UTF-8. 
    But if a UTF-16 program  puts an UTF-8 XML message on MQ can i then get a problem with code conversion ?  Assuming that all characters in the message are UTF-16 compatible.

    What is the best pracice for MQ if you look at the example above ? 
    Are we running into problems if we stay on UTF-8 as default and should we change it to UTF-16 ?

    Thanks


    ------------------------------
    Bernard Pittens
    Integration Engeneer
    Sligro Foodgroup B.V.
    Veghel
    ------------------------------


  • 2.  RE: Code conversion on MQ using UTF-16, UTF-8, ISO8859-1.....

    Posted Mon August 06, 2018 05:54 AM
    Hi,
    We have worked on this:
    In my mind UTF-16 was only double byte. That's not true, UTF-16 is also multibyte(segments of 2 bytes) if I am correct.  UTF-8 and UTF-16 are both transformabel to the unicode tabel.

    Ok so UTF8 and UTF16 we should be able to support without any problem.
    The real problem we have is the SQL Server instance this is latin-1 because we use varchar and char. If we replace that with nvarchar() and nchar then our problems for now should be fixed.

    Kind regards

    ------------------------------
    Bernard Pittens
    Integration Engeneer
    Sligro Foodgroup B.V.
    Veghel
    ------------------------------

    Original Message