Informix

 View Only
Expand all | Collapse all

ESQL/C: Fetching LVARCHAR with ifx_var_getdata() when DB and client locale do not match

  • 1.  ESQL/C: Fetching LVARCHAR with ifx_var_getdata() when DB and client locale do not match

    Posted Tue February 09, 2021 09:00 AM
    Hello!

    Configuration:

    Platforms: Linux Debian 10.7 or Windows 10
    Informix server: IDS IDS-14.10.FC5
    Informix client: CSDK 4.50.FC5
    DB_LOCALE = en_us.utf8
    CLIENT_LOCALE = en_us.8859-15 (Linux) / en_us.1252 (Windows)

    We experience character encoding conversion issues when fetching an LVARCHAR column, when binding the fetch handler with ifx_sqlvar_t sqltype=CLVCHARPTRTYPE ...

    Then we use the ifx_var_getdata() API to get the string value.

    But it appears that the sequence of bytes returned by the ifx_var_getdata() function is pure UTF-8, and is not converted to ISO-8859-15 ... is this normal or a bug?

    Note: When fetching a CHAR, VARCHAR or LVARCHAR column into an sqlvar C char[n] buffer that is allocated before the fetch, the UTF-8 characters are correctly converted to ISO-8859-15 ...

    Can I find more details about charset conversions with ifx_var_getdata()?

    Maybe we are missing something?

    ------------------------------
    Sebastien FLAESCH
    ------------------------------

    #Informix


  • 2.  RE: ESQL/C: Fetching LVARCHAR with ifx_var_getdata() when DB and client locale do not match

    IBM Champion
    Posted Tue February 09, 2021 10:53 AM

    I have used the mi_get_vardata function inside UDRs, AFAIR you just get the raw data so when you try to globalize your UDR you need to consider server-side locale processing.

     

    Now mi_get_vardata is the engine equivalent of the client library call, but that doesn't mean they are the same

     

    Cheers

    Paul

     






  • 3.  RE: ESQL/C: Fetching LVARCHAR with ifx_var_getdata() when DB and client locale do not match

    Posted Tue February 09, 2021 12:11 PM
    Thank you Paul,

    Logically client-side ifx_var_getdata() should do the charset conversions...

    The documentation of ifx_var_getdata() is quite limited and does not mention charset conversion rules.

    Waiting for IBM/HCL fellows to give some details here.

    ------------------------------
    Sebastien FLAESCH
    ------------------------------



  • 4.  RE: ESQL/C: Fetching LVARCHAR with ifx_var_getdata() when DB and client locale do not match

    IBM Champion
    Posted Tue February 09, 2021 12:28 PM
    You might have a mi_get_vardata in the client libraries - you would need to check the docs. But even if the name is the same then it doesn't mean they are same function

    Cheers
    Paul

    Paul Watson
    Oninit LLC
    +1-913-387-7529
    www.oninit.com
    Oninit®️ is a registered trademark of Oninit LLC





  • 5.  RE: ESQL/C: Fetching LVARCHAR with ifx_var_getdata() when DB and client locale do not match

    Posted Wed February 10, 2021 07:32 AM
    Hi Sebastien,

    It appears that since bind type is pointer (CLVCHAR*PTR*TYPE), it doesn't go through possible conversion. Could you try with VARCHAR or any other suitable binding? Also, it would be good, if you could provide small testcase to understand the problem better.

    Thanks
    -Shesh


    ------------------------------
    Sheshnarayan Agrawal
    ------------------------------



  • 6.  RE: ESQL/C: Fetching LVARCHAR with ifx_var_getdata() when DB and client locale do not match

    Posted Wed February 10, 2021 09:12 AM
    Hi Shesh, thanks for your quick answer.

    The usage context is with our Genero runtime system (www.4js.com), using our Informix driver/interface, written in ESQL/C.

    The code is quite complex and I would need some time to write a simplified ESQL/C sample.

    However, I have tested different binding combinations (as we obviously support different SQL types like CHAR/VARCHAR/LVARCHAR and have different Genero BDL variable types CHAR/VARCHAR/STRING):

    I confirm that when fetching an LVARCHAR into a C buffer (with fixed size memory allocated) and binding with sqlvar.sqltype = CVCHARTYPE or CCHARTYPE, the conversion is done as expected. This occurs when the target Genero/BDL variable is a CHAR(size) or VARCHAR(size) with a known size (memory is already allocated)

    However, we have a proprietary STRING type in our language, with no theoretical max size. When fetching SQL CHAR or VARCHAR, we can get the max size from the descriptors (sqlvar.sqllen) and we can allocate the target buffer, but when the source SQL type is LVARCHAR, we use ifx_var_getdata() to get the string and then do a copy...

    Summary:

    If you confirm that binding with CLVCHARPTRTYPE does not do any charset conversion and that this is a feature rather than a bug, we would need another type id that does the conversion. However, this looks more like a bug.

    Would a binding with CVCHARTYPE do the job? Assuming that there is no size limitation issue, since the SQL type VARCHAR() is limited to 255 bytes?

    ------------------------------
    Sebastien FLAESCH
    ------------------------------



  • 7.  RE: ESQL/C: Fetching LVARCHAR with ifx_var_getdata() when DB and client locale do not match

    Posted Wed February 10, 2021 10:10 AM
    Looking at the ESQL/C doc, I can read that when using an "lvarchar pointer host variable", it handles ASCII chars:

    https://informix.hcldoc.com/14.10/help/index.jsp?topic=%2Fcom.ibm.esqlc.doc%2Fids_esqlc_0007.htm

    "..... it supports ASCII data as opposed to binary data."

    So it that is an indicator that no charset conversion occurs?

    ------------------------------
    Sebastien FLAESCH
    ------------------------------



  • 8.  RE: ESQL/C: Fetching LVARCHAR with ifx_var_getdata() when DB and client locale do not match

    Posted Wed February 10, 2021 11:08 AM
    Hi Sebastien,

    Looking at the code LVARCHAR with "pointer variable" is treated similar to var binary (which doesn't go through the code set conversion). So you are right, it doesn't go through the conversion though document doesn't state specifically. You may want to open feature request if this is what you need for your applications. Copy pasting the part of documentation about lvarchar and var binary...

    https://informix.hcldoc.com/14.10/help/index.jsp?topic=%2Fcom.ibm.esqlc.doc%2Fids_esqlc_0007.htm

    "..The lvarchar pointer type is not equivalent to a C-language char pointer. Informix® ESQL/C maintains its own internal representation for the lvarchar pointer type. This representation is identical to the representation of a var binary host variable, except that it supports ASCII data as opposed to binary data..."

    Thanks
    -Shesh

    ------------------------------
    Sheshnarayan Agrawal
    ------------------------------



  • 9.  RE: ESQL/C: Fetching LVARCHAR with ifx_var_getdata() when DB and client locale do not match

    Posted Wed February 10, 2021 11:26 AM

    Shesh,

    This clarifies the behavior! Thanks a lot for this.

    Could you help me? Where can I file a request like this?
    Just like a regular IBM support case?
    I am a bit lost with IBM/HCL and this forum is my main entry point nowadays.



    ------------------------------
    Sebastien FLAESCH
    ------------------------------



  • 10.  RE: ESQL/C: Fetching LVARCHAR with ifx_var_getdata() when DB and client locale do not match

    Posted Wed February 10, 2021 11:23 PM
    Hi Sebastien,

    Please use your IBM support contact number to open the feature/defect.

    Thanks
    -Shesh


    ------------------------------
    Sheshnarayan Agrawal
    ------------------------------



  • 11.  RE: ESQL/C: Fetching LVARCHAR with ifx_var_getdata() when DB and client locale do not match

    Posted Thu February 11, 2021 10:33 AM
    I am asking where.
    https://www.ibm.com/developerworks/rfe/
    ?

    ------------------------------
    Sebastien FLAESCH
    ------------------------------



  • 12.  RE: ESQL/C: Fetching LVARCHAR with ifx_var_getdata() when DB and client locale do not match

    IBM Champion
    Posted Thu February 11, 2021 04:35 PM
    https://ibm-data-and-ai.ideas.aha.io/?project=INFX ->  Add a new idea

    ------------------------------
    Andreas Legner
    ------------------------------



  • 13.  RE: ESQL/C: Fetching LVARCHAR with ifx_var_getdata() when DB and client locale do not match

    Posted Fri February 12, 2021 05:04 AM
    That was helpful Andreas, thats a lot.
    https://ibm-data-and-ai.ideas.aha.io/ideas/INFX-I-400
    Seb

    ------------------------------
    Sebastien FLAESCH
    ------------------------------