Optim - Test Data Management and Archiving

Expand all | Collapse all

working LUA on linux enviroment

  • 1.  working LUA on linux enviroment

    Posted Mon November 30, 2020 09:50 AM
    Hello!
    I wrote a simple LUA code to mask phone numbers. It works well when I run it locally. My local environment is windows 10, Optim 11.3.0.8 64-bit. Here's my LUA program:

    function cm_transform()
    local oldvalue=optim.source.getcolumnvalue()
    local newvalue=''

    if string.sub(oldvalue,1,3) == '380' then
     newvalue = optim.mask(oldvalue,'PRO=AFF, MTD=REP, COPY=(1,5), FLDDEF1=(NAME=NUM_PHONE, DT= WVARCHAR_SZ)')
     else
     newvalue = optim.mask(oldvalue,'PRO=AFF, MTD=REP, COPY=(1,2), FLDDEF1=(NAME=NUM_PHONE, DT= WVARCHAR_SZ)')
     end
     print("Old value " .. oldvalue .. " Newvalue " .. newvalue)
     optim.target.setcolumnvalue(newvalue)
    end


    When I run a job on a server, an error message is formed:
    User DLL or Proc requested termination while processing ORAOPTIMDBA. OPTIMDATA1. CLI_PHONE Column: NUM_PHONE: Error in Column Map procedure or expression: while executing cm_transform():

    In the log of the server found an error message:
    ENVW32IF IcuW2MB (9393) EXCP 11/26/2020 13:16:13 000037 000074ED 000016C
    Could not convert from Unicode (as UChar) to Code Page 1251, reason 10 = U_INVALID_CHAR_FOUND, ICU=ibm-1251_P100-1995. See the attached file for more information.
    My OPTIM server works under OS Redhat 7.8, OPTIM version 11.3.0.8 32-bit, Oracle client 11.2.0.1 32-bit, Oracle base 18.3.0.0 64 bit. Optim db alias configured  as single by format(ASCII/EBCDIC). Full server login
    Can someone get into such a mistake and can help the council how to fix it?





    ------------------------------
    Best regards,
    Valentin Sukhanyuk
    ------------------------------

    Attachment(s)

    zip
    PR0SVER_LOG.zip   7 KB 1 version


  • 2.  RE: working LUA on linux enviroment

    Posted Tue December 01, 2020 02:57 AM
    Hi Valentin,

    It seems that you have a character-conversion problem here because you run client and server with different codepages.
    Every locale (or its translation) that the server is required to handle must reside on the server machine. In other words, the server must have access to the locale of the delegating workstation. A utility, PR0LOCL.exe, is provided to tell you the locales that are installed on a machine and the locales with which it is compatible.

    Regards,
    Fritz

    ------------------------------
    FRIEDRICH PFNEISL
    Optim SME
    Austria
    ------------------------------



  • 3.  RE: working LUA on linux enviroment

    Posted Mon December 07, 2020 11:34 PM
    Hello Valentin,

    Could you please try this code below, I have added the CPTYPE and CODEPAGE to your LUA code :-

    function cm_transform()
    local oldvalue=optim.source.getcolumnvalue()
    local newvalue=''

    if string.sub(oldvalue,1,3) == '380' then
     newvalue = optim.mask(oldvalue,'PRO=AFF, MTD=REP, COPY=(1,5), FLDDEF1=(NAME=NUM_PHONE, DT= WVARCHAR_SZ, CPTYPE=ORACLE, CODEPAGE=437)')
     else
     newvalue = optim.mask(oldvalue,'PRO=AFF, MTD=REP, COPY=(1,2), FLDDEF1=(NAME=NUM_PHONE, DT= WVARCHAR_SZ. CPTYPE=ORACLE, CODEPAGE=437)')
     end
     print("Old value " .. oldvalue .. " Newvalue " .. newvalue)
     optim.target.setcolumnvalue(newvalue)
    end

    ------------------------------
    Suraj Shingri
    ------------------------------



  • 4.  RE: working LUA on linux enviroment

    Posted Tue December 01, 2020 08:02 AM
    Hi Fritz!

    First, I'll be good for your support in solving the problem.

    I created a test case in which the locale of the workstation and server are equal. But I also received a conversion error message in trace logs. In the same test case, I comment on all the calls of types  source.getxxxx and source.column.getxxxx and get a successful result. I've added screenshots from trace logs.

    What other options are possible ?

    Respectfully,
    Valentin

    ------------------------------
    Valentin Sukhanyuk
    ------------------------------



  • 5.  RE: working LUA on linux enviroment

    Posted Tue December 01, 2020 08:27 AM

    Hi Valentin,

    Unfortunately your screen-shots don't work.
    However if the procedure works on your local server and does not work on the Linux server there "must" be a difference between those two and the error-message clearly points to a conversion-error of code-pages.

    I suggest you open a support-case.

    Regards,
    Fritz



    ------------------------------
    FRIEDRICH PFNEISL
    Optim SME
    Austria
    ------------------------------



  • 6.  RE: working LUA on linux enviroment

    Posted Tue December 08, 2020 05:27 AM
    Edited by Valentin Sukhanyuk Tue December 08, 2020 05:27 AM
      |   view attached
    Hi Suraj!

    I installed the suggested code and perform the conversion service.
    The error is still the same, "Could not convert from Unicode".
    I will show a part of the service execution trace file, I will attach the complete trace file to this message.

    EADBD5D4 0001A0 16000000 3A000000 20000000 62000000 ....:... ...b...
    EADBD5E4 0001B0 61000000 64000000 20000000 61000000 a...d... ...a...
    EADBD5F4 0001C0 72000000 67000000 75000000 6D000000 r...g...u...m...
    EADBD604 0001D0 65000000 6E000000 74000000 20000000 e...n...t... ...
    EADBD614 0001E0 23000000 31000000 20000000 74000000 #...1... ...t...
    EADBD624 0001F0 6F000000 20000000 27000000 73000000 o... ...'...s...
    EADBD634 000200 75000000 62000000 27000000 20000000 u...b...'... ...
    EADBD644 000210 28000000 73000000 74000000 72000000 (...s...t...r...
    EADBD654 000220 69000000 6E000000 67000000 20000000 i...n...g... ...
    EADBD664 000230 65000000 78000000 70000000 65000000 e...x...p...e...
    EADBD674 000240 63000000 74000000 65000000 64000000 c...t...e...d...
    EADBD684 000250 2C000000 20000000 67000000 6F000000 ,... ...g...o...
    EADBD694 000260 74000000 20000000 66000000 75000000 t... ...f...u...
    EADBD6A4 000270 6E000000 63000000 74000000 69000000 n...c...t...i...
    EADBD6B4 000280 6F000000 6E000000 29000000 00000000 o...n...).......
    ENVW32IF IcuW2MB (9406) EXCP 12/08/2020 11:37:02 000036 00018625 0000016E
    Bad char length = 1 =
    F3F03AB0 000000 2000F6F3 .ö󠠠
    COLCTRNX COLCMTransformExec (1149) EXCP 12/08/2020 11:37:02 000037 00018625 0000016E
    Exit returns PST_CMW_EXIT_ABORT_PROCESS, message "Column: NUM_PHONE: Error in Column Map procedure o
    r expression: while executing cm_transform():  "
    XFMCROW XFMIntGetRow (2518) EXCP 12/08/2020 11:37:02 000038 00018625 0000016E
    ErrMsg :
    Line : 2518 Function: XFMIntGetRow Module: ../SrcRes/XFMCROW.C
    RetCode: XFMERR_EXIT_TERM(11317) Get:User Exit Abort
    AuxCode: COLERR_EXIT_ABORT(08421) Exit Aborts the Process
    OpsCode: (01113) No mapping for the Unicode character exists in the target multi
    Token2: COL_CMTransformExec


    I think the optim.source.getcolumnvalue () function is not working correctly. The string it returns contains Unicode characters that are not in the code page 1251. I looked at the codes of the characters in the table by reference https://icu4c-demos.unicode.org/icu-bin/convexp?conv=ibm-1251_P100-1995&s=ALL.  

    A runtime error is called in the function string.sub (oldvalue, 1,3) because the string in the oldvalue variable is not correct.

    Thank you for your help.

    Best regards, Valentine.

    p.s.  The original LUA code was executed locally in the Optim version of Windows 32-bit, it works without error.


    ------------------------------
    Valentin Sukhanyuk
    ------------------------------

    Attachment(s)

    zip
    PR0SVER.zip   6 KB 1 version