Db2 (On Premises and Cloud)

Expand all | Collapse all

Table Function SYSPROC.SNAP_GET_UTIL_PROGRESS Fails With Termination Error

  • 1.  Table Function SYSPROC.SNAP_GET_UTIL_PROGRESS Fails With Termination Error

    Posted 10 days ago
    Hello DMC-Team,

    ​for ​DMC (version 3.1.1) utility monitoring function, the monitored database frequently logs info and warning messages in the db2diag.log of the type shown below. Is that something that asks for further attention or can it be ignored?

    The monitored database is a DB2 LUW V10.5 DPF installation with  8 partitions.
    >db2level
    DB21085I  This instance or install (instance name, where applicable:"cardm1in") uses "64" bits and DB2 code release "SQL10059" with levelidentifier "060A010E".Informational tokens are "DB2 v10.5.0.9", "special_37311", "IP24025_37311", andFix Pack "9".Product is installed at "/opt/IBM/db2/V10.5.AESE.FP9".



    ### Example  from db2diag.log


    2020-03-17-00.09.01.526248+060 I2492A7938           LEVEL: Info
    PID     : 35323984             TID : 39566          PROC : db2sysc 4
    INSTANCE: cardm1in             NODE : 004           DB   : CARD
    APPHDL  : 0-60540              APPID: xx.xx.xxx.xxx.37496.200316230850
    AUTHID  : DB2MON1              HOSTNAME: xxxxvmcrd010.xxx.xxxxl.com
    EDUID   : 39566                EDUNAME: db2agntp (CARD) 4
    FUNCTION: DB2 UDB, routine_infrastructure, sqlerInvokeFencedRoutine, probe:88
    DATA #1 : String, 37 bytes
    Routine failed with termination error
    DATA #2 : Hexdump, 1456 bytes
    0x0A0003000032AE60 : 5546 4F42 0000 0000 0000 B998 0000 0000    UFOB............
    0x0A0003000032AE70 : 0000 0000 0000 0000 0780 0000 03AF 47B8    ..............G.
    0x0A0003000032AE80 : 0000 0000 0000 0000 0A00 0300 0032 2378    .............2#x
    0x0A0003000032AE90 : 0000 0000 0000 0000 0780 0000 018B A5B8    ................
    0x0A0003000032AEA0 : 0000 0000 0000 0000 0A00 0300 0032 B7D8    .............2..
    0x0A0003000032AEB0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032AEC0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032AED0 : 0000 0000 0000 0000 0A00 0300 0032 B850    .............2.P
    0x0A0003000032AEE0 : 0A00 0300 0032 B8D0 0000 0000 0000 0000    .....2..........
    0x0A0003000032AEF0 : 0000 0000 0000 0000 0A00 0300 0032 B410    .............2..
    0x0A0003000032AF00 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032AF10 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032AF20 : 0000 0001 0000 0000 FFFF FFFF 0000 0052    ...............R
    0x0A0003000032AF30 : 0000 0000 0000 0000 0000 0AF0 0000 0000    ................
    0x0A0003000032AF40 : 0A00 0300 0032 B800 0000 0000 0000 0000    .....2..........
    0x0A0003000032AF50 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032AF60 : 0000 0000 0000 0000 9D04 5871 0099 5C8C    ..........Xq..\.
    0x0A0003000032AF70 : 0000 0000 0000 0000 0001 01B7 0000 0000    ................
    0x0A0003000032AF80 : 4600 0000 0000 0000 0000 0000 0000 0000    F...............
    0x0A0003000032AF90 : 0000 0000 0000 0000 0000 0000 0000 0C00    ................
    0x0A0003000032AFA0 : 0000 0002 FF00 FFFF 0000 0001 0148 1067    .............H.g
    0x0A0003000032AFB0 : 0000 0000 0000 0000 0000 0000 0001 01B7    ................
    0x0A0003000032AFC0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032AFD0 : 0000 0002 0000 0003 0000 0001 0000 0001    ................
    0x0A0003000032AFE0 : 0000 0005 001E 00BC 0000 0000 001E 0000    ................
    0x0A0003000032AFF0 : 0007 0019 0000 000B 0000 000B 0000 0000    ................
    0x0A0003000032B000 : 0000 000B 0000 0000 0000 0008 0000 0000    ................
    0x0A0003000032B010 : 0064 0000 0000 0023 0000 0000 0000 0000    .d.....#........
    0x0A0003000032B020 : 0000 04B8 0000 002A 000A 001A 5359 5350    .......*....SYSP
    0x0A0003000032B030 : 524F 432E 534E 4150 5F47 4554 5F55 5449    ROC.SNAP_GET_UTI
    0x0A0003000032B040 : 4C5F 5052 4F47 5245 5353 0000 0000 0000    L_PROGRESS......
    0x0A0003000032B050 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B060 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B070 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B080 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B090 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B0A0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B0B0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B0C0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B0D0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B0E0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B0F0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B100 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B110 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B120 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B130 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B140 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B150 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B160 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B170 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B180 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B190 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B1A0 : 0000 0000 0000 0000 0000 0000 0000 0053    ...............S
    0x0A0003000032B1B0 : 4E41 505F 4745 545F 5554 494C 5F50 524F    NAP_GET_UTIL_PRO
    0x0A0003000032B1C0 : 4752 4553 535F 4150 0000 0000 0000 0000    GRESS_AP........
    0x0A0003000032B1D0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B1E0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B1F0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B200 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B210 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B220 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B230 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B240 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B250 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B260 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B270 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B280 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B290 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B2A0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B2B0 : 5359 5349 424D 2020 0000 0000 0000 0000    SYSIBM  ........
    0x0A0003000032B2C0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B2D0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B2E0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B2F0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B300 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B310 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B320 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B330 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B340 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B350 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B360 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B370 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B380 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B390 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B3A0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B3B0 : 0001 0802 0001 0000 0000 0000 0000 0000    ................
    0x0A0003000032B3C0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B3D0 : 0000 0000 0000 0008 0A00 0000 23FF FED8    ............#...
    0x0A0003000032B3E0 : 0000 0001 15D5 9920 0000 9A8E 0000 0000    ....... ........
    0x0A0003000032B3F0 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x0A0003000032B400 : 0000 0000 0000 0000 0000 0000 0000 0000    ................

    2020-03-17-00.09.01.527548+060 E10431A3772          LEVEL: Warning
    PID     : 35323984             TID : 39566          PROC : db2sysc 4
    INSTANCE: cardm1in             NODE : 004           DB   : CARD
    APPHDL  : 0-60540              APPID: xx.xx.xxx.xxx.37496.200316230850
    AUTHID  : DB2MON1              HOSTNAME: xxxxvmcrd010.xxx.xxxxl.com
    EDUID   : 39566                EDUNAME: db2agntp (CARD) 4
    FUNCTION: DB2 UDB, routine_infrastructure, sqlerReturnFmpToPool, probe:900
    DATA #1 : String, 50 bytes
    Marking fmp as unstable, fmp is forced or aborted:
    DATA #2 : Hex integer, 1 bytes
    0x02
    DATA #3 : String, 20 bytes
    Fmp entry use count:
    DATA #4 : unsigned integer, 4 bytes
    1
    DATA #5 : String, 8 bytes
    Fmp TID:
    DATA #6 : Hexdump, 4 bytes
    0x07800000018BA708 : 0000 0000                                  ....
    DATA #7 : String, 8 bytes
    Fmp row:
    DATA #8 : sqlerFmpRow, PD_SQLER_TYPE_FMP_ROW, 544 bytes
     fmpPid: 25690154
     fmpPoolList Ptr: 0x0000000000000000 fmpForcedList Ptr: 0x0000000000000000
     nextFmpCB Ptr: 0x07800000018bb760 prevFmpCB Ptr: 0x0000000000000000
     fmpIPCList Ptr: 0x07800000018ba7e0
     stateFlags: 0x00000010 numFmp32Attaches: 0
     numActiveThreads: 0 numPoolThreads: 0
     fmpCodePage: 0 fmpRowUseCount: 1
     active: 0x01  rowLoaderValidate: 0x00
     ipcLatch:
    0x07800000018BA5A4 : 0000 0000 003F 0000                        .....?..
     rowLatch:
    0x07800000018BA5AC : 0000 0000 01D2 0000                        ........
     fmpAgentList:
    0x07800000018BA5B8 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA5C8 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA5D8 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA5E8 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA5F8 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA608 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA618 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA628 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA638 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA648 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA658 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA668 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA678 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA688 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA698 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA6A8 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA6B8 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA6C8 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA6D8 : 0000 0001 13E7 9020 0780 0000 1012 0094    ....... ........
    0x07800000018BA6E8 : 0780 0000 1012 F094 0000 F000 0000 0000    ................
    0x07800000018BA6F8 : 0000 0000 0000 02DA 0000 02DA 0000 0000    ................
    0x07800000018BA708 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA718 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA728 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA738 : 0000 0000 0000 0000 0000 0000 0000 0000    ................
    0x07800000018BA748 : 0780 0000 03AE 6900 0780 0000 018B A5A0    ......i.........
    0x07800000018BA758 : 0780 0000 018B A260 0780 0000 018B A340    .......`.......@
    0x07800000018BA768 : 0000 0001 0000 0001 0000 0000 0000 0000    ................
    0x07800000018BA778 : 0000 0000 0000 0000                        ........

    2020-03-17-00.09.01.528882+060 E14204A595           LEVEL: Warning
    PID     : 35323984             TID : 39566          PROC : db2sysc 4
    INSTANCE: cardm1in             NODE : 004           DB   : CARD
    APPHDL  : 0-60540              APPID: xx.xx.xxx.xxx.37496.200316230850
    AUTHID  : DB2MON1              HOSTNAME: xxxxvmcrd010.xxx.xxxxl.com
    EDUID   : 39566                EDUNAME: db2agntp (CARD) 4
    FUNCTION: DB2 UDB, routine_infrastructure, sqlerReturnFmpToPool, probe:2000
    DATA #1 : String, 55 bytes
    Sending a signal to clean up NOT THREADED FMP process:
    DATA #2 : Process ID, 4 bytes
    25690154

    2020-03-17-00.09.01.529643+060 I14800A1976          LEVEL: Event
    PID     : 35323984             TID : 39566          PROC : db2sysc 4
    INSTANCE: cardm1in             NODE : 004           DB   : CARD
    APPHDL  : 0-60540              APPID: xx.xx.xxx.xxx.37496.200316230850
    AUTHID  : DB2MON1              HOSTNAME: xxxxvmcrd010.xxx.xxxxl.com
    EDUID   : 39566                EDUNAME: db2agntp (CARD) 4
    FUNCTION: DB2 UDB, oper system services, sqlossig, probe:10
    MESSAGE : Sending SIGKILL to the following process id
    DATA #1 : signed integer, 4 bytes
    25690154
    CALLSTCK: (Static functions may not be resolved correctly, as they are resolved to the nearest symbol)
      [0] 0x090000000CF9E4F0 sqlossig + 0xA0
      [1] 0x090000000CE906C4 sqlerReturnFmpToPool__FcT1P14sqlerFmpHandleP8sqeAgent + 0x1010
      [2] 0x090000000D59933C sqlerInvokeFencedRoutine__FP13sqlerFmpParms + 0x2900
      [3] 0x090000000D5561C4 sqlerInvokeFencedRoutine__FP13sqlerFmpParms + 0x5EC
      [4] 0x090000000D50D0EC sqlriInvokeInvoker__FP10sqlri_ufobb + 0x4198
      [5] 0x090000000B91C9F0 sqlriutf__FP8sqlrr_cb + 0x298
      [6] 0x090000000EA8DB90 sqlriopn__FP8sqlrr_cbP9sqlri_taoPi@OL@23731 + 0x3CC
      [7] 0xFFFFFFFFFFFFFFFC ?unknown + 0xFFFFFFFF
      [8] 0x090000000D7E77F4 sqlriopn__FP8sqlrr_cbP9sqlri_taoPi + 0x1CC
      [9] 0x090000000D8FB9BC sqlriSectInvoke__FP8sqlrr_cbP12sqlri_opparm + 0x578
      [10] 0x090000000CA7CCE8 sqlrr_subagent_router__FP8sqeAgentP12SQLE_DB2RA_T + 0x8C84
      [11] 0x090000000CA7CCE8 sqlrr_subagent_router__FP8sqeAgentP12SQLE_DB2RA_T + 0x8C84
      [12] 0x090000000CAB0F2C sqlrr_subagent_router__FP8sqeAgentP12SQLE_DB2RA_T + 0x107C
      [13] 0x090000000F62D120 @82@sqleSubRequestRouter__FP8sqeAgentPUiPUl + 0xA9C
      [14] 0x090000000E4EBE54 RunEDU__8sqeAgentFv + 0x36CBC
      [15] 0x090000000E500634 RunEDU__8sqeAgentFv + 0x11C
      [16] 0x090000000DE10E2C EDUDriver__9sqzEDUObjFv + 0x134
      [17] 0x090000000E41464C sqloEDUEntry + 0x394
      [18] 0x090000000056AE10 _pthread_body + 0xF0
      [19] 0xFFFFFFFFFFFFFFFC ?unknown + 0xFFFFFFFF

    2020-03-17-00.09.01.530669+060 I16777A1011          LEVEL: Info
    PID     : 35323984             TID : 39566          PROC : db2sysc 4
    INSTANCE: cardm1in             NODE : 004           DB   : CARD
    APPHDL  : 0-60540              APPID: xx.xx.xxx.xxx.37496.200316230850
    AUTHID  : DB2MON1              HOSTNAME: xxxxvmcrd010.xxx.xxxxl.com
    EDUID   : 39566                EDUNAME: db2agntp (CARD) 4
    FUNCTION: DB2 UDB, routine_infrastructure, sqlerInvokeFencedRoutine, probe:99
    MESSAGE : ZRC=0x804B0003=-2142568445=SQLJS_INTRPT "INTERRUPT DETECTED"
              DIA8003C The interrupt  has been received.
    DATA #1 : SQLCA, PD_DB2_TYPE_SQLCA, 136 bytes
     sqlcaid : SQLCA     sqlcabc: 136   sqlcode: 0   sqlerrml: 0
     sqlerrmc:
     sqlerrp : SQL10059
     sqlerrd : (1) 0x00000000      (2) 0x00000000      (3) 0x00000000
               (4) 0x00000000      (5) 0x00000000      (6) 0x00000004
     sqlwarn : (1)      (2)      (3)      (4)        (5)       (6)   
               (7)      (8)      (9)      (10)        (11)    
     sqlstate:     

    2020-03-17-00.09.01.532645+060 E17789A458           LEVEL: Error
    PID     : 31719594             TID : 258            PROC : db2wdog 4 [cardm1in]
    INSTANCE: cardm1in             NODE : 004
    HOSTNAME: xxxxvmcrd010.xxx.xxxxl.com
    EDUID   : 258                  EDUNAME: db2wdog 4 [cardm1in]
    FUNCTION: DB2 UDB, base sys utilities, sqleChildCrashHandler, probe:7967
    DATA #1 : <preformatted>
    Beginning cleanup of Process id: 25690154, OSS term code: 0x102, signal: 9

    2020-03-17-00.09.01.533337+060 E18248A412           LEVEL: Error
    PID     : 35323984             TID : 258            PROC : db2sysc 4
    INSTANCE: cardm1in             NODE : 004
    HOSTNAME: xxxxvmcrd010.xxx.xxxxl.com
    EDUID   : 258                  EDUNAME: db2sysc 4
    FUNCTION: DB2 UDB, base sys utilities, sqleCleanupFmp, probe:776
    DATA #1 : <preformatted>
    A 64-bit fenced mode process crashed. Process id: 25690154

    ------------------------------
    Matthias Gärtner-Schür
    DBA
    ------------------------------


  • 2.  RE: Table Function SYSPROC.SNAP_GET_UTIL_PROGRESS Fails With Termination Error

    Posted Wed March 18, 2020 11:15 PM

    Hi Matthias,

    We received your question. Our developer is investigating it and will update with you ASAP. Thanks.



    ------------------------------
    Li Hui Yan (Autumn Yan)
    From DMC Team
    ------------------------------



  • 3.  RE: Table Function SYSPROC.SNAP_GET_UTIL_PROGRESS Fails With Termination Error

    Posted 10 days ago
    Edited by Jessica Long 10 days ago
    Hello Matthias,

    Thank you for contacting us for this problem. We need logs of the console to investigate how the error was triggered. You can follow this doc of step 2 to collect logs https://www.ibm.com/support/knowledgecenter/SS5Q8A_3.1.x/com.ibm.datatools.dsweb.ots.troubleshooting.doc/topics/ts_exchangingdata.html
    I'll send you my email in a private note. Could you please email the logs to me? Thanks in advance.

    Besides, did you notice any abnormal functions for utility monitor of the console?

    Thanks,
    Equal

    ------------------------------
    KE CHEN
    ------------------------------



  • 4.  RE: Table Function SYSPROC.SNAP_GET_UTIL_PROGRESS Fails With Termination Error

    Posted 10 days ago
    Edited by Jessica Long 10 days ago
    Hello Matthias,

    I can not view my private reply to you. Please let me konw if you can not see my message and email.

    Thanks,
    Equal

    ------------------------------
    KE CHEN
    ------------------------------