Db2

Db2

Connect with Db2, Informix, Netezza, open source, and other data experts to gain value from your data, share insights, and solve problems.

 View Only
Expand all | Collapse all

Locking and Utility Monitoring requiring WLM ? why ?

  • 1.  Locking and Utility Monitoring requiring WLM ? why ?

    Posted Wed November 27, 2019 05:46 PM
    Edited by System Admin Fri January 20, 2023 04:23 PM
    I have enabled in my monitor profile, both Locking and Utility event monitoring. 

    But,  when I try to "see" the history events for these two categories,  I receive the warning saying to run the requirement script on target db. 
    Review and run theto make sure that your database satisfies all the prerequisites for event monitoring.
    --NOTICE: please check whether there is 32kb pagesize temporary tablespace firstly, if not please create a new one.
    
    --create a bufferpool with page size 32kb
    CREATE BUFFERPOOL CONSOLEPOOL ALL DBPARTITIONNUMS SIZE 1000 AUTOMATIC PAGESIZE 32768;
    --create a partition spannning all partitions
    CREATE DATABASE PARTITION GROUP CONSOLEGROUP ON ALL DBPARTITIONNUMS;
    --create the 32kb tablespace with max size 2G specified
    CREATE TABLESPACE TS4MONITOR IN CONSOLEGROUP PAGESIZE 32768 MANAGED BY AUTOMATIC STORAGE AUTORESIZE YES INITIALSIZE 100M MAXSIZE 2G BUFFERPOOL CONSOLEPOOL;
    --check whether there is 32kb pagesize temporary table space
    SELECT COUNT(1) FROM SYSCAT.TABLESPACES WHERE DATATYPE = 'T' AND PAGESIZE = 32768;
    --Don't create TEMPSPACE2 if there is already a 32kb pagesize temporary tablespace. If no 32kb pagesize temporary tablespace, create a required one.
    CREATE TEMPORARY TABLESPACE TEMPSPACE2 PAGESIZE 32K MANAGED BY AUTOMATIC STORAGE EXTENTSIZE 4 BUFFERPOOL CONSOLEPOOL;
    
    --Define a workload to filter the monitored console executed statements. Because the COLLECT ACTIVITY DATA option is not specified, the statement history will not collect the statements of the workload.
    CREATE WORKLOAD DSM_WORKLOAD APPLNAME ('DSMAu*','DSMRt*','DS_ConnMgt*','DSSNAP*','DSMOQT','UC_*');
    --Capture the statements for the default two workloads on the coordinator node. If you want to collect the activity data for all nodes of a multi-partition database, change the COORDINATOR option to ALL.
    ALTER WORKLOAD SYSDEFAULTUSERWORKLOAD COLLECT ACTIVITY DATA ON COORDINATOR WITH DETAILS;
    ALTER WORKLOAD SYSDEFAULTADMWORKLOAD COLLECT ACTIVITY DATA ON COORDINATOR WITH DETAILS;
    --Capture the aggregate responsiveness workload information
    ALTER WORKLOAD SYSDEFAULTUSERWORKLOAD COLLECT AGGREGATE ACTIVITY DATA BASE;
    ALTER WORKLOAD SYSDEFAULTADMWORKLOAD COLLECT AGGREGATE ACTIVITY DATA BASE;​


    I have done it partially only. Until the tablespaces , but I did not changed WLM collect activity stuff, as my target DBs are workgroup or express-C databases,  and WLM is not available. 

    as far as I understand,  enabled activity collect on WLM level,  will be used just for responsive metriecs and statements on DMC.   This is not a requirement for locking and utility event monitors to run and capture the desired information.  

    Why DMC is forcing WLM collect to be enabled for this ? 

    On DSM,  this used to work without WLM changes.  DSM creates locking and event monitors on target database and that's it.   Requiring WLM changes will restrict a lot DMC usage,  as only "Advanced" editions will be available for this.  :-( 


    Regards


    ------------------------------
    Samuel Pizarro
    ------------------------------
    #Db2


  • 2.  RE: Locking and Utility Monitoring requiring WLM ? why ?

    Posted Mon December 02, 2019 09:05 AM
    @Samuel Pizarro​ You are correct. Lock event monitor and utility event monitor does not need workload to be created.  And it should not affect lock and utility event monitor if workload was created. One quick question is does your lock event monitor and utility event monitor work well?   If it doesn't work it might be some issues other than workload create.

    In DMC,  to make the 4 event monitor setup process consistent and simple, we combine all prerequisite steps of 4 event monitors in one script and deliver the script in pages which need event monitors.  Any feedback  and suggestion of the event monitor setup  is highly appreciated. It can help us continuously improve the user experience. Thanks.

    ------------------------------
    HAO WU
    ------------------------------



  • 3.  RE: Locking and Utility Monitoring requiring WLM ? why ?

    Posted Mon December 02, 2019 02:13 PM
    Hi  @HAO WU

    One quick question is does your lock event monitor and utility event monitor work well?   If it doesn't work it might be some issues other than workload create.
    No,  not working..   That's why I am raising the question here..  I have enabled the locking and utility event monitor in the profile. 
    But none of them got created in the target db. 

    Here are the current event monitors that exists in the target db..  both of them existed already before DMC implementation, and both are disabled. 

    EVMONNAME                      STATE       VERSIONNUMBER TARGET_TYPE
    ------------------------------ ----------- ------------- -----------
    LOCKEVMON                                0      10050500 U
    TRACE_EVMON                              0      10050500 T​

    Here is the screen shoot of my monitor profile.

    I understand, that you have consolidated the 4 event-monitors pre-reqs in a single script..  That's fine.
    (Maybe the statistics one should be separate script, which is the only one that really demmands changes in WLM statistics collection to be done.. So,  if one particular user does not checks to use this,  but wants only to enables the other 3,  it will be prompted to make WLM changes, where this is really not necessary., but this is another subject... )  
     Problem is that,  when I try to see lock and utility history in DMC,  I am still prompted with the warning message that the pre-reqs are not met and need to review the script. 

    But I have done everything  that I believe should be necessary,  I have created the special bufferpool, created the monitor tablespace, and also created the systemp space..   I just did not execute the statements that enables WLM statistics collection. 

    Follow the details about the bps and tbspaces pre-reqs created on target db: 

    BUFFERPOOLID BPNAME                NPAGES      PAGESIZE
    ------------ --------------------  ----------- -----------
               1 IBMDEFAULTBP                   -2        4096
               2 BPNB04K                        -2        4096
               3 BPNB08K                        -2        8192
               4 BPEVMON32K                   1000       32768
               5 BPNB32K                        -2       32768
               6 BPBK2404KTINT                  -2        4096
               7 CONSOLEPOOL                    -2       32768
    
    
    DATABASE PARTITION GROUP  PMAP_ID DATABASE PARTITION NUMBER    IN_USE
    ------------------------- ------- ---------------------------- ------
    CONSOLEGROUP                    3                            0 Y
    IBMCATGROUP                     0                            0 Y
    IBMDEFAULTGROUP                 1                            0 Y
    
    TBSPACE              TBSPACEID   PAGESIZE    TBSPACETYPE DATATYPE DBPGNAME             BUFFERPOOLID
    -------------------- ----------- ----------- ----------- -------- -------------------- ------------
    TSST32K0001                  185       32768 S           T        IBMTEMPGROUP                    4
    TS4MONITOR                   286       32768 D           L        CONSOLEGROUP                    7​

    What else is missing ? 

    Regards




    ​​

    ------------------------------
    Samuel Pizarro
    ------------------------------



  • 4.  RE: Locking and Utility Monitoring requiring WLM ? why ?

    Posted Tue December 03, 2019 03:59 AM
    Thanks @Samuel Pizarro for the reply.    We will take a look whether this is a defect which lock event monitor and utility event monitor should work if it doesn't.    We will open a git issue and update status here.    We will also track your suggestion to put statistic pre-req script separately in the git issue and will evaluate.   Thanks for the good suggestion. ​​​​​​

    ------------------------------
    HAO WU
    ------------------------------



  • 5.  RE: Locking and Utility Monitoring requiring WLM ? why ?

    Posted Tue December 03, 2019 04:22 AM
    Edited by System Admin Fri January 20, 2023 04:28 PM

    Hi Samuel,
    We tracked your mentioned problem in our Zenhub.

    Besides, we welcome you open defects or requests in our Zenhub whenever what problem you met in using DMC. Your comments and suggestion is greatly helpful for our product improvement. Thanks a lot.



    ------------------------------
    LI HUI YAN
    ------------------------------



  • 6.  RE: Locking and Utility Monitoring requiring WLM ? why ?

    Posted Tue December 03, 2019 03:06 PM
    Hi,
    I agree with Samuel that WLM should not be neccessary for utility monitoring (ok maybe for historical data collection).
    But I also want to see currently running utilities as DSM displays under Applications->Utilities.
    To have only finished utilities is not really useful. And furthermore the historical utility collected data is unreadable .. Too much data/entries for 1 utility run...
    Regards, 
    Gerhard

    ------------------------------
    Gerhard Paulus
    ------------------------------



  • 7.  RE: Locking and Utility Monitoring requiring WLM ? why ?

    Posted Mon December 16, 2019 05:03 AM
    Edited by System Admin Fri January 20, 2023 04:36 PM
    @Gerhard Paulus​  Thanks for the feedback. It is a known gap that we will cover 1H2020 releases.  DMC only get data from event monitor for now and we will add data extraction from table function then user can see more up-to-date running utilities.

    ------------------------------
    HAO WU
    ------------------------------



  • 8.  RE: Locking and Utility Monitoring requiring WLM ? why ?

    Posted Wed December 11, 2019 09:52 AM
    Hi @Samuel Pizarro​ , WLM configurations are not required for utility / lock event monitor. After you've done the scripts, created BUFFERPOOL, DATABASE PARTITION GROUP, TBSPACE done, utility / lock event monitor could be used. The problem why they are not created should be caused by other reasons.
    • One known reason is DB connection timeout. When console is trying to get db connection to validate the event monitor prerequisites, it timeout in 1 mintues, then returned validation failure to frontend. Frontend still reports the prerequisites validation failure.
    • Another known reason is the db connection's authorization issue. It's better to using "db2inst1" in connection profile to enable data collection. Some other user id may not have the required privilege to perform the operation.
    If you already user db2inst1, but still cannot using utility/ lock event monitor function after running DDL. One workaround to bypass the issue is restarting console.  After console is restarted, utility/ lock event monitor should be created there.

    We are continue working on improving the stability of event monitor. Hope those above would be helpful. Thanks for your feedback!

    ------------------------------
    XIAO MIN ZHAO
    ------------------------------



  • 9.  RE: Locking and Utility Monitoring requiring WLM ? why ?

    Posted Wed December 11, 2019 10:30 AM
    Hi @xiao mingguang

    ​​​​Thanks for your answers. 

    1.  No issues related to connectivity..   DMC is running in same host for the target db (localhost).  I have no issues related to connectivity. 

    2. Lack of authorization. 
         I followed DMC documentation for that..  (actually, none). So,  the Id being used to monitor the db,  has the same level of privileges required by DSM.  (DSM has this pretty well documented...  DMC has not).  This was working fine when running with DSM, and I am using the same user id, and same target db.  So the user should have the privileges to create the event-monitors. 
     If this is really happening, please,  let me know the log file where I can find the errors being logged to make sure this is the case. 
    We are still "guessing" here.  
     
    No,  I won't use the instance onwer..   Separete of dutties..  this is bad practice..  and imposes challenges on day-by-day operational support of the database,  when you have a monitoring tool with excessive privileges against the database.. 

    I have restarted DMC several times, and yet,  the locking and utility event monitors are not created. 
    I am using DMC and db both on windows system. 

    Please provide some detailed information on how can I determine through the logs,  what error is happening..  and how to solve them. 
    I have not found dmc logs. where are them  ? 

    Regards 


    ------------------------------
    Samuel Pizarro
    ------------------------------



  • 10.  RE: Locking and Utility Monitoring requiring WLM ? why ?

    Posted Wed December 11, 2019 11:29 PM
    Hi @Samuel Pizarro​ ,  the log is recorded in files folder /ibm-datamgmtconsole/logs/monitor.*, monitor.0 includes the latest log. And would you please share with me the API's output by insect Chrome browser's network, as following: Another workaround to bypass the problem is: Please try disable / enable event monitor options in monitor profile. These operation will trigger console to drop /create event monitors in the monitored Database immediately if all the prerequistes are meet.


    ------------------------------
    XIAO MIN ZHAO
    ------------------------------



  • 11.  RE: Locking and Utility Monitoring requiring WLM ? why ?

    Posted Sun December 15, 2019 10:52 AM

    Hi @xiao mingguang

    Now we are talking...

    Thanks for the log information...  I am using windows, and the path location are bit different.
    ​On windows the logs are located at 'Install-path\resources\bin\logs' sub dir.  

    So,  I kept eye on monitor.0 log , and deactivated the event monitor in the profile, as you suggested, and enabled it back to force their creation. 

    It was really lack of privileges... 

    Initially lack of SYSMON authority. 

    ### Error querying database.  Cause: com.ibm.db2.jcc.am.SqlException: The requested command or operation failed because the user ID does not have the authority to perform the requested command or operation.  User ID: "DSMUSR1".. SQLCODE=-1092, SQLSTATE=     , DRIVER=4.26.14
    ### The error may exist in com/ibm/aps/tools/monitor/collector/mybatis/mapper/EventMonitorLockingMapper.xml
    ### The error may involve com.ibm.aps.tools.monitor.collector.mybatis.mapper.EventMonitorLockingMapper.queryFixedSize-Inline
    ### The error occurred while setting parameters
    ### SQL: select tbsp_type,tbsp_auto_resize_enabled,tbsp_max_size  from sysibmadm.tbsp_utilization where tbsp_name = 'TS4MONITOR'
    ### Cause: com.ibm.db2.jcc.am.SqlException: The requested command or operation failed because the user ID does not have the authority to perform the requested command or operation.  User ID: "DSMUSR1".. SQLCODE=-1092, SQLSTATE=     , DRIVER=4.26.14


    This one was solved by granting SYSMON to the DMC-user-id monitoring this db. 

    Then, in the next monitor cycle..  another error, due lack of privileges: 

    monitor (197)   NOXGR   db2admin        Dec 15, 2019 1:19:46 PM [16edb6a761436633dda7294edd322398]      ERROR    Mymon MonitorCollectorRunnable::run:  Event monitor EventMonitorLocking failed to create.
    org.apache.ibatis.exceptions.PersistenceException:
    ### Error updating database.  Cause: com.ibm.db2.jcc.am.SqlSyntaxErrorException: The statement failed because the authorization ID does not have the required authorization or privilege to perform the operation.  Authorization ID: "DSMUSR1".  Operation: "CREATE EVENT MONITOR". Object: "TS4MONITOR".. SQLCODE=-551, SQLSTATE=42501, DRIVER=4.26.14
    ### The error may involve com.ibm.aps.tools.monitor.collector.mybatis.mapper.EventMonitorLockingMapper.createEvmon-Inline
    ### The error occurred while setting parameters
    ### SQL: create event monitor RTMON_EVMON_LOCKING for locking write to table   LOCK (table ibm_rtmon.lock_event    in TS4MONITOR    ),   LOCK_PARTICIPANTS (table ibm_rtmon.lock_participants    in TS4MONITOR    ),   LOCK_PARTICIPANT_ACTIVITIES (table ibm_rtmon.lock_participant_activities    in TS4MONITOR    ) manualstart
    ### Cause: com.ibm.db2.jcc.am.SqlSyntaxErrorException: The statement failed because the authorization ID does not have the required authorization or privilege to perform the operation.  Authorization ID: "DSMUSR1".  Operation: "CREATE EVENT MONITOR". Object: "TS4MONITOR".. SQLCODE=-551, SQLSTATE=42501, DRIVER=4.26.14
    


    This one, was solved by granting 'USE' privilege in the target tablesapce TS4MONITOR. 

    db2 "grant use of tablespace TS4MONITOR TO USER DSMUSR1"

    Please,  we need these privileges/authorities required to be documented..   Not only these,  but ALL privileges and authorities required by the DSM monitor user in the target dbs..  And we need to be specific here..   not asking excessive privileges like SYSADM or DBADM authority which grants excessive power to DMC monitoring users. 


    After these two missing privileges were granted..  I could see the event monitor got created in the target db. 

    Regards



    ------------------------------
    Samuel Pizarro
    ------------------------------



  • 12.  RE: Locking and Utility Monitoring requiring WLM ? why ?

    Posted Mon December 16, 2019 02:48 AM
    Hi @Samuel Pizarro​ , nice to see event monitors work now. Thanks for your feedback on the privileges. We will discuss how to improve it next step. Thanks!

    ------------------------------
    XIAO MIN ZHAO
    ------------------------------