Db2

 View Only
Expand all | Collapse all

Timeout Errors - fresh new install

  • 1.  Timeout Errors - fresh new install

    Posted Sat February 22, 2020 03:09 PM
    Hi DMC team, 

    AS I was not able to fix my issue reported here, after upgrading to 3.1.5 release,  , I decided to start over. 

    Uninstalled and deleted DMC completely from my system.  
    Dropped my original repository db. 

    Download GUI installer,  and performed  a fresh install. 
    Created new repository db (empty). 

    After installation,  at the 1st execution (login) to the console, was prompted to configure the repo,  and add a new db connections. 
    All good at this point. 

    But,  when I got redirected to HOME page,  the recently added db was showing as not-monitored ,  and after long time,  got several timeout errors. 


    After these,  I tried refresh the page,,  and after few minutes,  other timeouts...  

    It has been 3 weeks, that I no longer have DMC running any more on this system.  cant make new release work here. 
    Target is not a HUGE db,  but also its not a small/dummy db.  Has hundreds of tablespaces and tables objects. 



    ------------------------------
    Samuel Pizarro
    ------------------------------

    #Db2


  • 2.  RE: Timeout Errors - fresh new install

    Posted Sun February 23, 2020 08:38 PM

    Hi Samuel,

    We attach great importance to your issues mentioned. Our developers are investigating them and will update you ASAP. Thanks for your patience.



    ------------------------------
    Li Hui Yan (Autumn Yan)
    From DMC Team
    ------------------------------



  • 3.  RE: Timeout Errors - fresh new install

    Posted Mon February 24, 2020 07:08 AM
    @LI HUI YAN

    Thanks for the feedback.

    Just would like to add,  that re-installing DMC from scrach didn`t solve my original issue. 

    After these timeouts,  I stopped DMC, The output in cmd window said it was stopped normally,  and .\status.bat also was saying it was not running.   
    But then java process was left running in memory, and  I had to kill it. (this is not the 1st time this happens with this release). 

    After killing the java,  I was able to restart DMC,  and this time,  I got no timetout errors anymore on dmc home page. 
    But,  I am still at the same stage of my original issue,   Summary page does not work.  nothing is shown.  But the target DB is not shown as "not monitored" any more. 

    Regards

    ------------------------------
    Samuel Pizarro
    ------------------------------



  • 4.  RE: Timeout Errors - fresh new install

    Posted Mon February 24, 2020 10:40 PM
    Edited by System Fri January 20, 2023 04:26 PM
    Hi Samuel,

     Our development team is looking into this case, some actions need from you. Thanks in advance.

    1. Change the DS_System and monitor log level to trace;
      The following method could be referred:
      Modify the following files and then restart the DMC server:
      ibm-datamgmtconsole/Config/DS_System_log.properties and ibm-datamgmtconsole/Config/monitor_log.properties, change the LogInitialLevel=6 to LogInitialLevel=13
    2. Open home page and then collect the log about at least 5 minutes;
    3. Send the logs folder(ibm-datamgmtconsole/logs) to us in this community.
      Any question or advice is welcome.


    ------------------------------
    Li Hui Yan (Autumn Yan)
    From DMC Team
    ------------------------------



  • 5.  RE: Timeout Errors - fresh new install

    Posted Wed February 26, 2020 08:00 PM
      |   view attached
    Hi @LI hui yan

    1. I have changed the property in config file.

    2​.  Per comments from Yan Yan Han (yanyh@cn.ibm.com) in our slack channel trhead,  I have started DMC using the desktop launcher icon this time.
    And I get the following warning​/error 

    The message is in Portuguese-BR,  which is the region/country set in my windows server, despite the "display language" has been set to english. 
    but the message would mean: 
    Error: Not possible to locate or load Db2 main classes

    But,  If I click "Continue", the next page says,  "Server stated successfully" 

    After opening DMC console webpage, the home is shown ,but the data initial data is never updated..   even after waiting for more than 30 minutes. 

    On Summary page,  it also stays freezed.  It looks that at least one initial capture was collected,  when DMC got restarted.  but nothing else,  so nothing get plotted at the graphs.  
    as you can see,  there was a capture collected at 21:20,  and I have waited more than 25min in this screen (current time on the top is 21:24)

    In order to capture the logs,  I need to stop DMC, otherwise the logs are opened by DMC and I cant include them in a zip file.  (access violation error). 
    And again,  once more,  after DMC has been stopped, (I stopped using desktop launcher icon this time),  javaw proccess was still left running on the system. 

    And I have to force kill it,  otherwise I got access violation errors trying to open console.log file the next time I try to start DMC,  

    I have uploaded the log files.  They are the complete set of logs,  since I re-installed DMC from scratch 4 days ago (this Saturday). 

    I suspect that one of my issues could be related to the Portuguese-BR language, as I see several java.lang erros in the logs. 

    Regards 
     



     


    ------------------------------
    Samuel Pizarro
    ------------------------------

    Attachment(s)

    zip
    dmc_311_win_samuel.logs.zip   1.18 MB 1 version


  • 6.  RE: Timeout Errors - fresh new install

    Posted Fri March 06, 2020 03:53 PM
    Hi DMC support

    @Yuan Gao ,  Per our chat on slack data-management-console channel,  I am trying to upload here the logs you asked for. 

    but,  work dir got 18GB of data..  not sure if the upload here will be able to handle that,  before timing out. 


    ​​​

    ------------------------------
    Samuel Pizarro
    ------------------------------



  • 7.  RE: Timeout Errors - fresh new install

    Posted Fri March 06, 2020 04:08 PM
      |   view attached
    attempt to upload the logs and dumps requested by @Yuan Gao

    ------------------------------
    Samuel Pizarro
    ------------------------------

    Attachment(s)

    zip
    dmc_bin_logs_work.zip   3.46 GB 1 version


  • 8.  RE: Timeout Errors - fresh new install

    Posted Wed March 11, 2020 12:34 PM
    Hi @Samuel Pizarro
    Thanks for collecting the logs and dumps.

    Could you please try to increase the Java heap size as a workaround to resolve this issue?
    Open the following file in your installation path of DMC C:/IBM_DB2_DMC/IBM Db2 Data Management Console/resources/bin/wlp/usr/servers/dsweb/jvm.options
    Change -Xmx2048m to -Xmx4096m or higher.
    Then restart DMC from C:/IBM_DB2_DMC/IBM Db2 Data Management Console/resources/bin/bin/restart.bat

    We have determined the root cause and will resolve this issue in future release. Thanks!

    ------------------------------
    Yuan Gao
    ------------------------------



  • 9.  RE: Timeout Errors - fresh new install

    Posted Thu April 02, 2020 09:47 AM

    Hi DMC support,  @Yuan Gao

    As we discussed in background, I have increased java heap to 4GB as requested.

    It helped to make the backgound monitoring to work again. It stays stable,  if I just keep DMC running,  just capturing background monitoring data. ​
    But I have two issues still with that. 

    1.  Its using too much memory (4GB),  for a system with 16 GB total ram available, where 12GB is being dedicated to the db2 instance (DMC running at the same server as the target db2 database being monitored.  Yes,  I know not optimal,  but thats what the customer has...)  
    But this env was running fine,  exactly with same server but running previous DMC release..   

    2.  The stability get broken If I start playing with DMC views, If I play around,  switching to different views in the console..   Request monitor data for package cache,  ordered by specific metric,  or tablespaces usages..  etc...  etc. etc.  eventually I get the OOM error again,  and DMC once again is affected, and stop working properly.  Really really unstable. 

    And looking at monitor.0 log,  I can see another OOM error again. 

    thanks 





    ------------------------------
    Samuel Pizarro
    ------------------------------



  • 10.  RE: Timeout Errors - fresh new install

    Posted Thu April 02, 2020 10:09 PM
    Hi Samuel ,

    1. Yes, it is not optimal to deploy DMC and the target database for monitoring on the same system. If resource is limited, DMC can work with the repository database together on the same system. 

    2. We've found the root cause of the previous OOM and fix this issue. It will be available in next release. However, it will be helpful if the dump files of the recent OOM can be shared with us, so we can make sure it is resolved. 

    Thanks a lot!

    ------------------------------
    Yuan Gao
    ------------------------------



  • 11.  RE: Timeout Errors - fresh new install

    Posted Sun April 05, 2020 09:58 AM
      |   view attached
    Hi @Yuan Gao

    Uploading the logs dir.  Let me know if you also need the work directory..  its huge,  so I have not uploaded it.

    I got the error, when I asked to sort the package cache grid by an specific metric (but I don't remember what metric was it, as it was attempt on March 10, and I havent restarted DMC since then. 

    monitor	(172)			Mar 10, 2020 12:29:21 PM		ERROR    NoxGR_172:exc = 
    ### Error querying database.  Cause: org.apache.ibatis.reflection.ReflectionException: Could not set property 'last_metrics_update_timestamp' of 'class com.ibm.aps.tools.monitor.base.model.monitor.pojo.Packagecache' with value '2020-03-10 15:22:34.148331' Cause: java.lang.OutOfMemoryError: Espaço de heap Java
    ### The error may exist in com/ibm/aps/tools/monitor/collector/mybatis/mapper/PackagecacheMapper.xml
    ### The error may involve com.ibm.aps.tools.monitor.collector.mybatis.mapper.PackagecacheMapper.collect110100
    ### The error occurred while handling results​


    Regards

    ------------------------------
    Samuel Pizarro
    ------------------------------

    Attachment(s)

    zip
    logs_Match10th.zip   4.46 MB 1 version


  • 12.  RE: Timeout Errors - fresh new install

    Posted Wed April 08, 2020 04:20 AM
    Hi Samuel,

    Thanks a lot for uploading the logs. I've done with investigation. I can confirm the OOM and web error are caused by the same code path, which will be fixed in the coming release.

    My suggestion:
    1. As a workaround, increase `-Xmx` to a higher value, e.g. `-Xmx8192m`;
    2. Upgrade DMC when the coming release is published.


    ------------------------------
    Yuan Gao
    ------------------------------



  • 13.  RE: Timeout Errors - fresh new install

    Posted Wed April 08, 2020 08:56 AM
    Thanks @Yuan Gao  for the feedback .

    Will wait for the next release,  as I don't have 8G free to run DMC on this host..    

    Any ideas when the next release will be made available ? 2Q at least ? 

    Regards



    ------------------------------
    Samuel Pizarro
    ------------------------------



  • 14.  RE: Timeout Errors - fresh new install

    Posted Wed April 08, 2020 10:03 AM
    @Samuel Pizarro ​ It will come in later this month.

    ------------------------------
    HAO WU
    ------------------------------



  • 15.  RE: Timeout Errors - fresh new install

    Posted Wed April 08, 2020 11:17 AM

    Hi @HAO WU

    Thank for the update.  



    ------------------------------
    Samuel Pizarro
    ------------------------------