Power

 View Only
Expand all | Collapse all

How NMON analyzer calculated the TOP(%CPU) and CPU_ALL(CPU%)? Because the values of %CPU in TOP and CPU% in CPU_ALL are much differ.

  • 1.  How NMON analyzer calculated the TOP(%CPU) and CPU_ALL(CPU%)? Because the values of %CPU in TOP and CPU% in CPU_ALL are much differ.

    Posted Mon May 29, 2023 11:47 AM
    Dear AIX for NMON and NMON analyzer,
     
    Once generated with NMON analyzer done.
    In TOP for PID=18613336 shown the "%CPU" i.e 41.81,39.8... as i shown below.
    If comparing the values of "%CPU" in TOP and "CPU%" in CPU_ALL worksheet. The TOP(%CPU) > 30-43% by Avg and in CPU_ALL(CPU%) is < 3-10
     
    Customer question:  
     How NMON analyzer calculated the TOP(%CPU) and CPU_ALL(CPU%)?
     Because the values of %CPU in TOP and CPU% in CPU_ALL are much differ.
     
     
     
    TOP(%CPU)
          Time        PID         %CPU  %Usr  %Sys  Threads     Size  ResText     ResData      CharIO      %RAM  Paging      Command     WLMclass    IntervalCPU%      WSet  User      Arg               
          04:31:36    18613336    41.81 37.81 4     1     11248 8     11248 76533054    0      10    mumps Unclassified                 cbsadm      /cbsprod/gtm_dist/mumps -direct                  
          04:32:36    18613336    39.8  35.89 3.91  1     11248 8     11248 74702743    0      9     mumps Unclassified                 cbsadm      /cbsprod/gtm_dist/mumps -direct                  
          ...   
          04:58:37    18613336    41.27 35.16 6.1   1     11252 8     11252 74676420    0      9     mumps Unclassified                 cbsadm      /cbsprod/gtm_dist/mumps -direct                  
          04:59:37    18613336    42.33 36.07 6.25  1     11252 8     11252 75298145    0      10    mumps Unclassified                 cbsadm      /cbsprod/gtm_dist/mumps -direct                  
          ..
          05:24:37    18613336    40.42 35.6  4.82  1     11668 8     11668 68027121    0      381   mumps Unclassified                 cbsadm      /cbsprod/gtm_dist/mumps -direct                  
          05:25:37    18613336    26.31 22.87 3.44  1     11680 8     11680 43052669    0      362   mumps Unclassified                 cbsadm      /cbsprod/gtm_dist/mumps -direct                         
     
    CPU_ALL(CPU%)
     
          CPU Total   User% Sys%  Wait% Idle% Busy  PhysicalCPUs      CPU%
          04:24:37    1.2   1.9   0.7   96.1  3.1   64                3.1
          04:25:37    1.4   2.5   1.2   94.9  3.9   64                3.9
          04:26:37    1.4   2.5   1.2   94.9  3.9   64                3.9
          ...
          ...
          05:25:37    3.4   3.3   0.8   92.4  6.7   64                6.7
          05:33:37    1.6   2.1   0.5   95.8  3.7   64                3.7
          05:34:37    1.5   1.9   0.5   96.1  3.4   64                3.4
     
    Best Regards,
    Charin Kumjudpai.


    ------------------------------
    CHARIN KUMJUDPAI
    ------------------------------


  • 2.  RE: How NMON analyzer calculated the TOP(%CPU) and CPU_ALL(CPU%)? Because the values of %CPU in TOP and CPU% in CPU_ALL are much differ.

    Posted Tue May 30, 2023 10:04 AM

    NMON TOP data does not report a process using a CPU at a 100% if it is CPU bound, as you (and I) would expect, but instead adjusts the usage down based on relative usage of available logical processors. This "AIX adjustment" is depending on SMT level, POWER CPU generation and, potentially even clock frequency.
    What I typically do is to run a quick "CPU eater" and document wat TOP reports as CPU usage for that known CPU constrained process. Then I use that number to normalize all NMON TOP CPU utilization percentages accordingly.
    I use the following "CPU eater" as my test (ksh)
    let a=1
    while [ 1 ]; do
    let a=a+1
    done

    Below the results when I did this testing in my lab (note that you want to verify those numbers for your specific environment as AIX adjustments may have changed since then!)



    ------------------------------
    Ralf Schmidt-Dannert
    ------------------------------



  • 3.  RE: How NMON analyzer calculated the TOP(%CPU) and CPU_ALL(CPU%)? Because the values of %CPU in TOP and CPU% in CPU_ALL are much differ.

    Posted Thu June 01, 2023 12:42 AM

    Dear Ralf Schmidt-Dannert,

    Thank you so much for your response my question and provided the excellent example.

    Regards.
    Charin Kumjudpai.



    ------------------------------
    CHARIN KUMJUDPAI
    ------------------------------



  • 4.  RE: How NMON analyzer calculated the TOP(%CPU) and CPU_ALL(CPU%)? Because the values of %CPU in TOP and CPU% in CPU_ALL are much differ.

    Posted Wed April 24, 2024 09:45 AM

    This is very good information - thank you for sharing it! 

    Would you happen to have the original IBM slide deck or reference doc that you could attach or link to?  I've done some looking for the IBM source and am coming up empty.

    Thanks!



    ------------------------------
    Craig
    ------------------------------



  • 5.  RE: How NMON analyzer calculated the TOP(%CPU) and CPU_ALL(CPU%)? Because the values of %CPU in TOP and CPU% in CPU_ALL are much differ.

    Posted Thu April 25, 2024 02:07 PM

    Craig,

    I did this work internally - not sure there is "official IBM documentation" spelling it out like the slides do. (By the way, I'm with IBM)

    The slides are in an "Oracle on AIX Best Practices" presentation I did for virtual TechU in 2022. FYI, I'll be re-doing those presentations, refreshed/updated, on 5/2 and 5/9 and material will be shared after the calls.

    See Power VUG WEB site for further details:  https://www.ibm.com/support/pages/power-virtual-user-group

    Ralf.



    ------------------------------
    Ralf Schmidt-Dannert
    ------------------------------



  • 6.  RE: How NMON analyzer calculated the TOP(%CPU) and CPU_ALL(CPU%)? Because the values of %CPU in TOP and CPU% in CPU_ALL are much differ.

    Posted Mon May 06, 2024 10:47 AM

    Thank you Ralf! I did not realize that this was original research and that you were the author!

    As it happens, I was able to attend that 5/2 presentation and was so stoked to get to see you presenting this material! It's super helpful and helps us understand a lot better how to interpret nmon and how to consider workloads that have sensitivity to thread performance. Thank you!



    ------------------------------
    Craig
    ------------------------------