Thank you Ralf! I did not realize that this was original research and that you were the author!
As it happens, I was able to attend that 5/2 presentation and was so stoked to get to see you presenting this material! It's super helpful and helps us understand a lot better how to interpret nmon and how to consider workloads that have sensitivity to thread performance. Thank you!
Original Message:
Sent: Thu April 25, 2024 02:07 PM
From: Ralf Schmidt-Dannert
Subject: How NMON analyzer calculated the TOP(%CPU) and CPU_ALL(CPU%)? Because the values of %CPU in TOP and CPU% in CPU_ALL are much differ.
Craig,
I did this work internally - not sure there is "official IBM documentation" spelling it out like the slides do. (By the way, I'm with IBM)
The slides are in an "Oracle on AIX Best Practices" presentation I did for virtual TechU in 2022. FYI, I'll be re-doing those presentations, refreshed/updated, on 5/2 and 5/9 and material will be shared after the calls.
See Power VUG WEB site for further details: https://www.ibm.com/support/pages/power-virtual-user-group
Ralf.
------------------------------
Ralf Schmidt-Dannert
Original Message:
Sent: Wed April 24, 2024 09:39 AM
From: Craig
Subject: How NMON analyzer calculated the TOP(%CPU) and CPU_ALL(CPU%)? Because the values of %CPU in TOP and CPU% in CPU_ALL are much differ.
This is very good information - thank you for sharing it!
Would you happen to have the original IBM slide deck or reference doc that you could attach or link to? I've done some looking for the IBM source and am coming up empty.
Thanks!
------------------------------
Craig
Original Message:
Sent: Tue May 30, 2023 10:04 AM
From: Ralf Schmidt-Dannert
Subject: How NMON analyzer calculated the TOP(%CPU) and CPU_ALL(CPU%)? Because the values of %CPU in TOP and CPU% in CPU_ALL are much differ.
NMON TOP data does not report a process using a CPU at a 100% if it is CPU bound, as you (and I) would expect, but instead adjusts the usage down based on relative usage of available logical processors. This "AIX adjustment" is depending on SMT level, POWER CPU generation and, potentially even clock frequency.
What I typically do is to run a quick "CPU eater" and document wat TOP reports as CPU usage for that known CPU constrained process. Then I use that number to normalize all NMON TOP CPU utilization percentages accordingly.
I use the following "CPU eater" as my test (ksh)
let a=1
while [ 1 ]; do
let a=a+1
done
Below the results when I did this testing in my lab (note that you want to verify those numbers for your specific environment as AIX adjustments may have changed since then!)


------------------------------
Ralf Schmidt-Dannert
Original Message:
Sent: Mon May 29, 2023 11:46 AM
From: CHARIN KUMJUDPAI
Subject: How NMON analyzer calculated the TOP(%CPU) and CPU_ALL(CPU%)? Because the values of %CPU in TOP and CPU% in CPU_ALL are much differ.
Dear AIX for NMON and NMON analyzer,
Once generated with NMON analyzer done.
In TOP for PID=18613336 shown the "%CPU" i.e 41.81,39.8... as i shown below.
If comparing the values of "%CPU" in TOP and "CPU%" in CPU_ALL worksheet. The TOP(%CPU) > 30-43% by Avg and in CPU_ALL(CPU%) is < 3-10
Customer question:
How NMON analyzer calculated the TOP(%CPU) and CPU_ALL(CPU%)?
Because the values of %CPU in TOP and CPU% in CPU_ALL are much differ.
TOP(%CPU)
Time PID %CPU %Usr %Sys Threads Size ResText ResData CharIO %RAM Paging Command WLMclass IntervalCPU% WSet User Arg
04:31:36 18613336 41.81 37.81 4 1 11248 8 11248 76533054 0 10 mumps Unclassified cbsadm /cbsprod/gtm_dist/mumps -direct
04:32:36 18613336 39.8 35.89 3.91 1 11248 8 11248 74702743 0 9 mumps Unclassified cbsadm /cbsprod/gtm_dist/mumps -direct
...
04:58:37 18613336 41.27 35.16 6.1 1 11252 8 11252 74676420 0 9 mumps Unclassified cbsadm /cbsprod/gtm_dist/mumps -direct
04:59:37 18613336 42.33 36.07 6.25 1 11252 8 11252 75298145 0 10 mumps Unclassified cbsadm /cbsprod/gtm_dist/mumps -direct
..
05:24:37 18613336 40.42 35.6 4.82 1 11668 8 11668 68027121 0 381 mumps Unclassified cbsadm /cbsprod/gtm_dist/mumps -direct
05:25:37 18613336 26.31 22.87 3.44 1 11680 8 11680 43052669 0 362 mumps Unclassified cbsadm /cbsprod/gtm_dist/mumps -direct
CPU_ALL(CPU%)
CPU Total User% Sys% Wait% Idle% Busy PhysicalCPUs CPU%
04:24:37 1.2 1.9 0.7 96.1 3.1 64 3.1
04:25:37 1.4 2.5 1.2 94.9 3.9 64 3.9
04:26:37 1.4 2.5 1.2 94.9 3.9 64 3.9
...
...
05:25:37 3.4 3.3 0.8 92.4 6.7 64 6.7
05:33:37 1.6 2.1 0.5 95.8 3.7 64 3.7
05:34:37 1.5 1.9 0.5 96.1 3.4 64 3.4
Best Regards,
Charin Kumjudpai.
------------------------------
CHARIN KUMJUDPAI
------------------------------