AIX Open Source

AIX Open Source

Share your experiences and connect with fellow developers to discover how to build and manage open source software for the AIX operating system

 View Only
  • 1.  metricbeat is not returning expected CPU usage values

    Posted Mon March 25, 2024 11:38 AM

    Hello,

         I have installed a metricbeat in an AIX 7.2 server and I need to obtain CPU usage values. The problem I have is these values seems not to reflect the current status of the hosts. This is an example:

    - CPU usage values of an AIX server obtained from metricbeat today at 10:50AM (return a document each 10 seconds):

    system.cpu.cores --> 16

    system.cpu.total.pct --> 9.924

    system.cpu.user.pct --> 9.497

    system.cpu.system.pct --> 0.427

    system.cpu.iowait.pct --> 0.002

    system.cpu.idle.pct --> 6.074

    - Values obtained from standard Topas Monitor in the same AIX server at the same time:

    CPU User% --> 81.5

    CPU Kern% --> 2.9

    CPU Wait% --> 0.0

    CPU Idle% --> 15.6

    CPU Physc --> 3.76

    CPU Entc% --> 187.97

    - Values obtained in Dynatrace tool at the same time:

    CPU User --> 80.1%

    CPU system --> 2.92%

    CPU I/O wait --> 0.009%

    CPU idle --> 17%

    It seems that Dynatrace is obtaining the same CPU metrics than Topas monitor (both based on perfstat API) but metricbeat values are totally different, because if the do the calcs for example with the CPU User, we would have the following:

    (CPU User divide by the number of cores) * 100 = (9.497/16)*100 = 59.36%, so we have 20% less using metricbeat than in Dynatrace.

    Does anybody knows what could be happening? The calc that I'm doing with the CPU usage divide by the number of cores is correct? If I modify the metricbeat specifying that I need the normalized values instead the current values, should I obtain the same values than now? I have discarded as well unexpected CPU peaks because all values 10 minutes after and before 10:50AM are returning similar values.

    I'm lost with this because we use two tools (Dynatrace and Elastic) but the values are totally different on both, and we need to know what are the correct values (Dynatrace&Topas vs Elastic).

    Many thanks in advance.

    Best regards.

    Juan M.



    ------------------------------
    Juan Manuel García Fernández
    ------------------------------


  • 2.  RE: metricbeat is not returning expected CPU usage values
    Best Answer

    Posted Wed March 27, 2024 02:27 AM

    Hello Juan,

    you are right. Metricbeat under AIX is not usable for CPU metrics. Just measures the threads, not the VP's used.
    And as we have multiple generations of Power Systems with different multithreading settings, we are not able to see or estimate how much workload we have on a managed system, even because metricbeat is not delivering managed system with in the data.
    We are using it just for memory... the rest comes from lpar2rrd.

    kind regards



    ------------------------------
    Joerg Kauke
    Unix Administrator
    COOP Switzerland
    ------------------------------



  • 3.  RE: metricbeat is not returning expected CPU usage values

    Posted Wed March 27, 2024 05:16 AM

    Hello Joerg,

         Many thanks for your fast response. As we have filebeat already installed on the servers and we need the CPU metrics available in Kibana, the idea that comes to my mind is to try to execute some command in the AIX servers that returns the CPU Usage, move the result to a file and inject that file using filebeat (after schedule the command execution with a specific frequency).

    Do you know any recommendation about what command we can use for this purpose?

    Many thanks in advance.

    Best regards.



    ------------------------------
    Juan Manuel García Fernández
    ------------------------------



  • 4.  RE: metricbeat is not returning expected CPU usage values

    Posted Wed March 27, 2024 05:40 AM

    We are using LPAR2RRD daily reports and sent the created csv via filebeat. We tried njmon first, but the nmon file has to many data points and can not be ingested to logstash. So, if you want to have the data in Kibana in real tome (every minute), you maybe can use the njmon file grep for the values you need and put in in a csv file before uploading it via filebeat.

    kind regards



    ------------------------------
    Joerg Kauke
    Unix Administrator
    COOP Switzerland
    ------------------------------



  • 5.  RE: metricbeat is not returning expected CPU usage values

    Posted Wed March 27, 2024 06:05 AM

    Hello Joerg,

         Many thanks again for your response. I have investigated a bit about this topic and as we don't want to install anything on the server, I think we can use vmstat command because we only need to obtain CPU usage values (memory and disk metrics are currently injected as well by metricbeat and they seem to be OK).

    Do you see that vmstat command is a good option? I suppose that we can easily export those values to a file and inject them in ELK using filebeat.

    Best regards.



    ------------------------------
    Juan Manuel García Fernández
    ------------------------------



  • 6.  RE: metricbeat is not returning expected CPU usage values

    Posted Wed March 27, 2024 07:49 AM

    Hello Juan,

    of course, vmstat is fine. Another command for system metrics would be "sar -u"
    https://www.ibm.com/docs/en/aix/7.3?topic=s-sar-command

    kind regards,
    Joerg



    ------------------------------
    Joerg Kauke
    Unix Administrator
    COOP Switzerland
    ------------------------------