HMC & CMC

 View Only

Using PCM REST APIs

By Archive User posted Fri May 20, 2016 03:42 AM

  

Originally posted by: Niraj Shah IBM


Overview

IBM Power System’s Hardware Management Consoles (HMCs) provide interfaces to monitor utilization of physical and virtual resources on the Power system. These are REST API and GUI interfaces for Performance and Capacity Monitoring or PCM. This blog is intended for developers who wants to write performance monitoring applications using the utilization data provided by PCM REST APIs.

 

PCM REST APIs

The PCM REST APIs provides four types of utilization data,

  1. Long Term Monitor (LTM) metrics: These metrics are collected from a resource for ongoing and continuous monitoring perspective. These are raw metrics and they are suitable for performance monitoring tools. The monitoring tool can tap into these data, do data mining and produce reports, resource utilization charts.
  2. Short Term Monitor (STM) metrics: These are detailed raw metrics, primarily useful for trouble shooting, error investigation. They are super set of LTM metrics and their collection frequency is higher than LTM metrics. They are also meant for performance monitoring tools. These metrics consumes additional server resources so it is advisable to run them for short duration.
  3. Processed metrics: HMC provides basic user interface for the performance monitoring. It uses the LTM metrics and processes them for the UI. The same processed data are also available through Processed metrics REST API.
  4. Aggregated metrics: HMC UI can show historic data utilization for up to one year. To retain the data for long duration while keeping the storage footprint low, HMC performs data aggregation by averaging the data. These data are also available through the REST API as Aggregated metrics.

Follow these steps to get started with the PCM REST APIs.

  1. Login to HMC using Logon request
  2. Enable performance monitoring for a managed system
  3. Fetch the utilization metrics

This blog focuses on fetching the LTM metrics. LTM metrics are snapshot of resource utilization at a particular point in time. They are collected at every 30 seconds and they are available on the HMC for 30 mins. Once the LTM metrics are enabled HMC continuously produces snapshot at 30 mins interval. At least once in 30 mins the REST client must make a request to fetch the generated data.

 

Logon request

Use the Logon API for doing user authentication.

URL

https://[hostname]:12443/rest/api/web/Logon

Replace the place holder with HMC host name or IP address.

HTTP header

Add following content type in HTTP header.

Content-Type: application/vnd.ibm.powervm.web+xml

Request body

Add the following text in the request body. Replace the placeholder for user name and password.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<LogonRequest xmlns="http://www.ibm.com/xmlns/systems/power/firmware/web/mc/2012_10/" schemaVersion="V1_3_0">

    <Metadata>

        <Atom/>

    </Metadata>

    <UserID kb="CUR" kxe="false">[User name]</UserID>

    <Password kb="CUR" kxe="false">[Password]</Password>

</LogonRequest>

HTTP method

Send HTTP PUT request.

 

Response

Upon successful execution of the request the HMC returns X-API-Session token in the response.

HTTP status code

The HTTP status code 200 in the response indicates successful execution of the request.

Response body

Retain and use the X-API-Session received from the response for the subsequent API calls.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<LogonResponse xmlns="http://www.ibm.com/xmlns/systems/power/firmware/web/mc/2012_10/" xmlns:ns2="http://www.w3.org/XML/1998/namespace/k2" schemaVersion="V1_3_0">

    <Metadata>

        <Atom/>

    </Metadata>

    <X-API-Session kxe="false" kb="ROR">DvrKltiYBeYm5x1cmdJJUwCIb8ORfjOpB7NF3k6Xg396cskq80bEFrR6QFS5s8mooLNS8_AfIXDbHWMH_gPaKielPfHta-Dus0HuoWH_veqgxK5ONSJ2KOq0_opuQsYH5hBxv6UX6wSICQRowmR5cRRwu0wLPvsj5F12biL1d8QvPDjiPh730Gj1avGz1DIcrmAgg9csUkg73rGbVTCSR_6YgirzvEv0SJoDu4LSTlo=</X-API-Session>

</LogonResponse>

 

Enable / disable performance monitoring preferences

By default the performance monitoring is not enabled. The user must enable it to start monitoring the system utilization. Use the Managed System PCM Preferences or Management Console PCM Preferences REST APIs to enable or disable performance monitoring.

The Managed System PCM Preferences REST API allows user to enable / disable PCM preferences for a specific managed system. The user must have appropriate permissions to perform this operation.

The Management Console PCM Preferences REST API allows user to enable / disable PCM preferences for multiple managed systems. It’s GET call lists all the managed systems according to user’s access rights. With a single POST call the user can enable/disable PCM preferences for one or more managed systems.

This blog focuses on enabling PCM monitoring for a particular managed system using Managed System PCM Preferences API.

 

Get Managed System PCM Preferences

Sent a HTTP get request to fetch PCM settings for a managed system.

URL

https://[hostname]:12443/rest/api/pcm/ManagedSystem/[uuid]/preferences

Replace the place holder with managed system UUID.

HTTP header

Add following HTTP headers. Replace the place holder X-API-Session with actual X-API-Session obtained through Logon call.

Content-Type: application/xml

X-API-Session: [X-API-Session]

Request body

Leave the request body empty.

HTTP method

Send HTTP GET request.

 

Response

Upon successful execution of the request the HMC returns PCM preference setting for the managed system.

HTTP status code

The HTTP status code 200 in the response indicates successful execution of the request.

Response body

The response is XML document.

<feed xmlns="http://www.w3.org/2005/Atom" xmlns:ns2="http://a9.com/-/spec/opensearch/1.1/" xmlns:ns3="http://www.w3.org/1999/xhtml">

    <id>8db6709f-7af6-3815-9f57-d38e2a4aa32e</id>

    <title type="text">Performance and Capacity Monitoring Preferences</title>

    <subtitle type="text"/>

    <link rel="self" href="https://[hostname]:12443/rest/api/pcm/ManagedSystem/8db6709f-7af6-3815-9f57-d38e2a4aa32e/preferences"/>

    <generator uri="IBM Power Systems Management Console" version="1"/>

    <entry>

        <id>1398112e-ee0b-4d9f-a38f-31d8a3cc71e1</id>

        <updated>2016-05-18T09:59:12.193+05:30</updated>

        <title type="text">Performance and Capacity Monitoring Preferences</title>

        <published>2016-05-18T09:59:12.193+05:30</published>

        <author>

            <name>IBM Power Systems Management Console</name>

        </author>

        <content type="application/xml">

            <ManagedSystemPcmPreference:ManagedSystemPcmPreference xmlns:ManagedSystemPcmPreference="http://www.ibm.com/xmlns/systems/power/firmware/pcm/mc/2012_10/" xmlns="http://www.ibm.com/xmlns/systems/power/firmware/pcm/mc/2012_10/" xmlns:ns2="http://www.w3.org/XML/1998/namespace/k2" schemaVersion="V1_3_0">

    <Metadata>

        <Atom>

            <AtomID>8db6709f-7af6-3815-9f57-d38e2a4aa32e</AtomID>

            <AtomCreated>1463545752121</AtomCreated>

        </Atom>

    </Metadata>

    <SystemName kxe="false" kb="ROR">server1</SystemName>

    <MachineTypeModelSerialNumber kxe="false" kb="ROR" schemaVersion="V1_3_0">

        <Metadata>

            <Atom/>

        </Metadata>

        <MachineType kxe="false" kb="ROR">9179</MachineType>

        <Model kb="ROR" kxe="false">MHD</Model>

        <SerialNumber kxe="false" kb="ROR">06064FV</SerialNumber>

    </MachineTypeModelSerialNumber>

    <LongTermMonitorEnabled kb="UOD" kxe="false">false</LongTermMonitorEnabled>

    <AggregationEnabled kxe="false" kb="UOD">false</AggregationEnabled>

    <ShortTermMonitorEnabled kb="UOD" kxe="false">false</ShortTermMonitorEnabled>

    <ComputeLTMEnabled ksv="V1_1_0" kxe="false" kb="UOD">false</ComputeLTMEnabled>

    <AssociatedManagedSystem kxe="false" kb="ROO" href="https://[hostname]:12443/rest/api/uom/ManagedSystem/8db6709f-7af6-3815-9f57-d38e2a4aa32e" rel="related"/>

</ManagedSystemPcmPreference:ManagedSystemPcmPreference>

        </content>

    </entry>

</feed>

 

Enable LTM metrics

To enable LTM metrics, send a POST request for the managed system PCM preference API.

URL

https://[hostname]:12443/rest/api/pcm/ManagedSystem/[uuid]/preferences

Replace the place holder with managed system UUID.

HTTP header

Add following HTTP headers. Replace the place holder X-API-Session with actual X-API-Session obtained through Logon call.

Content-Type: application/xml

X-API-Session: [X-API-Session]

Request body

<ManagedSystemPcmPreference:ManagedSystemPcmPreference xmlns:ManagedSystemPcmPreference="http://www.ibm.com/xmlns/systems/power/firmware/pcm/mc/2012_10/" xmlns="http://www.ibm.com/xmlns/systems/power/firmware/pcm/mc/2012_10/" xmlns:ns2="http://www.w3.org/XML/1998/namespace/k2" schemaVersion="V1_3_0">

    <Metadata>

        <Atom>

            <AtomID>8db6709f-7af6-3815-9f57-d38e2a4aa32e</AtomID>

            <AtomCreated>1463545752000</AtomCreated>

        </Atom>

    </Metadata>

    <SystemName kxe="false" kb="ROR">Server1</SystemName>

    <MachineTypeModelSerialNumber kxe="false" kb="ROR" schemaVersion="V1_3_0">

        <Metadata>

            <Atom/>

        </Metadata>

        <MachineType kxe="false" kb="ROR">9179</MachineType>

        <Model kb="ROR" kxe="false">MHD</Model>

        <SerialNumber kxe="false" kb="ROR">06064FV</SerialNumber>

    </MachineTypeModelSerialNumber>

    <LongTermMonitorEnabled kb="UOD" kxe="false">true</LongTermMonitorEnabled>

    <AggregationEnabled kxe="false" kb="UOD">false</AggregationEnabled>

    <ShortTermMonitorEnabled kb="UOD" kxe="false">false</ShortTermMonitorEnabled>

    <ComputeLTMEnabled ksv="V1_1_0" kxe="false" kb="UOD">false</ComputeLTMEnabled>

    <AssociatedManagedSystem kxe="false" kb="ROO" href="https://[hostname]:12443/rest/api/uom/ManagedSystem/8db6709f-7af6-3815-9f57-d38e2a4aa32e" rel="related"/>

</ManagedSystemPcmPreference:ManagedSystemPcmPreference>

HTTP method

Send HTTP POST request.

 

Response

Upon successful execution of the request the HMC returns updated PCM preference setting for the managed system.

HTTP status code

The HTTP status code 200 in the response indicates successful execution of the request.

Response body

The response is XML document. The response returns the updated status of the performance monitoring for the managed system.

Disabling the metrics

You can disable a particular metrics by sending the same request with the metrics configuration value as “false”.

HMC also has auto – disable function. In case of inactivity HMC automatically disables the collection. Auto – disable function,

  • LTM metrics: If HMC doesn’t receive any request for LTM metrics in 30 mins, it automatically disables the collection.
  • STM metrics: STM metrics are auto-disabled by HMC after 15 mins. To continue with STM collection after 15 mins, user must issue a preferences REST API to enable it.
  • Aggregated metrics: There is not auto-disable function for the Aggregated metrics. They remain enabled until user disables them. If the Aggregated metrics is enabled, it internally calls LTM REST API to fetch the raw metrics. I.e if the user enables Aggregated metrics it also enables LTM metrics.

 

Fetch the LTM metrics

This section explains how to fetch the LTM metrics for the managed system. The other APIs i.e. Processed metrics, Aggregated metrics or STM metrics are very similar.

Get LTM metrics

Sent a HTTP GET request to fetch LTM metrics for a managed system.

URL

https://[hostname]:12443/rest/api/pcm/ManagedSystem/[uuid]/ RawMetrics/LongTermMonitor

Replace the place holders hostname and uuid with actual HMC hostname / ip address and managed system’s UUID.

URL with query parameters and timezone

The URL also takes query parameters, they are start timestamp and end timestamp. They are optional parameters. If specified, the response has the metrics within the specified time range. E.g. to fetch the data between 4:50 pm to 5:00 pm on 16 May 2016, issue the URL with the time range in the query parameters.

https://[hostname]:12443/rest/api/pcm/ManagedSystem/[uuid]/ RawMetrics/LongTermMonitor?StartTS=2016-05-18T16:50:00+0530&EndTS=2016-05-18T17:00:00+0530

Here the +0530 is the REST request timezone. The HMC may be running in a different timezone than the request timezone. The HMC is fetching the data from the managed system and VIOSes running on the managed system. Each of them may have different timezone setting. HMC converts all of these timezones into UTC and responds with files falling within the specified time range.

Note regarding Linux shell: The Linux shell uses “&” symbol to put a program into background. If you are issuing an URL on curl with “&” symbol; put the entire URL within double quotes (“”).

HTTP header

Add following HTTP headers. Replace the place holder X-API-Session with actual X-API-Session obtained through Logon call.

Content-Type: application/xml

X-API-Session: [X-API-Session]

Request body

Leave the request body empty.

HTTP method

Send HTTP GET request.

 

Response

Upon successful execution of the request the HMC returns links to the LTM metrics for the managed system.

HTTP status code

The HTTP status code 200 in the response indicates successful execution of the request.

Response body

The response is XML document.

<feed xmlns="http://www.w3.org/2005/Atom" xmlns:ns2="http://a9.com/-/spec/opensearch/1.1/" xmlns:ns3="http://www.w3.org/1999/xhtml">

    <id>8db6709f-7af6-3815-9f57-d38e2a4aa32e</id>

    <updated>2016-05-18T17:00:00.000+05:30</updated>

    <title type="text">LongTermMetrics</title>

    <subtitle type="text">ManagedSystem 8db6709f-7af6-3815-9f57-d38e2a4aa32e</subtitle>

    <link rel="self" href="https://[hostname]:12443/rest/api/pcm/ManagedSystem/8db6709f-7af6-3815-9f57-d38e2a4aa32e/RawMetrics/LongTermMonitor?StartTS=2016-05-18T16:50:00+0530&amp;EndTS=2016-05-18T17:00:00+0530"/>

    <generator uri="IBM Power Systems Management Console" version="1"/>

    <entry>

        <id>dc5fe56a-7f06-4b71-8e8b-b627cf9aa1e6</id>

        <updated>2016-05-18T16:55:30.000+05:30</updated>

        <title type="text">LTM_9179-MHD*06064FV_phyp_20160518T165530+0530.json</title>

        <published>2016-05-18T16:55:30.000+05:30</published>

        <link type="application/vnd.ibm.powervm.pcm.json" href="https://[hostname]:12443/rest/api/pcm/ManagedSystem/8db6709f-7af6-3815-9f57-d38e2a4aa32e/RawMetrics/LongTermMonitor/LTM_9179-MHD*06064FV_phyp_20160518T165530+0530.json"/>

        <author>

            <name>IBM Power Systems Management Console</name>

        </author>

        <category term="phyp"/>

    </entry>

      <!-- |  More entires

           |

           |      -->

    <entry>

        <id>6c48e1a1-e8bc-405f-a442-79551f3340ff</id>

        <updated>2016-05-18T16:59:30.000+05:30</updated>

        <title type="text">LTM_9179-MHD*06064FV_vios_5_20160518T165930+0530.json</title>

        <published>2016-05-18T16:59:30.000+05:30</published>

        <link type="application/vnd.ibm.powervm.pcm.json" href="https://[hostname]:12443/rest/api/pcm/ManagedSystem/8db6709f-7af6-3815-9f57-d38e2a4aa32e/RawMetrics/LongTermMonitor/LTM_9179-MHD*06064FV_vios_5_20160518T165930+0530.json"/>

        <author>

            <name>IBM Power Systems Management Console</name>

        </author>

        <category term="vios_5"/>

    </entry>

      <!-- |  More entires

           |

           |      -->

    <entry>

        <id>21896694-05cf-4ce0-a7d6-816d705f1812</id>

        <updated>2016-05-18T16:57:00.000+05:30</updated>

        <title type="text">LTM_9179-MHD*06064FV_vios_1_20160518T165700+0530.json</title>

        <published>2016-05-18T16:57:00.000+05:30</published>

        <link type="application/vnd.ibm.powervm.pcm.json" href="https://[hostname]:12443/rest/api/pcm/ManagedSystem/8db6709f-7af6-3815-9f57-d38e2a4aa32e/RawMetrics/LongTermMonitor/LTM_9179-MHD*06064FV_vios_1_20160518T165700+0530.json"/>

        <author>

            <name>IBM Power Systems Management Console</name>

        </author>

        <category term="vios_1"/>

    </entry>

      <!-- |  More entires

           |

           |      -->

    <entry>

        <id>0f2d8711-102e-47f9-a306-efe4ae64458c</id>

        <updated>2016-05-18T16:59:30.000+05:30</updated>

        <title type="text">LTM_9179-MHD*06064FV_vios_3_20160518T165930+0530.json</title>

        <published>2016-05-18T16:59:30.000+05:30</published>

        <link type="application/vnd.ibm.powervm.pcm.json" href="https://[hostname]:12443/rest/api/pcm/ManagedSystem/8db6709f-7af6-3815-9f57-d38e2a4aa32e/RawMetrics/LongTermMonitor/LTM_9179-MHD*06064FV_vios_3_20160518T165930+0530.json"/>

        <author>

            <name>IBM Power Systems Management Console</name>

        </author>

        <category term="vios_3"/>

    </entry>

</feed>

 

Send the following HTTP GET request to fetch the actual utilization data.

E.g. get PHYP’s LTM data using,

https://[hostname]:12443/rest/api/pcm/ManagedSystem/8db6709f-7af6-3815-9f57-d38e2a4aa32e/RawMetrics/LongTermMonitor/LTM_9179-MHD*06064FV_phyp_20160518T165530+0530.json

Get VIOS 1’s LTM data using,

https://[hostname]:12443/rest/api/pcm/ManagedSystem/8db6709f-7af6-3815-9f57-d38e2a4aa32e/RawMetrics/LongTermMonitor/LTM_9179-MHD*06064FV_vios_1_20160518T165700+0530.json

 

The performance metrics

Using the LTM metrics data the user can get / compute the following metrics.

Compute Utilization

  • Processor allocation and utilization
  • Memory assignment and utilization
  • Installed and configurable processors
  • Installed and configurable memory
  • Processor pool and memory pool utilization
  • Partition’s affinity score, dispatch wait time

IO Utilization

  • Network Bridge Traffic
    • Sent and received bytes and packets
    • Metrics per partition and per VIOS
  • SRIOV Traffic
    • For physical ports and logical ports
    • Metrics per partition and per VIOS
    • Sent and received bytes and packets
  • Partition’s Virtual LAN Traffic
    • Sent and received bytes and packets
  • Virtual SCSI Traffic
    • Metrics per partition and per VIOS
    • Sent and received bytes and packets
  • Fiber Channel Traffic
    • Metrics per partition and per VIOS
    • Sent and received bytes and packets
  • SSP Utilization
    • Total and used storage Capacity
    • Per VIOS Traffic in and out of pool

 

Metrics computation

The LTM API returns utilization snapshot data. For computation you need two samples (ideally two consecutive samples). E.g. sample1 was captured at time t1 and sample2 was captured at time t2 i.e. t1 + 30 secs. Followings are few examples of it.

CPU utilization for a LPAR / VIOS

Use the following algorithm / formula to compute CPU utilization by a LPAR.

Step 1: From sample1’s PHYP LTM data compute the units used by a LPAR. I.e. Go to utilSample -> lparsUtil -> processor. Units consumed by LPAR = (utilizedCappedProcCycles + utilizedUnCappedProcCycles - idleProcCycles)
Step 2: get the timeBasedCycles from sample1. I.e. Go to utilSample -> timeBasedCycles
Step 3: Repeat the step 1 for sample2
Step 4: Repeat the step 2 for sample2
Step 5: CPU utilization for a LPAR = (sample2 units consumed by a LPAR – sample1 units consumed by a LPAR) / (timeBasedCycles for sample2 - timeBasedCycles for sample1)

If the two samples are 30 secs apart, this gives the CPU utilization by a LPAR over the 30 seconds.

Apply the same formula to compute the units consumed by a VIOS.

If the LPAR is using sharedProcPool (i.e. poolId >= 0) then do not consider idleProcCycles in the calculation.

 

Host Wide CPU utilization calculation

Use the following algorithm / formula to compute host wide CPU utilization.

Step 1: Compute and sum the CPU utilization for all the LPARs
Step 2: Compute and sum the CPU utilization for all the VIOSes
Step 3: get utilized proc cycles by system firmware for sample1. I.e. utilSample -> systemFirmware -> utilizedProcCycles
Step 4: get utilized proc cycles by system firmware for sample2.
Step 5: Utilization by systemFirmware = (utilizedProcCycles for sample2 - utilizedProcCycles for sample1) / (timeBasedCycles for sample2 - timeBasedCycles for sample1)
Step 6: Host Wide CPU utilization = Utilization by Partitions (LPARs + VIOSes) + Utilization by systemFirmware

 

Per VM I/O rate metrics

To compute network metrics for a VM (LPAR or VIOS) apply the following formulas for each network adapter. It is assumed that the two samples are 30 secs apart so I'm dividing the delta by 30. Sum up network adapters traffic for all adapters to find out overall network traffic per second by the LPAR or VIOS.

Sent bytes = (sentBytes for sample2 - sentBytes for sample1)/30
Received bytes = (receivedBytes for sample2 - receivedBytes for sample1)/30

 

To compute storage metrics for a VM (LPAR or VIOS) apply the following formulas for each storage adapter. It is assumed that the two samples are 30 secs apart so I'm dividing the delta by 30. Sum up the bytes across all the storage adapters to find out overall storage IO per second by the LPAR or VIOS.

Read bytes = (readBytes for sample2 - readBytes for sample1)/30
Write bytes = (writeBytes for sample2 - writeBytes for sample1)/30

 

Authorization

The PCM REST API user requires appropriate authorization permissions to use the APIs. The user authorization is reusing HMC’s resource and task based authorization mechanism.

Tasks and Resources

There are three different tasks defined in HMC to use the PCM REST APIs.

  1. ListUtilizationData: The task is applicable on the managed system resource. The task allows user to view / fetch the PCM utilization metrics for the managed system. The user can also view the PCM settings for the managed system.
  2. ManageUtilizationData: The task is applicable on the managed system resource. It allows user to enable or disable performance monitoring for the managed system.
  3. ChangeUtilizationConfiguration: This task is applicable on the ManagementConsole resource. It allows user to change storage duration for the aggregated metrics.

 

References

PCM REST API documentation in the IBM knowledge center, https://www.ibm.com/support/knowledgecenter/POWER8/p8ehl/concepts/PerformanceCapacityMonitor.htm?cp=POWER8%2F1-4-3-14

For any queries, suggestions, feedback use the HMC developerworks forum, https://www.ibm.com/developerworks/community/forums/html/forum?id=4bbda4a1-294f-4cc2-a33c-45a4160ab15a

 

0 comments
8 views

Permalink