DataPower

DataPower

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Expand all | Collapse all

Traffic details

  • 1.  Traffic details

    Posted Mon July 17, 2023 09:29 AM

    Hi Community,

    How can we check the average response times of transactions of all the transactions reaching the datapower appliance.
    Also, details like total number of transactions, average response time, failed transactions count etc.

    I tried to look these in object like Tx Throughput, transaction rate ; but couldn't fetch much details.

    Thanks 



  • 2.  RE: Traffic details

    Posted Mon July 17, 2023 09:39 AM

    One way to gather statistics is to look at each transaction's latency record. You can create a log target to subscribe to this latency record and record them in a separate file. Latency records are not created for failed transactions.

    https://www.ibm.com/docs/en/datapower-gateway/10.0.1?topic=targets-latency-messages



    ------------------------------
    Charlie Sumner
    ------------------------------



  • 3.  RE: Traffic details

    Posted Mon July 17, 2023 10:23 AM
    Edited by Joseph Morgan Mon July 17, 2023 10:24 AM

    To add to Charlie's answer, there are several other ways:

    1) Look into count monitors for counts.

    2) Look into duration monitors for latency.

    3) Look into the SLM policy, which is really, really granular, and still really cool, if you need to get beyond simple counts and durations provided by the Count and Duration monitors.

    4) The probe data actually has latency data, but, you have to take that with a grain of salt, being that the probe is on, it will reflect higher than normal latency.   Kumbasoft's DPAA (free Community Edition) will show you per action latency in its aggregated probe view, even if the transaction fails. 



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 4.  RE: Traffic details

    Posted Tue July 18, 2023 02:11 AM

    Consider using DPOD (DataPower Operations Dashboard) - IBM's premier tool for monitoring and management of IBM DataPower Gateways - but that requires purchasing a separate license. It has much more than what you mentioned.

    A quick overview video (6 min) is available at https://ibm.biz/dpod-docs



    ------------------------------
    Amit Munwes
    ------------------------------



  • 5.  RE: Traffic details

    Posted Tue July 25, 2023 02:39 AM

    Thanks for the suggestion.

    Also, we can have the use of SNMP to read the particular MIB/OID's.

    Can you please help with the most tracked metrics and their MIB and OID's



    ------------------------------
    Sunil Chaurasia
    ------------------------------



  • 6.  RE: Traffic details

    Posted Tue July 25, 2023 01:30 PM

    >  Also, we can have the use of SNMP to read the particular MIB/OID's.

    Yes.  To get the MIBs off of the box, log into the appliance's default domain, in the control panel, search for SNMP settings and open the configuration.  Then select the "Enterprise MIBs" tab.  There you can get the MIBs and feed them into whatever tool you are using for SNMP.

    > Can you please help with the most tracked metrics and their MIB and OID's

    This is a complicated issue, but certainly start with memory, load and CPU.  You'll likely also want to monitor free file system and encrypted file system space. 

    What are you most interested in?

    For other considerations, are you using Physical or Virtual machines, because this can matter.



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 7.  RE: Traffic details

    Posted Wed July 26, 2023 07:58 AM
    Edited by Sunil Chaurasia Wed July 26, 2023 07:59 AM

    Thanks @Joseph for helping on the first point.

    As of now, we have just started with CPU, Memory, reloads alerts.

    We would like to track :-
    1) Total number of transactions processed by an appliance in 10 min, 1 day.
    2) Number of users logged in?
    3) Manual Configuration changes like cache flush

    What is the difference between these two?
    "dpStatusConnectionsAcceptedtenMinutes"  -- .1.3.6.1.4.1.14685.3.1.13.3 

    "dpStatusHTTPTransactions2tenMinutes" -- (tps) .1.3.6.1.4.1.14685.3.1.153.1.5


    We are using Physical Appliances.

    ------------------------------
    Sunil Chaurasia
    ------------------------------



  • 8.  RE: Traffic details

    Posted Wed July 26, 2023 01:07 PM

    What firmware version are you running?  I don't think HTTP Transactions status (original nor 2) are used anymore.   HTTP Connections status might be more useful, though is it per XML Manager and must be aggregated across all domains to get a total picture.

    Connections Accepted is pretty self-explanatory, but may not represent the actual number of transactions, depending upon how one defines a "transaction".  That is, I don't consider an accepted connection as necessarily a transaction.

    When you say number of users logged in, I suspect you're not talking about active DataPower users?  I suspect you are talking about the number of unique users accessing services, correct?   If, however, you are talking about active DataPower users, then the Active Users status is what you're seeking.

    Flushing caches is not a configuration change.  Depending upon the activity, you might can get the configuration changes via the audit log, but, you have to poll it and will likely not get what you're seeking.  The best way to look for configuration changes is to create a logging target subscribed to the 'mgmt' category, and then a subset of the event codes under that category.  You'll not want all of them for sure if you're just seeking configuration changes.



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 9.  RE: Traffic details

    Posted Wed July 26, 2023 04:06 PM

    Hi Joseph,

    Please find my comments inline


    What firmware version are you running?  I don't think HTTP Transactions status (original nor 2) are used anymore.   HTTP Connections status might be more useful, though is it per XML Manager and must be aggregated across all domains to get a total picture.

    IDG.10.0.1.11
    Do we need to configure in order to get the aggregated value or fetching the details on OID "dpStatusConnectionsAcceptedtenMinutes"  -- .1.3.6.1.4.1.14685.3.1.13.3  give the consolidated count?



    Connections Accepted is pretty self-explanatory, but may not represent the actual number of transactions, depending upon how one defines a "transaction".  That is, I don't consider an accepted connection as necessarily a transaction.

    When you say number of users logged in, I suspect you're not talking about active DataPower users?  I suspect you are talking about the number of unique users accessing services, correct?   If, however, you are talking about active DataPower users, then the Active Users status is what you're seeking.

    Looking for Number of active users on datapower.
    I think this will give the details dpStatusActiveUsersname .1.3.6.1.4.1.14685.3.1.1.1.2




    ------------------------------
    Sunil Chaurasia
    ------------------------------



  • 10.  RE: Traffic details

    Posted Wed July 26, 2023 11:34 PM

    I believe dpStatusActiveUsersname will give you currently active users.   dpStatusConnectionsAcceptedTenMinutes will give you the number of connections over 10 minutes, if that is satisfactory to you.  This is not, however, the same as number of transactions.

    You shouldn't need to configure anything special other than the user and SNMP credentials for your SNMP client to obtain these.    Not knowing your client, I cannot tell you about how to configure it.



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 11.  RE: Traffic details

    Posted Thu July 27, 2023 02:42 AM

    Thanks for your support Joseph.

    dpStatusConnectionsAcceptedTenMinutes will give you the number of connections over 10 minutes, if that is satisfactory to you.  This is not, however, the same as number of transactions.

    Any Specific OID to track transaction counts ?
    Although i was unable to find, if any.


    You shouldn't need to configure anything special other than the user and SNMP credentials for your SNMP client to obtain these.    Not knowing your client, I cannot tell you about how to configure it.

    We are aware of the Client and client is able to connect to DP and gather CPU, Memory etc details.


    You can also suggest few other OID's which you think can be very useful in terms of monitoring.



    ------------------------------
    Sunil Chaurasia
    ------------------------------



  • 12.  RE: Traffic details

    Posted Thu July 27, 2023 12:14 PM

    Sorry for the delay, I spent some time going over the MIBs.  It seems many are not-accessible, but I'm not sure that is literal, mainly because if all of the MIBs listed as not-accessible were truly that, I'm not sure why one would want to monitor DataPower via SNMP.

    The dpStatusActiveUsersTable, if accessible, should provide all users logged into the DataPower appliance somewhere (WebGUI, CLI, XMLMgt, etc). 

    The dpStatusConnectionsAccepted may be useful for monitoring for general traffic hits on the appliances, but these don't always result in a transaction.  However, if you're seeking baselines and want to alert when over normal baselines, this could definitely be useful.

    I'm saying all this just looking at the MIB files with my eyeballs.  I'll setup a client and point it at my lab and see what I get.



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 13.  RE: Traffic details

    Posted Mon August 07, 2023 11:25 AM

    Hi Joseph,

    dpStatusConnectionsAccepted will basically count the number of hits received by this particular appliance by Sources (external hits, internal hits from other connected DataPower appliances), if i am understanding correctly.

    Please provide further more info for this attribute(MIB), on what actually details it gathers.



    ------------------------------
    Sunil Chaurasia
    ------------------------------



  • 14.  RE: Traffic details

    Posted Mon August 07, 2023 12:02 PM

    Lots of good discussion below and I wanted to add the simplest way to see average latency for a given service is via the 'show http' in the domain with the service(s) of interest.  I get at this status provided via rest-mgmt interface using this json
    {
        "statusProviders": [
            {
                "domain": "apiconnect",
                "name": "HTTPMeanTransactionTime2",
                "keys": [ "proxy" ],
                "fields": [ "tenSeconds" ]
            },
            {
                "domain": "apiconnect",
                "name": "HTTPTransactions2",
                "keys": [ "proxy" ],
                "fields": [ "tenSeconds" ]
            },
            {
                "domain": "default",
                "name": "SystemUsage2Table",
                "keys": [ "TaskID", "TaskName" ],
                "fields": [ "Load", "WorkList", "CPU", "Memory", "FileCount" ]
            }
        ]
    }



    ------------------------------
    Ivan Heninger
    ------------------------------



  • 15.  RE: Traffic details

    Posted Mon August 21, 2023 09:24 AM

    I agree with Ivan, a lot of great discussion on this topic.  I'll add one "got ya" I remember about SNMP from my days in the field years ago that tripped me up. MIBs that are keyed, for example, transaction rate and mean transaction times are keyed by the service type and service name, these MIBs are dynamic.  Said another way, if you were to create a new service in your domain, the ordering of those keys may change the MIBs of status that you were actively using as the new service is placed before the service you were monitoring.  That change of MIBs would also not take place until the appliance is rebooted.  Just something to be aware of.

    Best Regards,
    Steve Linn



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------