DataPower

DataPower

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
  • 1.  Need help in extracting last modified date

    Posted Fri January 12, 2024 01:55 PM

    Hi All , 

    I have a use case in which I have to extract the last modified date for the log file created and to compare it with the current date and send email if the log has not been formed.Can you anyone please help if this can be done using CLI commands in datapower .We dont want to make the changes in existing rules in datapower .Only want to run script from external Linux server and connect it with datapower.

    ----------

    Thanks !

    Ruchi Ghiya

    -----------



    ------------------------------
    Ruchi Ghiya
    ------------------------------


  • 2.  RE: Need help in extracting last modified date

    Posted Fri January 12, 2024 03:23 PM
    Edited by Hermann Stamm-Wilbrandt Fri January 12, 2024 03:23 PM

    Hi,

    what is"the log file created", do you mean DataPower system log?

    > and to compare it with the current date and send email if the log has not been formed.
    > Can you anyone please help if this can be done using CLI commands in datapower.
    >
    Not with CLI, but you might use XML management to get the date of system log.

    > We dont want to make the changes in existing rules in datapower .Only want to run script from external Linux server and connect it with datapower.
    >
    Do the timestamp comparison and email sending on Linux, based on the date you get returned from xml management command.



    ------------------------------
    Hermann Stamm-Wilbrandt
    Compiler Level 3 support, IBM DataPower Gateways
    IBM
    Boeblingen Germany
    ------------------------------



  • 3.  RE: Need help in extracting last modified date

    Posted Sun January 14, 2024 08:54 PM

    Hi Herman , 

    The logs are custom logs not the system logs which we generate in every seconds once the queries comes to datapower.What we need to make sure that in prod both the datapowers are receving the traffic .Simplest way to achieve this by identifying the log file in the logstore is updated with the current date and timestamp.

    We can run the code or command to see if the logs are updated and getting matched with the current timestamp .If the timestamp is not getting updated for an hour or two , we can send an alert that the datapower is not receiving any query traffic.



    ------------------------------
    Ruchi Ghiya
    ------------------------------



  • 4.  RE: Need help in extracting last modified date

    Posted Sat January 13, 2024 09:35 PM

    The CLI to get the last modified date of a file is somewhat like:

    co
    flash
    dir logstore:
    

    or "dir logtemp:" or whatever directory underneath. 

    To get the system time, just run either "show clock" or "show time":

    show time
    

    However, with that said, I'm not sure how you'd extract the last modified date from the list with DataPower CLI and compare it to the current date.

    So, as Hermann suggests, you should be able to connect with SSH from the Linux machine to DataPower, run the commands, extract the values and then send an email from the Linux server.  

    This is going to be interesting, though, as the log you are monitoring might be rotating very quickly, thus spamming the email box. 

    What log is this?  Sometime produced by a particular logging target subscribed to a set of event codes?



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 5.  RE: Need help in extracting last modified date

    Posted Sun January 14, 2024 08:58 PM

    Hi Joseph ,

    Yes you are correct that the logs are getting rotated every seconds thus we need to monitor in case the logs are not generated for anhour or two we can send out the alerts in the mail box that the datapowers are not receiving any traffic.

    These logs are generated on the basis of incoming qeeries.Usually the queries are coming round the clock in the datapowers

    Not sure how to check on the last modified date.

    Regards

    Ruchi



    ------------------------------
    Ruchi Ghiya
    ------------------------------



  • 6.  RE: Need help in extracting last modified date

    Posted Sun January 14, 2024 10:21 PM

    I gave you how to get the last modified date from the CLI above.  However, based upon your answer to Hermann, I disagree the easiest way to tell is by monitoring the log's last modified date.  This is because logging in DataPower is best-effort.  There is no guarantee you'll get all the logging you seek.

    What kind of service is it?

    Have you considered using something like a message count monitor and checking the message count status?  This will give you the number of transactions through the service in a number of time segments, one of which is by the hour.  If the number hasn't changed since the last check, then you'll know.

    An even better way is via an SLM Policy, because then, you can setup peers and monitor the traffic across a "cluster" of appliances via SLM status's total message metric.  



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 7.  RE: Need help in extracting last modified date

    Posted Mon January 15, 2024 11:16 PM

    Hi Joseph , 

    This is a service related to customers querying the transactions .I will check on the SLM setting if this can be facilitated in our requirement.I have checked on monitor but I suppose it will have some impact on the performance of the current service running.

    Fyi,In our existing service we need to maintain the SLA for the query customer produce.

    Regards

    Ruchi



    ------------------------------
    Ruchi Ghiya
    ------------------------------



  • 8.  RE: Need help in extracting last modified date

    Posted Tue January 16, 2024 09:42 AM

    I've never noticed a deleterious effect on performance using monitors.   Like you, I scratch my head for a moment and think it must, but it is so small as to not be reasonably measurable.

    Certainly, though, if you have an SLA with a customer, this is what the SLM Policy was designed to help you do.  It can be simple or very complex.  It is, though, more involved than the simpler Count Monitor.

    I do understand the other side of your problem, which is, is your service receiving requests at all.  That is, the various monitors and policies you can apply to DataPower generally are designed to monitor and potentially control the amount of traffic a service receives.  They generally don't care too much about lower thresholds, which is possibly an indicator of another kind of problem.

    If you aren't trying to control traffic flow, but just want to know if traffic is arriving, I'd seriously consider a count monitor.   As I stated, any performance hit these may produce are so minimal you likely won't notice.  It does report transactions per hour.  If that number were to drop to zero, then you'd know.    You can also set one up to count errors.  

    There are some other status providers you might consider using for this if the concern is just about number of transactions, though you'd have to filter on some things.   HTTP Connections status will tell you how many HTTP connections the appliance is making on behalf of an XML Manager.  It has several interesting metrics, such as "Created One Hour", etc.  However, if you use persistent connections (which you should) and the traffic flows consistently within the persistent connection timeout, this might lead you astray.

    Another is the Gateway Transactions Status.  This one doesn't give you a running count, per se, but provides very rich information on the URL on which traffic is received and its start time.  However, this information is fleeting, and you'd have to be lucky to capture one if the transaction count is low.

    I'll be honest with respect to monitoring for the "no traffic" situation.  I've always had other means to do it, like sending logs to an analytical tool like Splunk or DataDog.  Tools like that allow you to create alerts on just about anything, and low or no activity is one of them.



    ------------------------------
    Joseph Morgan
    ------------------------------