DataPower

 View Only
Expand all | Collapse all

How can I get the list of domains and list of services in each domain through GatewayScript

  • 1.  How can I get the list of domains and list of services in each domain through GatewayScript

    Posted Tue March 12, 2024 12:51 PM

    How can I get the list of domains and list of services in each domain through GatewayScript?



    ------------------------------
    Soubhagyalaxmi Sahoo
    ------------------------------


  • 2.  RE: How can I get the list of domains and list of services in each domain through GatewayScript

    Posted Tue March 12, 2024 01:41 PM

    What is your use case? You are trying to mix administrative information with transactional processing. It's not very easy to do this. 



    ------------------------------
    Charlie Sumner
    ------------------------------



  • 3.  RE: How can I get the list of domains and list of services in each domain through GatewayScript

    Posted Wed March 13, 2024 07:06 AM

    Hi, Thanks for your response.

    I want to automate the disabled probe for the services available in all domains in a DataPower device.

    For this, I have configured the scheduled policy rule in XML Manager and provided the rule and action. For this, I want to write a gateway script code, which will fetch all the Domains and all the services in each Domain, if the probe is enabled for any service, I want to disable those probes.

    Appreciate your quick help on this.

    Thanks



    ------------------------------
    Soubhagyalaxmi Sahoo
    ------------------------------



  • 4.  RE: How can I get the list of domains and list of services in each domain through GatewayScript

    Posted Wed March 13, 2024 08:38 AM

    You really shouldn't be using the scheduled rule to perform regular management tasks on DataPower. It's best practice to use an external system to send a management payload and take an action if the response indicates a failure. You can use something like "Cron" to run a Python script on a regular interval.

    I suggest to read up on using the XML Management Interface (XMI) to query the domains, then loop through the domains and for each domain query the objects types such as Multiprotocol Gateways, WebService Proxies, or XML Firewalls. For each one of these objects,  send a payload which performs a modify-config command that turns DebugMode to off.

    It's really not a good practice for DataPower to be managing itself like you are suggesting.



    ------------------------------
    Charlie Sumner
    ------------------------------



  • 5.  RE: How can I get the list of domains and list of services in each domain through GatewayScript

    Posted Wed March 13, 2024 10:22 AM

    Thanks for your information.

    However, I want to understand how I can get the list of domains in the gateway script and the services list from each domain.

    Is there any possibility?

    Appreciate your help.

    Thanks



    ------------------------------
    Soubhagyalaxmi Sahoo
    ------------------------------



  • 6.  RE: How can I get the list of domains and list of services in each domain through GatewayScript

    Posted Wed March 13, 2024 12:03 PM
    Edited by Steve Linn Wed March 13, 2024 12:10 PM

    Hi Soubhagyalaxmi,

    I agree with Charlie that it is definitely not a best practice to mix management tasks with transactional tasks, and ultimately I don't believe what you want to accomplish, specifically to disable probes currently enabled,  is doable.  However, to answer your specific question about domain status, to my knowledge this is not possible in GatewayScript.   You'd could implement an xslt stylesheet and use a system status enumeration variable.  See https://www.ibm.com/docs/en/datapower-gateway/10.5.x?topic=variables-varservicesystemstatus.   For the list of domains, you'd need the domain status provider, ie, var://service/system/status/DomainStatus where you'll get an XML nodelist with 1..n XML elements that look like

    				<DomainStatus xmlns:env="http://www.w3.org/2003/05/soap-envelope">
    					<Domain>domain-name</Domain>
    					<SaveNeeded>on</SaveNeeded>
    					<TraceEnabled>off</TraceEnabled>
    					<DebugEnabled>on</DebugEnabled>
    					<ProbeEnabled>off</ProbeEnabled>
    					<DiagEnabled>off</DiagEnabled>
    					<CurrentCommand/>
    					<QuiesceState/>
    					<InterfaceState>ok</InterfaceState>
    					<FailsafeMode>none</FailsafeMode>
    					<QuarantineEnabled>off</QuarantineEnabled>
    				</DomainStatus>

    You could use a xsl:for-each to iterate over each one to get the domain name and you'll see if the probe is enabled via the DebugEnabled child element for each domain.  There is also in each domain an active services status provider (enumeration type ServicesStatus) which could be used to get the list of active services in that domain.  I've issued this caution on previous posts for these status enumeration service variable, but I'll repeat it here.  These status service variables are heavy weight operations and should never be used by normal transaction processing, ie, never executed under any type of load.  If simply queried once and a while in a processing rule that is either matching some type of infrequently inbound url or a scheduled rule that executes with a long interval, it shouldn't cause any performance issues.  Of course, an inbound transaction would still be an opening for a potential DoS attack that would flood the appliance with these request urls, so with that in mind, a schedule rule would be the only place I'd think they should be used.

    Having said all of that, disabling a probe is a completely different story.  First off, the domain status will show that a probe is enabled in the domain, but disablement of a probe is not done on a domain basis, but on a service (MPGW, XML Firewall, Web Service Proxy) basis, so now you'd also be need to find out what service within that domain has the probe enabled, and I don't see a status provider that shows that, again, you can only get a list of services but you won't know which one has the probe enabled.  Also, for the legacy probe, disabling the probe is a configuration change to the service which has side effects.  Any configuration change to a DataPower service will cause the service to temporarily go down and then be restarted.  This will result in a momentary loss of service for end users, with the potential for some in flight transactions to fail.  Another type of change you could make to the probe would be to specify a probe trigger to capture only specific transactions, but if that was done for an already enabled probe, that would also be a configuration change with the same issue. Thus, the probe should never be enabled, changed, or disabled while the service is receiving production traffic.  At best, the domain should be quiesced to allow all in flight requests to complete before the probe state is changed, and then the domain may be un-quiesced to allow traffic to flow back to it.  The complexity of doing all of this even before you even attempt to make a service configuration change programmatically is not something you should undertake.

    Finally, a chance to plug the new and enhanced probe.  The legacy probe, for all of the benefits it provides, is best used in development use cases and if used in production has the issues as described above and needs to be used with care.  The legacy probe also may modify the behavior of the running service in its collection of data (yet another reason to not use this in production), and there are some limitations of the data that the probe would collect and show depending upon the complexity of the service.  It also was specific to a individual service on an appliance, and since you most probably have multiple appliances servicing traffic using the same service, you either had to enable the probe on all appliances, or you had to somehow direct traffic you wanted to capture to a specific appliance IP where the probe was enabled instead of having that traffic go through a load balancer, or wait for a load balanced request to finally be sent to the appliance with the probe enabled.  All of these limitations were taken into consideration in the design of the new probe, which is not intrusive to the service and displays significantly more information than the legacy probe.  The captures are configured up front to only capture a specific number of requests, and when that is reached capturing stops. It also uses a peering instance so that the probe data is saved for all appliances and can be viewed / captured from any appliance in the load balanced group.  I suggest you check that out at https://www.ibm.com/docs/en/datapower-gateway/10.5.x?topic=troubleshooting-probe.

    Bottom line, although getting the list of domains and the services within the domain via an xslt stylesheet is possible, there is no mechanism to change the probe status dynamically within service code outside of using that service to create and send SOMA requests to itself to get configuration and modify configuration which I would strongly recommend against doing. I'd recommend a SOMA management request that would be sent externally to get the status of your domains that you can run periodically:

    <?xml version="1.0" encoding="UTF-8"?>
    <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
    <soapenv:Body>
      <dp:request xmlns:dp="http://www.datapower.com/schemas/management" domain="your-domain-name">
        <dp:get-status class="DomainStatus"/>
      </dp:request>
    </soapenv:Body>
    </soapenv:Envelope>

    and send that request with curl -kv -u admin:admin1 https://applianceIPorHostname:5550/service/mgmt/current --data-binary @soma_g
    et-domain-status.xml and if that domain is found to have a probe enabled, look at the service configurations in that domain to know the services that need this probe disabled, and then schedule a maintenance period where it can be disable. I also think that you should examine your access control procedures to the appliances to determine how the probe is being enabled in the first place.  Instead of attempting to "fix" this after the fact, you need to ensure the probe is never enabled in production except under very controlled circumstances, and when the appropriate data has been retrieved, to disable the probe before that controlled use case is considered complete.  Finally, examine the new probe to determine how enabling this probe may be useful, even in production.

    Regards,
    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 7.  RE: How can I get the list of domains and list of services in each domain through GatewayScript

    Posted Wed March 13, 2024 12:23 PM

    Let's say that you figure out how to obtain the list of domains in a GatewayScript.  Now that you have a list of domains, you probably were thinking to query each domain for the list of objects that might or might not have the probe enabled. Unfortunately, you cannot query or modify objects in other domains from a domain, even the default domain.

    How are you planning on turning off the probe in each of your application domains from the default domain?



    ------------------------------
    Charlie Sumner
    ------------------------------



  • 8.  RE: How can I get the list of domains and list of services in each domain through GatewayScript

    IBM Champion
    Posted Mon March 18, 2024 03:02 PM

    >> " Unfortunately, you cannot query or modify objects in other domains from a domain, even the default domain."

    This is not true.  You most definitely can.



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 9.  RE: How can I get the list of domains and list of services in each domain through GatewayScript

    Posted Mon March 18, 2024 09:40 PM

    Do tell! :-)



    ------------------------------
    Charlie Sumner
    ------------------------------



  • 10.  RE: How can I get the list of domains and list of services in each domain through GatewayScript

    IBM Champion
    Posted Mon March 18, 2024 09:58 PM
      |   view attached

    See the attached uploaded zip of a service that does display services having probes on and allows the user to turn them off.

    Upload into any domain of an appliance... 

    You may have to make minor changes to make it work for you, and then hit it with a browser.

    Have fun!



    ------------------------------
    Joseph Morgan
    ------------------------------

    Attachment(s)

    zip
    AutomationService.zip   804 KB 1 version


  • 11.  RE: How can I get the list of domains and list of services in each domain through GatewayScript

    Posted Tue March 19, 2024 12:07 PM

    Hi Joseph,
    I would have to believe you're using internal SOMA POSTs to https://localhost:5550/services/mgmt/current?  That's the only mechanism to get and modify configuration from a DataPower service, and I'd still say it is a management operation that isn't run on a service transaction basis that is under load for the same reasons expressed above.
    Regards,
    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 12.  RE: How can I get the list of domains and list of services in each domain through GatewayScript

    IBM Champion
    Posted Tue March 19, 2024 12:14 PM

    Yes, Yes and Agreed!  (But,,, Charlie challenged me.... I had to do something!   LOL)



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 13.  RE: How can I get the list of domains and list of services in each domain through GatewayScript

    IBM Champion
    Posted Tue March 19, 2024 12:18 PM

    Correction:  Not localhost, but to the appliance's XMI interface.



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 14.  RE: How can I get the list of domains and list of services in each domain through GatewayScript

    IBM Champion
    Posted Mon March 18, 2024 03:11 PM

    The problem with using a scheduled rule for this is as Charlie and Steve have mentioned.  It is altering the configuration of the service and, thus, causing it to restart.   Also, how are you controlling the time to ensure it is a safe time to disable?

    You should definitely consider an external script which a scheduler to run at a particular time.

    Commercially, MyArch's DPBuddy enables easier scripting to find and disable probes.   KumbaSoft's DPAA will allow you to easily find services where probes are enabled and then just turn them off as well.   I still wouldn't recommend either without due consideration of timing.



    ------------------------------
    Joseph Morgan
    ------------------------------



  • 15.  RE: How can I get the list of domains and list of services in each domain through GatewayScript

    Posted Wed March 13, 2024 11:45 AM

    You can use the SOAP/REST Interface provided by Datapower. As you have mentioned Gatewayscript, you can probably use the Rest  Management Interface of Datapower to get the information. You need to call the /mgmt/domains/config/ to retrieve the domains and then iterate over each domain to get the list of services.

    You can find more information on this in https://www.ibm.com/support/pages/part-1-introduction-rest-management-interface-and-status-monitoring .



    ------------------------------
    R K
    ------------------------------



  • 16.  RE: How can I get the list of domains and list of services in each domain through GatewayScript

    Posted Fri March 15, 2024 03:43 AM

    > How can I get the list of domains ...
    >
    That part is easy:

    'var://service/multistep/contexts'


    ------------------------------
    Hermann Stamm-Wilbrandt
    Compiler Level 3 support, IBM DataPower Gateways
    IBM
    Boeblingen Germany
    ------------------------------