DataPower

  • 1.  Best way for 2 MPGs in same domain to talk

    IBM Select
    Posted Wed May 05, 2021 05:15 PM
    MPG_A and MPG_B exist in the same domain.
    MPG_A needs to call MPG_B.
    This will likely be HTTP.

    What addressing scheme should MPG_A use when making the call to ensure the traffic stays on the appliance and does not go out onto the network only to turn around and come back to the same appliance?  If we create a Host Alias that reference the appliance's IP address would that guarantee a send to the Host Alias name stays on the appliance?

    What should the Front Side Handler in MPG_B do to ensure it only accepts traffic from inside the appliance? From only inside this Domain?
    Configuring the HTTP Front Side Handler to use an Access Control List with the IP address of the appliance seems like an option, but we would have to configure that uniquely on every appliance. Is there any way to make this work use Host Aliases? To further restrict it to the parent Domain?

    We want to stay on the appliance between MPG_A and MPG_B for performance reasons.

    Additionally if we know A to B stays on the appliance and B only accepts traffic from on the appliance we could consider not using TLS between A and B to save complexity, ongoing cert management and performance overhead for encryption. Yes, yes, yes, we already do all this all over the place for traffic arriving or leaving the appliance. But for use cases like this where we could ensure traffic stays on the appliance between 2 MPGs in the same domain, is it crazy to consider using HTTP and not HTTPs, just for this hop between MPG_A and MPG_B in the same Domain on the same appliance.

    ------------------------------
    Peter Potkay
    Windsor CT
    ------------------------------


  • 2.  RE: Best way for 2 MPGs in same domain to talk

    Posted Wed May 05, 2021 07:18 PM
    Create a host alias named localhost1 and set it to 127.0.0.1. Route the traffic to loclahost1.  Create a front side handler to listen on the localhost ip. Traffic never leaves the box and no external traffic will get to mpgw b. You can also create additional host aliases for 127.0.0.2 etc.  to isolate other mpgws internal traffic.

    ------------------------------
    Charlie Sumner
    ------------------------------



  • 3.  RE: Best way for 2 MPGs in same domain to talk

    Posted Wed May 05, 2021 07:24 PM
    And yes it's crazy to use https between two services in the same box.  And also I failed to mention... turn off persistent connections whenever chaining services like you are doing.

    ------------------------------
    Charlie Sumner
    ------------------------------



  • 4.  RE: Best way for 2 MPGs in same domain to talk

    Posted Thu May 06, 2021 02:39 AM
    Hi Peter,

    Charlie covered this pretty comprehensively with his description of using localhost-domain or localhost-x host aliases and giving each domain its own 127.0.0.x address for internal communications (and not using TLS, and disabling persistent connections). If you follow his advice and bind the front side handler to any localhost address, you don't need an ACL.

    He didn't address your question about interdomain communication though.

    Binding the front side handler for the second MPG will ensure that it can only be reached from a service (MPG or whatever) running on the same appliance. However, there is no way to control which domain on the appliance the source gateway is on. All domains have equal access to the network interfaces, including localhost (and the equivalent 127.0.0.x addresses). This is a weakness that has long been recognised, but I suspect is baked into the system architecture and cannot feasibly be "fixed". Similarly, there is no way to enforce that all front side handlers in a specific domain bind to a particular address, leaving other addresses to other domains. 

    The best that you can probably do to control this and ensure domain internal connections (and no cross domain connections) is to have strong governance and enforcement of standards, most probably using automation for deployment and deployment policies to enforce target address replacements using the current domain name as a component of the target host alias. It's trick if back end addresses are embedded in XML documents though. User Agent configuration to replace back end addresses might also help, although I haven't tried that myself.

    Regards,

    ------------------------------
    Neil Casey
    Senior Consultant
    Syntegrity Solutions
    Melbourne, Victoria
    IBM Champion (Cloud) 2019-21
    ------------------------------



  • 5.  RE: Best way for 2 MPGs in same domain to talk

    IBM Select
    Posted Sat May 08, 2021 09:24 AM
    Thank you Charlie and Neil. Good stuff. 

    I found this article that talks more about your advice not to use Persistent connections.

    https://www.ibm.com/support/pages/best-practices-configuring-local-chained-services-ibm-datapower-gateways

    Does the sending MPG use a Host Alias or a Static Host mapped to 127.0.0.1? You mentioned Host Alias (eg localhost1). Will the presence of the Host Alias prevent calls to the DNS servers looking for an IP for "localhost1"? I always assumed this was the purpose of Static Hosts.  I thought use Static Hosts for outbound (backside) calls and use Host Aliases for front side handlers.  Or do both Static Hosts and Host Aliases intercept and prevent calls to the DNS servers?

    Maybe easier / better to just hard code 127.0.0.1 for both the sending MPG and the receiving Front Side Handler? That would still be portable between environments, no?

    Regarding 127.0.0.2.  The developers raised a question with multiple logical environments on the same non Prod appliances. We put them in their own domains and would use secondary IPs on the appliance to allow the use of the same port # for the FSH. Does the same concept apply here?
    • MPG_A.1 sends to 127.0.0.1 and MPG_B.1 listens on 127.0.0.1 on port 7777 (for example)
    • MPG_A.2 sends to 127.0.0.2 and MPG_B.2 listens on 127.0.0.2 on port 7777 (same port by design)
    • MPG_A.3 sends to 127.0.0.3 and MPG_B.3 listens on 127.0.0.3 on port 7777 (same port by design)
    Will that work? All these 127.0.0.* are "just there" and ready to be used?

    Peter 

    I did thumb thru my copy of the IBM DataPower Handbook Volume II: DataPower Networking and very surprised to find nothing about all this 127.0.0.* stuff!

    ​​

    ------------------------------
    Peter Potkay
    ------------------------------



  • 6.  RE: Best way for 2 MPGs in same domain to talk

    Posted Sat May 08, 2021 12:46 PM
    >> I thought use Static Hosts for outbound (backside) calls and use Host Aliases for front side handlers.  

    Yes, that's absolutely correct. Some use Host Aliases for both outbound and front side handlers, and although it does work, it's not a documented feature, and the outbound aspect may disappear at some future date. This is one of the things I check for when I perform a health check, and so I recommended not using Host Aliases for outbound addressing. That's what static hosts are for.

    >> Will that work? All these 127.0.0.* are "just there" and ready to be used? Yes, although I've never personally tried using anything above 127.0.0.3.

    >> I did thumb thru my copy of the IBM DataPower Handbook Volume II: DataPower Networking and very surprised to find nothing about all this 127.0.0.* stuff!

    Yea, I guess it's just a basic networking concept that is a given... it's documented in Wikipedia so it must be true :-)
    localhost - Wikipedia

    --Charlie



    ------------------------------
    Charlie Sumner
    ------------------------------



  • 7.  RE: Best way for 2 MPGs in same domain to talk

    Posted Sun May 09, 2021 07:16 PM

    Hi Peter,

    As Charlie said, 127.0.0.x just works. It's build into TCP/IP. 

    One major consideration of using multiple host addresses to address "localhost" like this is that in high throughput environments, ephemeral ports can become a limiting resource. When a socket is closed, TCP puts it into a "close wait" state for a while before releasing it so it can be reused. If you are allocating and releasing sockets fast enough you can run out. I have seen this happen on DataPower appliances with lots of chained gateways and high throughput. We resolved it by using different 127.0.0.x addresses for each gateway, as each address gets its own set of ephemeral ports.

    So your proposal to use the same port to represent the same service in three separate domains is exactly the right path in my view.

    On your question regarding accessing the address using host alias or static host, you are again correct. To be "pure" and follow best practice, your front side handlers should use a host alias (say "localhost-dev" or "localhost-test" where the suffix could be the domain name). These need to be customised on deployment because Host Aliases are shared by all domains (configured in default domain). The sending gateway back end address (or url-open or whatever) should reference either: host address, static host, or load balancer group. It works if you reference the host alias, but as you know and Charlie confirmed, it's not ideal.

    If you use a static host (or just the host address) for this, then the back end address has to be fixed up during deployment into each domain. This isn't too hard if it's references as a static back end, but can get tricky if it's inside an XML configuration file or through dynamic selection of the back end. There are lots of potential places where back end addresses can be configured.

    The advantage of using a load balancer group for this purpose is that it is configured in the application domain instead of in the default domain. You can therefore have your gateways have identical configuration in each domain, with the back end address being "target-gatewayname-lbg" or "target-domainname-lbg" or something like that. The configuration of the LBG in each domain will contain the correct address (127.0.0.x) for that specific domain so then you don't have to worry about modifying the references to the back end address in each environment. Even if you configured the LBG using deployment policy, there's only one policy needed for each domain, with one substitution, rather than the large potential number if using a static host name, and deployment policy can't handle changes to XML documents so then you would need two separate mechanisms to deal with the issue.

    You can either have one LBG per gateway, in which case the port number can be configured in the LBG too. Or you can have one LBG for each domain, and not include the port number or health checks on the LBG, but all gateways in the domain can use the same LBG, so there's less configuration to deal with.

    If you haven't used Load Balancer Groups before, remember that you have to include them in the XML Manager used by the gateway which references them. This is one of many good reasons to have different XML Managers configured for your different gateways.

    Regards,



    ------------------------------
    Neil Casey
    Senior Consultant
    Syntegrity Solutions
    IBM Champion (Cloud) 2019-21
    ------------------------------