Informix

Informix

Connect with Db2, Informix, Netezza, open source, and other data experts to gain value from your data, share insights, and solve problems.

 View Only

  • 1.  Connection manager hang issue

    Posted 03/13/23 02:29 PM

    Hi, Tom and  Gurus,

     

    We have a similar issue with CM hang as it described in the following message threads.   Our CSDK is 4.10.FC1.

     

    What are the tips, workaround or solutions to this issue?

     

    Thanks

    Frank

     

     

     

    Connection Manager Issue (Rampant Speculation Invited!)

    Follow 

    TOM GIRSCH

    TOM GIRSCHThu September 15, 2022 04:48 PM

    In which I describe a problem and invite rampant speculation. We have a CM (Connection Manager) with ...

    ·   1.  Connection Manager Issue (Rampant Speculation Invited!)

    0 Like

    Image

    IBM Champion

    TOM GIRSCH

    Actions Options Dropdown

    Posted Thu September 15, 2022 04:48 PM

    Mark As Inappropriate

    In which I describe a problem and invite rampant speculation.

    We have a CM (Connection Manager) with multiple redirect mode TLS SLAs where     one of those SLAs would routinely stop responding. Sessions would connect and redirect as expected for a brief period of time, then all subsequent connection attempts would hang. The other SLAs would continue responding as normal. Upon restarting the CM process, the port would briefly resume function but quickly start hanging again.

    I spent the better part of a day troubleshooting this, and thanks to some selective wireshark-ing, I found one client IP that seemed to have been responsible for the issue. Every time that particular client tried to connect to that port, the hang would recur. And it looks to have been some sort of auto-spawned process, because it would pop up in bursts no more than about four minutes apart. I blocked that specific IP at the CM machine's firewall, and the problem ceased. [The owner of the client machine claims ignorance, but that's another story.]

    I tell you all of that to tell you this: I have no idea how I'd reproduce that behavior even if I were _trying_ to. In all of our direct-to-that-port testing, there were three basic outcomes: the connection was rejected because of a protocol mismatch; the client rejects the connection (because of a TLS certificate mismatch); or the connection succeeds and is immediately redirected, at which point the connection is closed. In any case, it really seems to me that a DoS against a connection manager shouldn't be _that_ easy to pull off.

    So, two questions: Does this sound like a CM bug to any of you? Because it sort of seems that way to me. And second, can you think of a way to make a program connect to that port and hold onto it in a way that would block any new connections from coming in?

    Thanks in advance.

    Note: Before I blocked the IP, I was able to see this behavior with both 4.50.FC8 and 4.50.FC7.



    ------------------------------
    TOM GIRSCH
    ------------------------------

    #informix



    ·   2.  RE: Connection Manager Issue (Rampant Speculation Invited!)

    0 Like

    Image

    IBM Champion

    Paul Watson

    Actions Options Dropdown

    Posted Thu September 15, 2022 04:56 PM

    Mark As Inappropriate

    This was the basis of the program I wrote back in the dark ages to repo' the xctl functionality of XPS within a IDS environment. Might help

     

    Cheers

    Paul

     

     




    Original Message



    ·   3.  RE: Connection Manager Issue (Rampant Speculation Invited!)

    0 Like

    Image

    IBM Champion

    TOM GIRSCH

    Actions Options Dropdown

    Posted Fri September 16, 2022 11:27 AM
    Edited by System Fri January 20, 2023 04:15 PM

    Mark As Inappropriate

    Paul: Say what now?

    ------------------------------
    TOM GIRSCH
    ------------------------------

    Original Message



    ·   4.  RE: Connection Manager Issue (Rampant Speculation Invited!)

    0 Like

    Image

    IBM Champion

    Paul Watson

    Actions Options Dropdown

    Posted Fri September 16, 2022 11:49 AM

    Mark As Inappropriate

    Hmmm should have a link included

     

    https://stackoverflow.com/questions/307692/how-to-open-and-use-a-socket-in-c

     

    Cheers

    Paul

     




    Original Message



    ·   5.  RE: Connection Manager Issue (Rampant Speculation Invited!)

    0 Like

    Image

    IBM Champion

    Art Kagel

    Actions Options Dropdown

    Posted Thu September 15, 2022 05:29 PM

    Mark As Inappropriate

    Tom:

    I'm guessing an older ISQL or 4GL app from a version before 7.51 that doesn't understand the redirection and really needs a proxy port.

    Art

    ------------------------------
    Art S. Kagel, President and Principal Consultant
    ASK Database Management Corp.
    www.askdbmgt.com
    ------------------------------

    Original Message



    ·   6.  RE: Connection Manager Issue (Rampant Speculation Invited!)

    0 Like

    Image

    IBM Champion

    TOM GIRSCH

    Actions Options Dropdown

    Posted Thu September 15, 2022 07:15 PM

    Mark As Inappropriate

    Nope. It's a JDBC program of some sort. Those are the only ones we have that would connect on that port.

     

    Tom Girsch

    Lead System Architect
    Auto Europe Group
    tgirsch@autoeurope.com

    Image
    " If you think there is something more important than a Client ... think again "




    Original Message



    ·   7.  RE: Connection Manager Issue (Rampant Speculation Invited!)

    0 Like

    Image

    IBM Champion

    Andreas Legner

    Actions Options Dropdown

    Posted Fri September 16, 2022 03:14 AM

    Mark As Inappropriate

    Even JDBC needed to be recent enough to understand redirection - just saying.

    ------------------------------
    Andreas Legner
    ------------------------------

    Original Message



    ·   8.  RE: Connection Manager Issue (Rampant Speculation Invited!)

    0 Like

    Image

    IBM Champion

    Art Kagel

    Actions Options Dropdown

    Posted Fri September 16, 2022 07:04 AM

    Mark As Inappropriate

    Tom:

    I agree with Andreas. Check the version of the installed CSDK or iClient package on that host. If it's not 4.50 try upgrading it there.

    Art

    ------------------------------
    Art S. Kagel, President and Principal Consultant
    ASK Database Management Corp.
    www.askdbmgt.com
    ------------------------------

    Original Message



    ·   9.  RE: Connection Manager Issue (Rampant Speculation Invited!)

    0 Like

    Image

    IBM Champion

    TOM GIRSCH

    Actions Options Dropdown

    Posted Fri September 16, 2022 09:28 AM

    Mark As Inappropriate

    Our environment had been running basically forever without issue on 4.10.JC7, which is plenty new enough to understand CM redirection, but we recently upgraded nearly everything to 4.50.JC8.

    In a different environment, we had a very old program running a very old version of CSDK -- think circa 2002 -- that did NOT understand the redirect, but that just barfed instantly without holding the port open.



    ------------------------------
    TOM GIRSCH
    ------------------------------

    Original Message



    ·   10.  RE: Connection Manager Issue (Rampant Speculation Invited!)

    0 Like

    Image

    IBM Champion

    TOM GIRSCH

    Actions Options Dropdown

    Posted Fri September 16, 2022 09:35 AM

    Mark As Inappropriate

    The only two JDBC versions installed on the machine are 4.10.JC10 and 4.50.JC8

    ------------------------------
    TOM GIRSCH

     

     

     




  • 2.  RE: Connection manager hang issue

    Posted 03/15/23 04:48 AM

    HI,

    When this happens get 

    • a pstack of the connection manager process
    • a lsof of the connection manager process
    • netstat -an on the connection manager machine and get the state of the socket between the machines

    If on Linux also use the ss -n command

    • With -e to get the inode and match that to lsof to find the socket
    • With -T to show the thread for the socket match that to the thread from the pstack
    • WIth -i to get more information

    If you can let us know the stack for the connection manager thread and the state of the socket that will help.

    Also if you can do the same on the client side (if known) that would also help.

    Regards,
    David.



    ------------------------------
    David Williams
    ------------------------------



Related Content