AIX

AIX

Connect with fellow AIX users and experts to gain knowledge, share insights, and solve problems.


#Power
 View Only

Socket Connections Locking and not Releasing

  • 1.  Socket Connections Locking and not Releasing

    Posted Fri November 30, 2007 04:31 PM

    Originally posted by: SystemAdmin


    We are having an issue where we have some socket connections from a Websphere instance on a AIX 5.3 box communicating to a TCP/IP app on an AIX 5.2.

    I really am not positive what to supply, but here are some symptoms that I have observed.

    netstat -an on 5.3 server:
    Proto Recv-Q Send-Q Local Address Foreign Address (state)
    tcp4 665 0 30.130.18.14.16000 30.130.20.31.49737 ESTABLISHED
    tcp4 663 0 30.130.18.14.16000 30.130.20.31.49738 ESTABLISHED
    tcp4 663 0 30.130.18.14.16000 30.130.20.31.49739 ESTABLISHED
    tcp4 663 0 30.130.18.14.16000 30.130.20.31.49741 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.31.49742 ESTABLISHED
    tcp4 663 0 30.130.18.14.16000 30.130.20.31.49763 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.31.49765 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.31.49788 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.31.56054 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.31.33856 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.31.39457 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.31.40065 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.32.59191 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.32.60574 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.32.60578 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.32.60585 ESTABLISHED
    tcp4 663 0 30.130.18.14.16000 30.130.20.32.60586 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.32.60587 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.32.60588 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.32.60589 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.32.60650 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.32.62848 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.32.64699 ESTABLISHED
    tcp4 663 0 30.130.18.14.16000 30.130.20.32.65148 ESTABLISHED
    tcp4 0 0 30.130.18.14.16000 30.130.20.32.41036 ESTABLISHED

    We never see the Recv-Q values change for the connections with Recv-Q > 0.

    On the Websphere server, there are unresponsive socket connections with numbers that correspond to the number of Recv-Q with value > 0.

    To fix, the Websphere instance is restarted. Here is a smal piece of the stack trace from Websphere:

    Thread Name : MessageListenerThreadPool : 2
    State : Runnable
    Java Stack
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java(Compiled Code))
    at java.io.DataInputStream.readFully(DataInputStream.java(Compiled Code))
    at java.io.DataInputStream.readInt(DataInputStream.java(Compiled Code))
    at madison.mpi.MpiNet.recvMsg(Unknown Source)

    I guess I would like to know why this would be occurring and if there is any configuration items that can release connections in AIX if they become stuck in this state. I am also following up with Websphere folks to see if anything can be done on their side to mitigate risks and trying to get the vendor application to provide some support also.

    Any help would be very appreciated.
    #AIX-Forum