Originally posted by: Tibor_B

Hi,

Our syslog shows unexplained ssh connection between our 2 servers, like:

Sep 9 03:16:09 serverX auth|security:info sshd[53739966]: Accepted publickey for oracle from 10.1.2.3 port 39066 ssh2

There is always a bunch of such connections every day early morning. Both servers hosts oracle databases, they have to communicate, but not via ssh service. Also our oracle admins claim there should be no ssh communication this way.

Connections are shortliving, in  fact I would say not successful because .profile requires user's response.

Both servers are production, so I cannot ban such connections or otherwise endanger their operations.

What can I do?

F.e. can I find what processes opened the ssh connections? (I repeat, they are shortliving connections)

Any ideas welcommed, thanks...

Originally posted by: AncientAIXer

There are several approaches you could take on the server that is initiating the connection.  The quickest thing I would do first is look at the crontabs, especially for the oracle user, to see what processes are being run.  You probably should look for scripts and then look at the scripts to see if they are using ssh.

If that doesn't reveal what's going on, you could configure the audit subsystem to report any PROC_Execute events.  This will reveal what username is running ssh and give you a trail to the ssh command.

The last resort I would use is to rename the ssh executable and use a wrapper script  to output the date and the user's environment variables to a log file and then execute the renamed ssh command with the command line options for the wrapper script.

I had thought about some of the network tracing tools such as iptrace or tcpdump, but I don't think they will give you the username or program/script name.

Originally posted by: Tibor_B

Hi,

I cannot find anything in scripts or crontabs. I believe this is something what oracle directly does, because  both databases cooperate somehow.

I am going to look at audit subsystem. I hope it will be able to report also parent (calling) process.

Originally posted by: Wouter Liefting

You mentioned that you actually have code in .profile that blocks the execution of .profile to wait for user input. In that case, it will not be a problem to do the following.

Temporary set up a password-less SSH keypair that allows you to log back in to the Oracle server as root, without ssh asking for a password. From the .profile, run a local netstat to figure out the ports that are used (or read that from the ssh log file), ssh back to the Oracle server, and use netstat and lsof/fuser/rmsock (plenty examples on the internet)  to figure out the PID and process name of the program that's using the port. Maybe throw in a ps -T 0 to record the process hierarchy.

Another thing you can do: Your logfile says "accepted pubkey for oracle". This means that the authentication was based on pubkeys, not on passwords, and this means there should be an authorized_keys or authorized_keys2 file in ~oracle/.ssh. You can look at the contents of this file, find the key itself or the key ID, and search for that key on the oracle server. Maybe it will tell you where the connection is from.

Originally posted by: Tibor_B

Hi Wouter Liefting,

I must say your solution is too complicated, f.e I dont like setting paswordless root ssh connection.

I just started auditing on source server and I will know more tomorrow. Both servers have oracle account with appropriate key, so the connection itself is no mystery, but I need to find out the parent process of outgoing ssh connection. I will see what audit log reveal....