Pretty full of database errors, so not much.
An interesting turn of events on the other "sudo hang"
In that case, we found, in audit, some mkdir errors for /var/run.
Turns out the they had an invalid link in /var/run, so sudo could not write to this directory.
So I found that the files in /var/run/sudo are timestamp files that sudo creates to track when a user last authenticated. These files determine how long a user can run sudo commands without re-entering their password, based on the timestamp_timeout setting in /etc/sudoers.
I created a hang by linking to a file, then deleting it, then doing sudo su - as a non-passwordless sudo user.
So at this point, I think we just have a "fluke" that multiple sudo hangs came in from multiple customers in 24 hours :-)
@Bryan Lenherr it might be good for us to check auditing for some other clues. I see they don't have anything useful being tracked at this point, but I will update that case with some recommendations. It seems the ansible is a local user, so I'm not sure what the LDAP impact could/should be on sudo. We could try to rule that out by just changing SYSTEM=compat for ansible. I will also get a truss from them.
------------------------------
Jan Harris
AIX Development Support (Liaison to the AIX Toolbox for Open Source)
IBM (Contract)
Austin
------------------------------
Original Message:
Sent: Tue November 05, 2024 11:56 AM
From: José Pina Coelho
Subject: Is Sudo 1.9.16.0 in plan for Toolbox update
Well, that's one superfluous sudo... anything on the console alog ?
------------------------------
José Pina Coelho
IT Specialist at Kyndryl
Original Message:
Sent: Tue November 05, 2024 10:03 AM
From: Jan Harris
Subject: Is Sudo 1.9.16.0 in plan for Toolbox update
I did notice they have a sudo process in inittab
inittab:cdstart:2:once:sudo su - cduser -c /cdndm/ONDDBP1/ndm/bin/cdstart > /dev/console 2>&1
I am not wondering if that could be related. I'm testing some options there too...
------------------------------
Jan Harris
AIX Development Support (Liaison to the AIX Toolbox for Open Source)
IBM (Contract)
Austin
Original Message:
Sent: Tue November 05, 2024 08:44 AM
From: Jan Harris
Subject: Is Sudo 1.9.16.0 in plan for Toolbox update
Thanks for your input, José!
Good point - but they are not using bash in at least one I'm working on, bash is not installed (72)
In that case, user's shell is ksh. It is difficult to know so far which is the "original" process that causes the issue, since what I have is proctrees of presumably subsequent hangs after the first issue.
------------------------------
Jan Harris
AIX Development Support (Liaison to the AIX Toolbox for Open Source)
IBM (Contract)
Austin
Original Message:
Sent: Tue November 05, 2024 05:39 AM
From: José Pina Coelho
Subject: Is Sudo 1.9.16.0 in plan for Toolbox update
Hi Jan,
Maybe it's driven/triggered by a couple of old bash hanging problems? (I think around the bash 4.2-3 level)
If you're using version 5 of bash, you're probably not subject to the bash issues.
------------------------------
José Pina Coelho
IT Specialist at Kyndryl
Original Message:
Sent: Mon November 04, 2024 08:29 PM
From: Jan Harris
Subject: Is Sudo 1.9.16.0 in plan for Toolbox update
I have tried multiple methods to cause a hang
As userA:
sudo su -Then in root shell:TMOUT=1sleep 300 &tail -f /tmp/file &The root shell exits, the sudo processes are both killed as expected.# proctree $$5046728 /usr/sbin/srcmstr 19071300 /usr/sbin/sshd 14156076 sshd: userA [priv] 14549354 sshd: userA@pts/1 19267970 -bash 15466956 sudo su - 11927892 sudo su - 14483884 -ksh 13107568 tail -f /tmp/file 17301986 proctree 14483884
In another shell:
kill -STOP 14549354 kill -CONT 14549354
I tried -STOP for other processes as well. In all cases, I could not produce a hang.
------------------------------
Jan Harris
AIX Development Support (Liaison to the AIX Toolbox for Open Source)
IBM (Contract)
Austin
Original Message:
Sent: Thu October 31, 2024 04:27 PM
From: Jan Harris
Subject: Is Sudo 1.9.16.0 in plan for Toolbox update
Bryan, do you have any good re-creation scenarios? I have been trying to reproduce this in my 73 LPAR with sudo-1.9.15p5-1.ppc, I have killed the parent ssh process, the parent shell, I have tried kill -STOP, then kill -CONT - I have tried everything but water boarding my shell :-> I not been able to reproduce any sudo hangs.
------------------------------
Jan Harris
AIX Development Support (Liaison to the AIX Toolbox for Open Source)
IBM (Contract)
Austin
Original Message:
Sent: Thu October 31, 2024 02:31 PM
From: Bryan Lenherr
Subject: Is Sudo 1.9.16.0 in plan for Toolbox update
Jan,
Is there an ETA on when IBM will put the new version of sudo into the toolbox. We are also being impacted by the sudo hangs, but we don't like to use any rpms except ones that are part of the IBM toolbox.
Thanks,
Bryan Lenherr
Fiserv
------------------------------
Bryan Lenherr
Original Message:
Sent: Thu October 31, 2024 02:15 PM
From: Jan Harris
Subject: Is Sudo 1.9.16.0 in plan for Toolbox update
Hello, Toolbox team!
We have seen a couple of cases reporting some sudo hangs, and have some suspicion this might be related to some fixes in 1.9.16
For the meantime, I have advised them to test the latest sudo bff to see if this resolves the hang. If it does resolve the hang, I will point to this thread for them to follow.
A summary of related changes for sudo 1.9.16.0 :
https://github.com/sudo-project/sudo/commit/41978a56e042977923c1a55191b887218c536145
2024-04-27 Todd C. Miller
- src/exec_monitor.c, src/exec_pty.c, src/sudo.h:
- If user's tty goes away, tell monitor to revoke the tty in its session.
- Previously, we would simply close the pty leader in the main sudo process. This had the effect of revoking the pty, but the foreground process would not necessarily receive SIGHUP. By using TIOCNOTTY in the monitor, the running command has a better chance of getting SIGHUP. Once the monitor has revoked the pty, the main sudo process will close the pty leader, invalidating the pty. GitHub issue #367. [3d5708b425df]
------------------------------
Jan Harris
AIX Development Support (Liaison to the AIX Toolbox for Open Source)
IBM (Contract)
Austin
------------------------------