AIX

 View Only
  • 1.  Strange init behavior on AIX

    Posted Wed May 19, 2021 04:50 AM
    Let me elaborate on this strange subject line :)

    I have created two symbolic links in /etc/rc.d/rc2.d directory that should start and stop my application during boot-up and shutdown respectively:

    #ls -l /etc/rc.d/rc2.d/*enigma
    lrwxrwxrwx 1 root system 19 May 19 10:13 /etc/rc.d/rc2.d/K30enigma -> ../init.d/enigma.sh
    lrwxrwxrwx 1 root system 19 May 19 09:42 /etc/rc.d/rc2.d/S30enigma -> ../init.d/enigma.sh
    #

    enigma.sh script is very simple:
    #!/bin/sh

    ##################################################
    # name: enigma.sh
    # purpose: start or stop the enigma agent
    ##################################################

    [ -f /apps/enigma/Start.sh ] && [ -f /apps/enigma/Stop.sh ] || exit 0

    case "$1" in
    start)
    sudo -u enigma -i /apps/enigma/Start.sh
    ;;
    stop)
    sudo -u enigma -i /apps/enigma/Stop.sh
    ;;
    *)
    echo "Usage: $0 (start | stop)"
    exit 1
    esac

    Start.sh and Stop.sh scripts currently only write text about starting and stopping an application in a log file:
    #cat /apps/enigma/Start.sh
    #!/bin/sh

    echo "enigma started at $(date)">>/var/log/enigma
    #cat /apps/enigma/Stop.sh
    #!/bin/sh

    echo "enigma stopped at $(date)">>/var/log/enigma
    #

    Why do I think that init behavior is strange?
    When I reboot a server, what I expected to see in the log file first was message about stopping the application before reboot and message about starting the application after reboot.
    What happens is that the application is stopped before reboot, then firstly stopped and then started after reboot!
    Log file clearly shows this:
    #tail /var/log/enigma
    enigma started at Tue May 18 14:57:15 CEST 2021
    enigma stopped at Tue May 18 15:01:06 CEST 2021
    enigma stopped at Tue May 18 15:03:05 CEST 2021
    enigma started at Tue May 18 15:03:06 CEST 2021
    enigma stopped at Tue May 18 15:07:40 CEST 2021
    enigma stopped at Tue May 18 15:09:42 CEST 2021
    enigma started at Tue May 18 15:09:43 CEST 2021
    enigma stopped at Wed May 19 09:54:28 CEST 2021   -before reboot (stopped via "K" symlink as expected)
    enigma stopped at Wed May 19 09:56:36 CEST 2021   -after reboot (why is it stopped after reboot?)
    enigma started at Wed May 19 09:56:38 CEST 2021     -after reboot (started after firstly being stopped)
    #

    Why is it firstly stopped after reboot and not only started?
    Is this expected (by design) behavior on AIX?

    If I delete the K30enigma symbolic link from /etc/rc.d/rc2.d directory, then the application only gets started after the reboot, not firstly stopped then started.
    The problem with this is that it does not get stopped before reboot.


    Regards,
    Romeo​​

    ------------------------------
    Romeo Mikulic
    ------------------------------


  • 2.  RE: Strange init behavior on AIX

    Posted Wed May 19, 2021 05:06 AM
    https://www.ibm.com/support/pages/starting-and-stopping-software-system-v-rc-directories

    Heading 5: Kill scripts may need to be put into a different runlevel than the Start script.

    Init interprets the KXX scripts in a runlevel as killing services that
    should not be running at that runlevel. At shutdown it executes all
    KXX scripts in all rcX.d directories.

    Perhaps you need to move your stop script to rc0.d.




  • 3.  RE: Strange init behavior on AIX

    Posted Wed May 19, 2021 05:47 AM
    You are right!

    rc0.d directory did not exist before:

    #ls -ld /etc/rc.d/rc*
    -r-xr--r-- 1 root system 1610 Jun 01 2015 /etc/rc.d/rc
    drwxr-xr-x 2 root system 4096 May 19 11:22 /etc/rc.d/rc2.d
    drwxr-xr-x 2 root system 256 Oct 10 2016 /etc/rc.d/rc3.d
    drwxr-xr-x 2 root system 256 Oct 10 2016 /etc/rc.d/rc4.d
    drwxr-xr-x 2 root system 256 Oct 10 2016 /etc/rc.d/rc5.d
    drwxr-xr-x 2 root system 256 Oct 10 2016 /etc/rc.d/rc6.d
    drwxr-xr-x 2 root system 256 Oct 10 2016 /etc/rc.d/rc7.d
    drwxr-xr-x 2 root system 256 Oct 10 2016 /etc/rc.d/rc8.d
    drwxr-xr-x 2 root system 256 Oct 10 2016 /etc/rc.d/rc9.d
    #

    After creating it and putting my "K" script in, I get the expected behavior: my application is stopped at reboot and started after.

    Log file shows this:
    #tail -6 /var/log/enigma
    enigma stopped at Wed May 19 09:54:28 CEST 2021
    enigma stopped at Wed May 19 09:56:36 CEST 2021
    enigma started at Wed May 19 09:56:38 CEST 2021
    enigma started at Wed May 19 10:03:58 CEST 2021
    enigma stopped at Wed May 19 11:22:24 CEST 2021  -stopped before reboot
    enigma started at Wed May 19 11:24:33 CEST 2021    -started after reboot
    #​

    At a quick glance, rc0.d directory does not exist by default on my AIX boxes (some AIX 6.1, mostly AIX 7.2).
    I am curious why it is not there...


    Thank you,

    ------------------------------
    Romeo Mikulic
    ------------------------------



  • 4.  RE: Strange init behavior on AIX

    Posted Thu May 20, 2021 10:31 AM
    That is odd that something in rc0.d was called, since it is not listed in inittab (by default); and the only entry in the initab that references runlevel 0, is the cons entry for the getty on the system console.  Does anyone know how this happened?  I do know that init monitors inittab for changes and will kill processes that no longer exist at the current runlevel (but a kill signal is different than a call to a shutdown script).  Also there is a race condition that happens in init when inittab is updated without proper locking -- this is why inittab should only be edited using chitab, mkitab, and rmitab.

    ------------------------------
    Edward Davignon
    ------------------------------



  • 5.  RE: Strange init behavior on AIX

    Posted Thu May 20, 2021 10:19 AM
    Russel,

    Would this apply to /etc/rc.d/rc2.d/Ksshd, as well, or perhaps the OpenSSH packages for IBM intended it to be restarted when entering full multi-user mode (i.e. run level 2 on AIX)?

    The article you referenced is very interesting.  Thanks for sharing it.

    ------------------------------
    Edward Davignon
    ------------------------------



  • 6.  RE: Strange init behavior on AIX

    Posted Thu May 20, 2021 10:49 AM
    On Thu, May 20, 2021 at 02:18:51PM +0000, Edward Davignon via IBM Community wrote:
    > Would this apply to /etc/rc.d/rc2.d/Ksshd, as well, or perhaps the
    > OpenSSH packages for IBM intended it to be restarted when entering
    > full multi-user mode (i.e. run level 2 on AIX)?

    It would apply to any script. As SSH is a lightweight service, and
    trying to stop it if it isn't running would be nearly instant. There
    is almost no cost to calling it more than once.

    It's OK that it's in rc2.d because during shutdown all the Kxx scripts
    are executed from all the rc*.d directories.

    In the example from the original poster he was curious why it behaves
    this way, and we have documentation that says that is the expected
    behavior. I have been curious if other SysV init OS's with rc.d behave
    that way.

    I think a well written stop script would exit gracefully if the
    application isn't running, so calling it twice shouldn't be a
    problem. As a result, I'd leave the Kxx scripts in the same rcX.d
    directory for clarity.

    I prefer to use /etc/rc.local with a line in inittab for startup
    customization, and /etc/rc.shutdown for shutdown customization. I
    always felt like /etc/rc.d/rc*.d are more for software installers and
    canned scripts from a vendor.




  • 7.  RE: Strange init behavior on AIX

    Posted Thu May 20, 2021 01:56 PM
    Interesting.  /etc/rc.d/rc disallows run levels 0 and 1, while /usr/sbin/shutdown processes each and every K* script in rc[0-9].d

    ------------------------------
    Edward Davignon
    ------------------------------