Informix

 View Only
Expand all | Collapse all

IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

  • 1.  IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    IBM Champion
    Posted Sat February 27, 2021 08:53 AM
    Before I open PMR, does anyone have this problem:
    Normal installation, DBSERVERALIAS for shm connection:
    DBSERVERNAME ifxpri
    DBSERVERALIASES lo_ifxpri,ifxpri_shm

    sqlhosts with shm record:
    ifxpri onsoctcp 192.168.193.209 ifxpri
    lo_ifxpri onsoctcp 127.0.0.1 lo_ifxpri
    ifxpri_shm onipcshm 127.0.0.1 ifxpri_shm

    Starting informix (oninit -v) ends with
    Initializing DR structures...succeeded
    Forking 1 'soctcp' listener threads...succeeded
    Forking 1 'soctcp' listener threads...succeeded
    Forking 1 'ipcshm' listener threads...informix@mate2004:~/i1410fc5/etc$

    And in online log I have this
    14:50:39 SQL_FEAT_CTRL value set to 0x8008
    14:50:39 SQL_FEAT_CTRL2 value set to 0x1010
    14:50:39 SQL_DEF_CTRL value set to 0x44004b0
    14:50:39 Assert Failed: shm_be.c, line 3415, thread 12, errno=13, could not open shm file.
    14:50:39 IBM Informix Dynamic Server Version 14.10.FC5
    14:50:39 Who: Session(6, informix@, 0, (nil))
    Thread(12, sm_listen, 0, 1)
    File: neterrb.c Line: 530
    14:50:39 stack trace for pid 888592 written to /home/informix/i1410fc5/tmp/af.3f44e2e
    14:50:39 See Also: /home/informix/i1410fc5/tmp/af.3f44e2e
    14:50:47 Thread ID 12 will NOT be suspended because
    it is deemed too critical to the server.
    14:50:47 See Also: /home/informix/i1410fc5/tmp/af.3f44e2e, shmem.3f44e2e.0
    14:50:48 Starting crash time check of:
    14:50:48 1. memory block headers
    14:50:48 2. stacks
    14:50:48 Crash time checking found no problems
    14:50:48 neterrb.c, line 530, thread 12, proc id 888592, shm_be.c, line 3415, thread 12, errno=13, could not open shm file..
    14:50:48 PANIC: Attempting to bring system down

    Without shm listener instance is starting as usual.
    Anyone?

    Hrvoje


    ------------------------------
    Hrvoje Zokovic
    ------------------------------

    #Informix


  • 2.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    Posted Sat February 27, 2021 10:28 AM

    Ask Art K ��

     

    Eric Vercelletto
    Data Management Architect and Owner / Begooden IT Consulting
    KandooERP Founder and CTO
    IBM Champion 2013,2014,2015,2016,2017,2018,2019,2020
    ibm-champion-rgb-130px

    Tel:     +33(0) 298 51 3210
    Mob : +33(0)626 52 50 68
    skype: begooden-it
    Google Hangout: eric.vercelletto@begooden-it.com
    Email:
    eric.vercelletto@begooden-it.com
    www :
    http://www.vercelletto.com
    www  https://kandooerp.org

     

     






  • 3.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    Posted Sat February 27, 2021 10:41 AM
    I haven't played with ubuntu in a while but I found this older document, 2017.


    Is it possible that the newer version of Ubuntu has it enabled by default? Peer speculation on my part here.

    David





  • 4.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    IBM Champion
    Posted Sat February 27, 2021 11:02 AM
    Thanks David but I believe it is not reason:
    root@mate2004:~# grep shh /etc/fstab
    root@mate2004:~#
    i.e. there's no shm in fstab.
    Hrvoje

    ------------------------------
    Hrvoje Zokovic
    ------------------------------



  • 5.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    Posted Sat February 27, 2021 11:53 AM
    Hrvoje,

    Your grep was for shh not shm? Was that what you meant?





  • 6.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    IBM Champion
    Posted Sat February 27, 2021 11:55 AM
    Yes, typo, but for shm it is the same:
    informix@ifx-tecaj:~/i1410$ grep shm /etc/fstab
    informix@ifx-tecaj:~/i1410$
    ​
    Hrvoje

    ------------------------------
    Hrvoje Zokovic
    ------------------------------



  • 7.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    Posted Sat February 27, 2021 11:58 AM
    Yeah saw your other email after I sent this one. Glad you got it resolved.





  • 8.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    IBM Champion
    Posted Sat February 27, 2021 10:54 AM
    Error 13 is a permission error....so, worth verifying ownership on $INFORMIXDIR, and that the oninit has the correct permissions and owner.

    I assume that the install was run as "root"?

    Maybe run onclean - make sure no leftover processes or memory segments?








    ------------------------------
    Mike Walker
    ------------------------------



  • 9.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    IBM Champion
    Posted Sat February 27, 2021 11:12 AM

    Thanks Mike, yes, normal root installation.
    ipcs shows nothing:

    informix@ifx-tecaj:~/i1410/bin$ ipcs
    
    ------ Message Queues --------
    key msqid owner perms used-bytes messages
    
    ------ Shared Memory Segments --------
    key shmid owner perms bytes nattch status
    
    ------ Semaphore Arrays --------
    key semid owner perms nsems
    
    informix@ifx-tecaj:~/i1410/bin$



    And permissions (on bin directory at least) seems OK to me:

    drwxr-xr-x 2 informix informix     4096 Feb 27 12:00 about_files
    -rw-r--r-- 1 informix informix      577 Nov 30 23:00 about.html
    -rwxr-xr-x 1 informix informix  3346896 Nov 30 23:22 archecker
    -rwxr-xr-x 1 informix informix  2400712 Nov 30 23:19 blademgr
    -rwxr-xr-x 1 informix informix  3495960 Nov 30 23:19 cdr
    -rwxr-xr-x 1 informix informix   897096 Nov 30 23:00 chkenv
    drwxr-xr-x 2 informix informix     4096 Feb 27 12:00 configuration
    -rwxr-xr-x 1 informix informix   897440 Nov 30 23:00 crtcmap
    -rwxr-xr-x 1 informix informix    61346 Nov 30 23:19 db2dbgm.jar
    -rwxr-xr-x 1 informix informix  2366624 Nov 30 23:19 dbaccess
    -rwxr-xr-x 2 informix informix     7847 Nov 30 23:19 dbaccessdemo
    -rwxr-xr-x 2 informix informix     7847 Nov 30 23:19 dbaccessdemo7
    -rwxr-xr-x 2 informix informix     6443 Nov 30 23:19 dbaccessdemo9
    -rwxr-xr-x 2 informix informix     6443 Nov 30 23:19 dbaccessdemo_ud
    -rwxr-xr-x 1 informix informix  2236536 Nov 30 23:19 dbexport
    -rwxr-xr-x 1 informix informix  1888592 Nov 30 23:19 dbimport
    -rwxr-xr-x 1 informix informix  1789008 Nov 30 23:19 dbload
    -rwxr-xr-x 1 informix informix  2235944 Nov 30 23:19 dbschema
    -rwxr-xr-x 1 informix informix  1985432 Nov 30 23:19 drdaprint
    -rwxr-xr-x 1 informix informix    56395 Nov 30 23:19 filtersym.sh
    -rwxr-xr-x 1 informix informix     5586 Nov 30 23:00 finderr
    -rwxr-xr-x 1 informix informix  1041512 Nov 30 23:00 GenMacKey
    -rwxr-xr-x 1 informix informix  1020480 Nov 30 23:00 genoncfg
    -rwxr-xr-x 1 informix informix     9721 Nov 30 23:00 glfiles
    -rwxr--r-- 1 informix informix     6357 Nov 30 23:19 hdrmkpri.sh
    -rwxr--r-- 1 informix informix     5178 Nov 30 23:19 hdrmksec.sh
    -rwxr-xr-- 1 informix informix  1373696 Nov 30 23:19 idsd
    -rwsr-s--- 1 root     informix     8272 Nov 30 23:19 ifmxgcore
    -r-xr-xr-x 2 informix informix    21183 Nov 30 23:19 ifmx_security.sh
    -rwxr-xr-x 1 informix informix  6276365 Nov 30 23:00 ifxbkpcloud.jar
    -rwxr-xr-x 1 informix informix  2643808 Nov 30 23:19 ifxclone
    -rwxr-xr-x 1 informix informix  1084960 Nov 30 23:19 ifxcollect
    -rwxr-xr-x 1 informix informix  1660736 Nov 30 23:00 ifxdeploy
    -rwxr-xr-x 1 informix informix    71162 Nov 30 23:00 ifxdeployassist
    -rwxr-xr-x 1 informix informix  1750944 Nov 30 23:19 ifxguard
    -rwxr-xr-x 1 informix informix    24392 Nov 30 23:21 ifxpipecat
    -rwxr-xr-x 1 informix informix   126904 Nov 30 23:21 infoshp
    -rwxr-xr-x 1 informix informix   892928 Nov 30 23:00 infxmsg
    -rwxr-xr-x 1 informix informix  4500024 Nov 30 23:19 ipload
    -rwxr-xr-x 1 informix informix  9999507 Nov 30 23:00 jsonListener.jar
    -rwxr-xr-x 1 informix informix   335360 Nov 30 23:00 libcairo-swt.so
    -rwxr-xr-x 1 informix informix   142464 Nov 30 23:21 loadshp
    -rwxr-xr-x 1 informix informix   892880 Nov 30 23:00 msgfile
    -rwsr-sr-x 1 root     informix  1810032 Nov 30 23:19 onaudit
    -rwxr-xr-x 1 informix informix     3998 Nov 30 23:22 onbar
    -rwsr-sr-x 1 root     informix  4774392 Nov 30 23:22 onbar_d
    -rwxr-sr-x 1 informix informix  3736488 Nov 30 23:19 oncheck
    -rwsr-sr-x 1 root     informix  1779288 Nov 30 23:19 onclean
    -rwxr-sr-x 1 informix informix  1335280 Nov 30 23:19 onconfig_diff
    -rwsr-sr-x 1 root     informix  3912680 Nov 30 23:22 ondblog
    -rwxr-xr-x 1 informix informix    11498 Nov 30 23:21 ondwachk
    -rwsr-sr-x 1 root     informix  2841664 Nov 30 23:19 onedcu
    -rwxr-sr-x 1 informix informix  1010904 Nov 30 23:19 onedpu
    -rwsr-sr-- 1 root     informix 27470872 Nov 30 23:19 oninit
    -rwxr-xr-x 1 informix informix   988048 Nov 30 23:00 onkstash
    -rwsr-sr-- 1 informix informix  1934536 Nov 30 23:19 onkstore
    -rwxr-sr-x 1 informix informix  2267576 Nov 30 23:19 onload
    -rwxr-sr-x 1 informix informix  4150648 Nov 30 23:19 onlog
    -rwsr-sr-x 1 root     informix  2762536 Nov 30 23:19 onmode
    -rwxr-sr-x 1 informix informix  3054096 Nov 30 23:19 onparams
    -rwxr-xr-x 1 informix informix  1012080 Nov 30 23:19 onpassword
    -rwxr-xr-x 1 informix informix  2594776 Nov 30 23:19 onperf
    -rwxr-xr-x 1 informix informix  2488528 Nov 30 23:19 onpladm
    -rwxr-sr-x 1 informix informix  4016128 Nov 30 23:19 onpload
    -rwxr-xr-x 1 informix informix  3043312 Nov 30 23:22 onpsm
    -rwxr-xr-x 1 informix informix  3342768 Nov 30 23:22 onrestorept
    -rwxr-xr-x 1 informix informix  1044664 Nov 30 23:19 onsecurity
    -rwsr-sr-x 1 root     informix  1092648 Nov 30 23:19 onshowaudit
    -rwxr-xr-- 1 informix informix     4096 Nov 30 23:19 onshutdown.sh
    -rwsr-sr-x 1 root     informix  4817496 Nov 30 23:22 onsmsync
    -rwsr-s--- 1 root     informix  4149896 Nov 30 23:21 onsnmp
    -rwxr-sr-x 1 informix informix  3053760 Nov 30 23:19 onspaces
    -rwsr-s--- 1 root     informix  1068280 Nov 30 23:21 onsrvapd
    -rwxr-sr-x 1 informix informix  8171568 Nov 30 23:19 onstat
    -rwsr-sr-x 1 root     informix  4214416 Nov 30 23:19 ontape
    -rwxr-sr-x 1 informix informix  2267584 Nov 30 23:19 onunload
    drwxr-xr-x 3 informix informix     4096 Feb 27 12:00 plugins
    -rwxr-xr-x 1 informix informix     4989 Nov 30 23:00 rofferr
    -rwxr-x--- 1 informix informix   614384 Nov 30 23:21 snmpdm
    -r-xr-xr-x 1 informix informix   939288 Nov 30 23:22 txbsapswd
    -rwxr-xr-x 1 informix informix   137112 Nov 30 23:21 unloadshp
    -rwxr-xr-x 1 informix informix  2694952 Nov 30 23:19 xtrace
    -rwxr-sr-x 1 informix informix  2844192 Nov 30 23:19 xtree
    


    Hrvoje



    ------------------------------
    Hrvoje Zokovic
    ------------------------------



  • 10.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    IBM Champion
    Posted Sat February 27, 2021 11:43 AM
    David, Mike, thank you - you pointed me to right google direction.
    It seems that you must
    sudo sysctl fs.protected_regular=0​

    Here's explanation (from https://github.com/sysrepo/sysrepo/issues/2080):
    This is not a Ubuntu-specific issue. It's caused by recent enough kernel (4.19+) and systemd (>= 241). The /dev/shm directory has the sticky bit set, this kernel patch added support for the fs.protected_regular sysctl knob, and this knob in turn gets enabled via systemd. As a result, one cannot use O_CREAT on "somebody else's files" on those world-writable sticky directories.

    That is NOT mentioned in documentation :)
    Regards,
    Hrvoje

    ------------------------------
    Hrvoje Zokovic
    ------------------------------



  • 11.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    Posted Mon March 01, 2021 04:42 AM

    I raised this internally end of last year ... (i.e. there is an internal defect)


    "Root cause is that the default kernel setting fs.protected_regular is 2

    On Ubuntu 20.04 the default setting is 2; setting this to 0 (not 1 or 2) and the instance starts successfully."

    Pending release notes / machine notes update.

    JJ



    ------------------------------
    Jon Ritson
    ------------------------------



  • 12.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    IBM Champion
    Posted Tue March 02, 2021 02:39 AM

    Hi JJ, yes I found that solution.
    The same problem is in on 14.10.FC4W1 as well (also for all v14 back to 14.10.fc1 but those were released before Apr 2020).
    V12 shm listener works fine (on ubuntu 20.04)

    What is best solution to have permanent fs.protected_regular=0 ?

    Regards
    Hrvoje



    ------------------------------
    Hrvoje Zokovic
    ------------------------------



  • 13.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    Posted Tue March 02, 2021 04:50 AM

    "Best" is subjective :)


    It works when fs.protected_regular=0


    For further "discussion" ...

    https://askubuntu.com/questions/1250974/user-root-cant-write-to-file-in-tmp-owned-by-someone-else-in-20-04-but-can-in



    ------------------------------
    Jon Ritson
    ------------------------------



  • 14.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    IBM Champion
    Posted Tue March 02, 2021 05:48 AM
    I'm afraid it is not working on 20.04:
    informix@mate2004:~/i1410fc5$ sudo sysctl fs.protected_regular
    [sudo] password for informix:
    fs.protected_regular = 2
    informix@mate2004:~/i1410fc5$ sudo sysctl fs.protected_regular=0
    fs.protected_regular = 0
    informix@mate2004:~/i1410fc5$ sudo sysctl fs.protected_regular
    fs.protected_regular = 0
    informix@mate2004:~/i1410fc5$ sudo reboot
    ...
    informix@mate2004:~$ sudo sysctl fs.protected_regular
    [sudo] password for informix:
    fs.protected_regular = 2
    informix@mate2004:~$

    Hrvoje

    ------------------------------
    Hrvoje Zokovic
    ------------------------------



  • 15.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    Posted Wed March 03, 2021 03:01 AM
    This is a problem of Ubuntu 20.04. After the kernel variables are set, kernel modules are still loaded which change settings again.
    The problem can be solved by calling a sysctl -p again in a later rc file, e.g. in rc.local if rc.local is active.

    You can read more about this here:
    https://bugs.launchpad.net/ubuntu/+source/procps/+bug/50093

    ------------------------------
    Andreas Seifert
    CURSOR Software AG
    http://www.admin-scout.com
    ------------------------------



  • 16.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    IBM Champion
    Posted Wed March 03, 2021 03:58 AM
    Hi Andreas,
    thank you for info and solution - I did that with systemd service
    Regards
    Hrvoje

    ------------------------------
    Hrvoje Zokovic
    ------------------------------



  • 17.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    Posted Wed March 03, 2021 07:18 AM
    The problem is a systemd problem. You can help yourself with rc.local or your own small systemd unit.

    ------------------------------
    Andreas Seifert
    CURSOR Software AG
    http://www.admin-scout.com
    ------------------------------



  • 18.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    IBM Champion
    Posted Mon March 01, 2021 06:22 PM

    13 is perms – something weird under SE Linux ?

     

    I have 14.10.FC5 running on Ubuntu.latest

     

    Cheers

    Paul

     






  • 19.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    IBM Champion
    Posted Tue March 02, 2021 02:05 AM
    Paul, SHM listener is problem. Do you have SHM listener started?

    Hrvoje

    ------------------------------
    Hrvoje Zokovic
    ------------------------------



  • 20.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    IBM Champion
    Posted Tue March 02, 2021 05:29 AM
    Does the service name you are using for the SHM connection (ifxpri_shm) exist in /etc/services? If so, try something completely dummy that doesn't like 'kagel' or 'dummy'. I have seen issues when the service name exists.



    ------------------------------
    Art S. Kagel, President and Principal Consultant
    ASK Database Management Corp.
    www.askdbmgt.com
    ------------------------------



  • 21.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    IBM Champion
    Posted Tue March 02, 2021 05:52 AM
    Art,
    there are only records added by installer in /etc/services:

    informix@mate2004:~$ grep ifxpri /etc/services
    ifxpri                   9088/tcp
    lo_ifxpri                3572/tcp
    ifxpri_json             31495/tcp #JSON listener for ifxpri
    informix@mate2004:~$

    Obvious it is kernel hardening repercussion.

    Hrvoje


    ------------------------------
    Hrvoje Zokovic
    ------------------------------



  • 22.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    IBM Champion
    Posted Tue March 02, 2021 03:46 PM

    This was sent a few days ago, no idea why IBM held it

     






  • 23.  RE: IDS 14.10.FC5 on Ubuntu 20.04 - can't start shm listener

    IBM Champion
    Posted Wed March 03, 2021 09:40 AM

    Replied 2021-03-02 14:47