Informix

 View Only
Expand all | Collapse all

KAIO: out of OS resources, errno = 11

  • 1.  KAIO: out of OS resources, errno = 11

    Posted Wed June 17, 2020 06:35 PM
    Edited by System Fri January 20, 2023 04:31 PM
    Hi,

    The following message continuously occurs in online.log and a service delay occurs.

    KAIO: out of OS resources, errno = 11, pid = 4784150

    Below is the onstat -g sql result.

    My customer is using Informix 12.10.FC6 on AIX, can setting the IFMX_AIXKAIO_NUM_REQ environment variable help solve the problem?
    And how can I check if it has been applied?

    ------------------------------
    SangGyu Jeong
    Software Engineer
    Infrasoft
    Seoul Korea, Republic of
    ------------------------------
    #Informix


  • 2.  RE: KAIO: out of OS resources, errno = 11

    Posted Thu June 18, 2020 02:26 AM
    Hi SangGyu,

    I think is a known issue on AIX
    and yes setting the env variable  IFMX_AIXKAIO_NUM_REQ  can help. Just have a look onto below URl

    https://www.ibm.com/support/pages/node/80163
    https://www.ibm.com/support/pages/node/662377

    ------------------------------
    Gaurav Kumar
    ------------------------------



  • 3.  RE: KAIO: out of OS resources, errno = 11

    Posted Thu June 18, 2020 02:41 AM
    Just to add, the maxium value for IFMX_AIXKAIO_NUM_REQ is 8192, it was increased from 4096 to 8192 in one of the older versions.

    I don't think there is a way to verify if the parameter is set.

    As far as I know, once this is enabled, the "KAIO : out of OS resources" does not appear anymore.

    They'll need to enable this env variable before starting oninit for it to be effective.

    Anyway the server has been up for 1427 days, it's time to rebounce :)


    ------------------------------
    Casey Tan
    ------------------------------



  • 4.  RE: KAIO: out of OS resources, errno = 11

    Posted Thu June 18, 2020 12:17 PM
    Hello Casey, 
    Thanks for your response. This is where my customers have the longest uptime.

    ------------------------------
    SangGyu Jeong
    Software Engineer
    Infrasoft
    Seoul Korea, Republic of
    ------------------------------



  • 5.  RE: KAIO: out of OS resources, errno = 11

    Posted Thu June 18, 2020 04:42 AM
    Use this command to view the environment of a running process, where in this case PID is an "oninit" process ID:

    ps eww PID | tr ' ' '\n' | grep = | sort

    Works on AIX and Linux. May need to be run as root.

    ------------------------------
    Doug Lawry
    Oninit Consulting
    ------------------------------



  • 6.  RE: KAIO: out of OS resources, errno = 11

    Posted Thu June 18, 2020 12:13 PM
    Thank you for the great tip. Doug.

    ------------------------------
    SangGyu Jeong
    Software Engineer
    Infrasoft
    Seoul Korea, Republic of
    ------------------------------



  • 7.  RE: KAIO: out of OS resources, errno = 11

    Posted Thu June 18, 2020 05:33 AM
    Hi,

    yes, setting IFMX_AIXKAIO_NUM_REQ to a higher value may resolve the problem ( see also https://www.ibm.com/support/pages/node/80163 ).
    (Obviously, doing so will require a shutdown and restart of the Informix server.)

    However, I think checking the AIX kernel parameter "maxreqs" should be done as well. If found being low, it may be necessary to (also) increase it.

    You should be able to see the environment of a running process (i.e. "oninit" process) with the command "ps -eww <pid>".

    Regards, Martin

    --
    Martin Fuerderer
    Informix Development Germany


    HCL Technologies Germany GmbH
    Frankfurter Ring 17
    80807 Munich, Germany
    http://www.hcltech.com/de
    --DISCLAIMER--
    ------------------------------------------------------------------------------------------------------------
    This document is intended for transmission to the named recipient only. If you are not that person, you should note that legal rights reside in this document and you are not authorized to access, read, disclose, copy, use or otherwise deal with it and any such actions are prohibited and may be unlawful. The views expressed in this document are not necessarily those of HCL Technologies Ltd. Notice is hereby given that no representation, contract or other binding obligation shall be created by this e-mail, which must be interpreted accordingly. Any representations, contractual rights or obligations shall be separately communicated in writing and signed in the original by a duly authorized officer of the relevant company.
    ------------------------------------------------------------------------------------------------------------

    ::DISCLAIMER::

    The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.






  • 8.  RE: KAIO: out of OS resources, errno = 11

    Posted Thu June 18, 2020 12:35 PM
    Thank you for answer. Martin.
    My customer is primarily considering fixing the application that is causing the problem or adjusting the application execution schedule.

    If the customer is unable to modify the application or the problem persists even if the application is modified, I will recommend the solutions presented here to the customer.

    ------------------------------
    SangGyu Jeong
    Software Engineer
    Infrasoft
    Seoul Korea, Republic of
    ------------------------------



  • 9.  RE: KAIO: out of OS resources, errno = 11

    Posted Thu June 18, 2020 12:13 PM
    Edited by System Fri January 20, 2023 04:48 PM
    Thank you all.

    After further checking, I noticed that while the error message occurred, the mutex wait and the buffer wait of the INSERT session were continued. (Thanks to InformixHQ)

    As shown in the screenshot, the time when the application that executes the INSERT statement in multiple sessions starts and the time when the error message occurs coincide. So I am guessing the problem is that the application causes excessive INSERTs at the same time. (About 300 sessions)

    In summary, the IFMX_AIXKAIO_NUM_REQ environment variable, maxreqs setting, application logic improvement, and lru queue tuning should also be considered.

    ------------------------------
    SangGyu Jeong
    Software Engineer
    Infrasoft
    Seoul Korea, Republic of
    ------------------------------



  • 10.  RE: KAIO: out of OS resources, errno = 11

    Posted Thu June 18, 2020 01:42 PM
    Hi,

    You need to set IFXM_AIXKAIO_NUM_REQ to tune your Kernel Asynchronous I/O (KAIO) subsystem.
    Its default value is 1024 if you does not set IFMX_AIXKAIO_NUM_REQ.

    Indeed, when you start your engine, Informix engine allocates a number of AIOCB structure (AIOCB = asynchronous I/O control block) specified by IFMX_AIXKAIO_NUM_REQ.
    The AIOCB is an Operating System structure used by Informix to manage several parameters of your asynchronous I/O requests (file descriptor, buffer location, file offset, length of your request ...) and to receive completion status information for asynchronous I/O requests.??
    Informix does not increase dynamically the number of AIOCB when all AIOCB blocks are used at the same time, informix must then wait until an AIOCB is free to be reused for a new asynchronous I/O request.
    The default value is not enough for your current asynchronous I/O requests activity.

    By example, you can set IFMX_AIXKAIO_NUM_REQ to 8192 or 16000 to tune your KAIO subsystem.

    Regards,

    --  Franck Thomas ConsultiX franck.thomas@consult-ix.fr http://www.consult-ix.fr T??l??phone : 33 (0) 1 39 12 18 00 Mobile    : 33 (0) 6 78 81 09 33 Fax       : 33 (0) 1 39 12 18 18  


    Le 18/06/2020 ?? 00:34, SangGyu Jeong via IBM Community a ??crit??:
    01000172c46b8b4a-a2f41c3a-d3d5-41e3-8823-43307932d0b2-000000@email.amazonses.com">
    Hi, The following message continuously occurs in online.log and a service delay occurs. KAIO: out of OS resources, errno = 11, pid = 4784150 No... -posted to the "Informix" group
    IBM Community

    Informix

    ?? Post New Message
    KAIO: out of OS resources, errno = 11
    Reply to Group Reply to Sender
    SangGyu Jeong
    Jun 17, 2020 6:35 PM
    SangGyu Jeong
    Hi,

    The following message continuously occurs in online.log and a service delay occurs.

    KAIO: out of OS resources, errno = 11, pid = 4784150

    No thread was running when the error message occurred. (onstat -g act)
    When do these messages occur?

    My customer is using Informix 12.10.FC6 on AIX, can setting the IFMX_AIXKAIO_NUM_REQ environment variable help solve the problem?
    And how can I check if it has been applied?

    ------------------------------
    SangGyu Jeong
    Software Engineer
    Infrasoft
    Seoul Korea, Republic of
    ------------------------------
    ?? Reply to Group Online ?? View Thread ?? Recommend ?? Forward ??



    ??
    You are subscribed to "Informix" as franck.thomas@consult-ix.fr. To change your subscriptions, go to My Subscriptions. To unsubscribe from this community discussion, go to Unsubscribe.
    Virtual UG Day Recap


      

    Garanti sans virus. www.avast.com





  • 11.  RE: KAIO: out of OS resources, errno = 11

    Posted Thu June 18, 2020 01:43 PM
    Hi,

    You need to set IFXM_AIXKAIO_NUM_REQ to tune your Kernel Asynchronous I/O (KAIO) subsystem.
    Its default value is 1024 if you does not set IFMX_AIXKAIO_NUM_REQ.

    Indeed, when you start your engine, Informix engine allocates a number of AIOCB structure (AIOCB = asynchronous I/O control block) specified by IFMX_AIXKAIO_NUM_REQ.
    The AIOCB is an Operating System structure used by Informix to manage several parameters of your asynchronous I/O requests (file descriptor, buffer location, file offset, length of your request ...) and to receive completion status information for asynchronous I/O requests.??
    Informix does not increase dynamically the number of AIOCB when all AIOCB blocks are used at the same time, informix must then wait until an AIOCB is free to be reused for a new asynchronous I/O request.
    The default value is not enough for your current asynchronous I/O requests activity.

    By example, you can set IFMX_AIXKAIO_NUM_REQ to 8192 or 16000 to tune your KAIO subsystem.

    Regards,

    --  Franck Thomas ConsultiX franck.thomas@consult-ix.fr http://www.consult-ix.fr T??l??phone : 33 (0) 1 39 12 18 00 Mobile    : 33 (0) 6 78 81 09 33 Fax       : 33 (0) 1 39 12 18 18  
      

    Garanti sans virus. www.avast.com