AIX Open Source

 View Only
Expand all | Collapse all

Error during installing xgboost

  • 1.  Error during installing xgboost

    Posted Mon September 18, 2023 10:23 AM

    Hi Ranjan,

    This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

    Thanks

    Mi

            ..........

            File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
              _build_ext.build_ext.build_extensions(self)
            File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
              self._build_extensions_serial()
            File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
              self.build_extension(ext)
            File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
              self.build_cmake_extension()
            File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
              os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
            File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
              return name
          UnboundLocalError: local variable 'name' referenced before assignment
          [end of output]



    ------------------------------
    Mi Wang
    ------------------------------


  • 2.  RE: Error during installing xgboost

    Posted Tue September 19, 2023 01:30 AM

    Hi Wang,

    This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

    Could you try the steps provided by Ayappan and let us know if it's working or not.

    Thanks



    ------------------------------
    Ranjit Ranjan
    ------------------------------



  • 3.  RE: Error during installing xgboost

    Posted Tue September 19, 2023 10:51 AM

    Hi Ranjit,

    Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

    1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

    2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

    mkdir build
    cd build
    export CC="gcc -maix64 -pthread"
    export CXX="g++ -maix64 -pthread"
    export OBJECT_MODE=64

    ............

    Thanks

    Mi



    ------------------------------
    Mi Wang
    ------------------------------



  • 4.  RE: Error during installing xgboost

    Posted Wed September 20, 2023 08:37 AM

    Okay, Wang.

    Make the changes for below file also .  you can take the change from previous thread.

    • setup.py,
    • xgboost/libpath.py,
    • xgboost/core.py

    To build and install, run the command 'python3 -m pip install . --root /usr/local'.  

    Before using it, export below two path.
    export LIBPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/xgboost/lib
    export PYTHONPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/

    This should work and it worked for me.  :) 



    ------------------------------
    Ranjit Ranjan
    ------------------------------



  • 5.  RE: Error during installing xgboost

    Posted Fri September 22, 2023 02:55 PM

    Hi Ranjit,

    As our Python 3.9 environment is temporarily not available for me, I have tried installing xgboost 1.6.2 on another Python 3.7 environment. I used the same steps from you and got the following error. Can you please help?

    Thanks

    Mi

      -- xgboost VERSION: 1.6.2
        -- Setting build type to 'Release' as none was specified.
        -- Looking for pthread.h
        -- Looking for pthread.h - not found
        CMake Error at /opt/freeware/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
          Could NOT find Threads (missing: Threads_FOUND)
        Call Stack (most recent call first):
          /opt/freeware/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
          /opt/freeware/share/cmake-3.22/Modules/FindThreads.cmake:238 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
          CMakeLists.txt:151 (find_package)
     
     
        -- Configuring incomplete, errors occurred!
        See also "/tmp/pip-req-build-4u3rb93v/build/temp.aix-7.2-3.7/CMakeFiles/CMakeOutput.log".
        See also "/tmp/pip-req-build-4u3rb93v/build/temp.aix-7.2-3.7/CMakeFiles/CMakeError.log".
        WARNING:XGBoost build_ext:Disabling OpenMP support.
        INFO:XGBoost build_ext:Run CMake command: ['cmake', 'xgboost', '-GUnix Makefiles', '-DUSE_OPENMP=0', '-DUSE_CUDA=0', '-DUSE_NCCL=0', '-DBUILD_WITH_SHARED_NCCL=0', '-DHIDE_CXX_SYMBOLS=1', '-DUSE_HDFS=0', '-DUSE_AZURE=0', '-DUSE_S3=0', '-DPLUGIN_DENSE_PARSER=0']
        -- CMake version 3.22.0
        -- xgboost VERSION: 1.6.2
        CMake Error at /opt/freeware/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):


    ------------------------------
    Mi Wang
    ------------------------------



  • 6.  RE: Error during installing xgboost

    Posted Mon September 25, 2023 02:49 AM

    what are the env exports used ?

    cmake is not able to find pthread.h that's why it's throwing above error. 

    Thanks

    Ranjit



    ------------------------------
    Ranjit Ranjan
    ------------------------------



  • 7.  RE: Error during installing xgboost

    Posted Mon September 25, 2023 10:09 AM

    Hi Ranjit,

    I use the following for defining env parameters. Should I also add export LIBPATH=/opt/freeware/lib64:$LIBPATH as well?

    export OBJECT_MODE=64
    export CC="gcc  -maix64 -pthread"
    export CXX="g++ -maix64 -pthread"

    Thanks

    Mi



    ------------------------------
    Mi Wang
    ------------------------------



  • 8.  RE: Error during installing xgboost

    Posted Mon September 25, 2023 10:35 AM

    By the way, I used the following command to install the package:

    python3 -m pip install . --root /

    Thanks

    Mi



    ------------------------------
    Mi Wang
    ------------------------------



  • 9.  RE: Error during installing xgboost

    Posted Mon September 25, 2023 01:56 PM
    Edited by Ranjit Ranjan Mon September 25, 2023 01:56 PM

    Above export env is okay.  No need to set any LIBPATH.

    I don't have python3.7 env to try it out. But it should work. 

    I Don't see any issue in python3.9. 

    Try freshly from start ,by extracting the source code of 1.6.2 , applying the suggested patches and run the same command for installation.

    you can add -v for verbose output 'python3 -m pip install . --root / -v' 

    Thanks

    Ranjit



    ------------------------------
    Ranjit Ranjan
    ------------------------------



  • 10.  RE: Error during installing xgboost

    Posted Mon September 25, 2023 05:21 PM

    Hi Ranjit,

    I just tried he installation of xgboost-1.7.5 on our Python 3.9 environment. I did the same environment variable exports. Please see following modifications to the 3 py scripts. The installation failed (see below). Please help.

    Thanks

    Mi

    Modification:

    /tmp/xgboost/xgboost-1.7.5: diff setup.py_orig setup.py
    46a47,48
    >     elif system() == 'AIX':
    >         name = 'libxgboost.a'

    /tmp/xgboost/xgboost-1.7.5/xgboost: diff libpath.py_orig libpath.py
    51a52,53
    >     elif sys.platform.startswith('aix'):
    >         dll_path = [os.path.join(p, 'libxgboost.a') for p in dll_path]

    /tmp/xgboost/xgboost-1.7.5/xgboost: diff core.py_orig core.py
    206c206
    <             lib = ctypes.cdll.LoadLibrary(lib_path)
    ---
    >             lib = ctypes.cdll.LoadLibrary(lib_path + '(libxgboost.so)')

    Error message:

     [ 96%] Building CXX object src/CMakeFiles/objxgboost.dir/__/rabit/src/engine.cc.o
          In file included from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:55,
                           from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/src/collective/socket.cc:4:
          /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h: In member function 'xgboost::collective::SockDomain xgboost::collective::TCPSocket::Domain() const':
          /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:294:41: error: 'SO_DOMAIN' was not declared in this scope; did you mean 'DOMAIN'?
            294 |         getsockopt(handle_, SOL_SOCKET, SO_DOMAIN, reinterpret_cast<char *>(&domain), &len), 0);
                |                                         ^~~~~~~~~
          /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/base.h:73:54: note: in definition of macro 'XGBOOST_EXPECT'
             73 | #define XGBOOST_EXPECT(cond, ret)  __builtin_expect((cond), (ret))
                |                                                      ^~~~
          /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:293:5: note: in expansion of macro 'xgboost_CHECK_SYS_CALL'
            293 |     xgboost_CHECK_SYS_CALL(
                |     ^~~~~~~~~~~~~~~~~~~~~~



    ------------------------------
    Mi Wang
    ------------------------------



  • 11.  RE: Error during installing xgboost

    Posted Tue September 26, 2023 12:24 AM

    For 1.7.5, you also need to apply the patch for socket.h . I think, you know the changes as you mentioned in the starting post message.

    Apply that change and try it out.



    ------------------------------
    Ranjit Ranjan
    ------------------------------



  • 12.  RE: Error during installing xgboost

    Posted Tue September 26, 2023 02:45 PM

    Hi Ranjit,

    After I mode the modification to the socket.h file, I am now getting below new error message. Please take a look.

    Thanks

    Mi

    INFO:XGBoost build_ext:Run CMake command: ['cmake', 'xgboost', '-GUnix Makefiles', '-DUSE_OPENMP=1', '-DUSE_CUDA=0', '-DUSE_NCCL=0', '-DBUILD_WITH_SHARED_NCCL=0', '-DHIDE_CXX_SYMBOLS=1', '-DUSE_HDFS=0', '-DUSE_AZURE=0', '-DUSE_S3=0', '-DPLUGIN_DENSE_PARSER=0']
      Could not load program cmake:
      Symbol resolution failed for cmake because:
            Symbol _ZTINSt6thread6_StateE (number 351) is not exported from dependent
              module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
            Symbol _ZTVNSt6thread6_StateE (number 352) is not exported from dependent
              module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
            Symbol _ZNSt6thread4joinEv (number 381) is not exported from dependent
              module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
            Symbol _ZNSt18condition_variableD1Ev (number 403) is not exported from dependent
              module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
            Symbol _ZNSt18condition_variableC1Ev (number 404) is not exported from dependent
              module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
            Symbol _ZNSt18condition_variable4waitERSt11unique_lockISt5mutexE (number 448) is not exported from dependent
              module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
            Symbol _ZNSt18condition_variable10notify_oneEv (number 449) is not exported from dependent
              module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
            Symbol _ZNSt18condition_variable10notify_allEv (number 450) is not exported from dependent
              module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
            Symbol _ZNSt6thread6_StateD2Ev (number 461) is not exported from dependent
              module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
            Symbol _ZNSt6thread15_M_start_threadESt10unique_ptrINS_6_StateESt14default_deleteIS1_EEPFvvE (number 462) is not exported from dependent
              module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
            Symbol _ZNSt6thread20hardware_concurrencyEv (number 463) is not exported from dependent
              module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
      Examine .loader section symbols with the 'dump -Tv' command.



    ------------------------------
    Mi Wang
    ------------------------------



  • 13.  RE: Error during installing xgboost

    Posted Wed September 27, 2023 02:36 AM

    Hi Wang,

    Looks ,  LIBPATH issue.

    are you able to run 'cmake' from the shell session? just type 'cmake' and enter'.

    are you setting any LIBPATH in your build session ? 

    Try without setting any LIBPATH.

    if you still see the above error message 'Could not load program cmake', then collect the output of below commands:

    • rpm -qa | grep -i cmake
    • ldd /opt/freeware/bin/cmake

    Thanks

    Ranjit



    ------------------------------
    Ranjit Ranjan
    ------------------------------



  • 14.  RE: Error during installing xgboost

    Posted Wed September 27, 2023 05:16 PM

    Hi Ranjit,

    I have followed your recommendation and the installation has got into normal 'building CXX...' stage. Then it hit with the error (see below for error messages) related to space. I have tried to increase the space for / and /opt filesystems (the /tmp has about 10 GB free space), but still got the same error. Can you please help to find out where this 'not enough space' complaint is for? 

    Thanks

    Mi

    ......
      [ 95%] Building CXX object src/CMakeFiles/objxgboost.dir/__/rabit/src/allreduce_base.cc.o
      [ 96%] Building CXX object src/CMakeFiles/objxgboost.dir/__/rabit/src/engine.cc.o
      /bin/sh[3]: 8389106 Killed
      /bin/sh[3]: 8126970 Killed
      /bin/sh[3]: 8782240 Killed
      /bin/sh[3]: 8454498 Killed
      /bin/sh[3]: 9109762 Killed
      /bin/sh[3]: 10813776 Killed
      g++: fatal error: Killed signal terminated program cc1plus
      compilation terminated.
      make: The signal code from the last command is 9.
     
      Stop.
      make: The error code from the last command is 2.

    ......

    g++: fatal error: Killed signal terminated program cc1plus
      compilation terminated.
      /bin/sh[3]: 11206954 Killed
      error: [Errno 12] Not enough space
      /bin/sh[3]: 11862358 Killed
      /bin/sh[3]: 13435344 Killed



    ------------------------------
    Mi Wang
    ------------------------------



  • 15.  RE: Error during installing xgboost

    Posted Thu September 28, 2023 03:11 AM

    Hi, 

    is the disk partition under which you extracted the source code , having the enough disk space ?

    check this also.

    once you hit this error, capture 'df -k' also.

    Thanks

    Ranjit



    ------------------------------
    Ranjit Ranjan
    ------------------------------



  • 16.  RE: Error during installing xgboost

    Posted Thu September 28, 2023 11:45 AM

    Hi Ranjit,

    That filesystem (for extracting the source code) has 74 GB free space. All other system related filesytems' space usage doesn't exceed 65 % except /usr, which has > 1GB free space. I am wondering could there be other type storage causing the error (I checked swap space and it's also fine)?

    Thanks

    Mi



    ------------------------------
    Mi Wang
    ------------------------------



  • 17.  RE: Error during installing xgboost

    Posted Thu September 28, 2023 05:21 PM

    Hi Ranjit,

    I just tried installing the xgboost using a different filesystem's storage for extracting the source code. I got exact the same result: the building process failed when building CXX objects reached 96 %. SO this is not related to the filesystem holding the building contents. It must be something else.

    Thanks

    Mi



    ------------------------------
    Mi Wang
    ------------------------------



  • 18.  RE: Error during installing xgboost

    Posted Fri September 29, 2023 01:45 AM

    Try increasing below ulimit values. 
    ulimit -d unlimited
    ulimit -m unlimited
    ulimit -n 20000



    ------------------------------
    Ayappan P
    ------------------------------



  • 19.  RE: Error during installing xgboost

    Posted Fri October 06, 2023 10:17 AM

    Hi Ranjit / Ayappan,

    After adding 3 GB space to paging space, I have successfully installed the Xgboost (v1.7.5) finally. Thanks for your help.

    Mi



    ------------------------------
    Mi Wang
    ------------------------------