AIX Open Source

 View Only
Expand all | Collapse all

Issue with C / C++ Compiling during the PIP installation for Xgboost

  • 1.  Issue with C / C++ Compiling during the PIP installation for Xgboost

    Posted Mon May 15, 2023 02:38 PM

    Hi,

    I got the following errors when trying to do pip install for Xgboost on python 3.7. Can AIX Toolbox experts help?

    1. "dmlc-core/include/dmlc/thread_local.h:76:8: error: 'mutex' in namespace 'std' does not name a type"
    Is this related to C++ setup? We already have libstdc++ and libstdc++10 installed.

    2. "rabit/include/rabit/./internal/../../dmlc/./logging.h:288:19: error: 'backtrace' was not declared in this scope"
    I see the following in base.sh to define the controlling variable to enable backtrace. Is it safe to comment out that portion? (considering backtrace is only for linux):
    "#if (!defined(DMLC_LOG_STACK_TRACE) \
         && defined(__GNUC__) && !defined(__MINGW32__) \
         && !defined(__sun) && !defined(__SVR4))
    #define DMLC_LOG_STACK_TRACE 1
    #endif
    "
    3. "make: cc: No such file or directory":
    I see in the general makefile the following environment settings (only for Darwin and Linux). Can you provide one for AIX so that can use it to replace following?
    ifeq ($(OS), Darwin)
    ifndef CC
    export CC = $(if $(shell which clang), clang, gcc)
    endif
    ifndef CXX
    export CXX = $(if $(shell which clang++), clang++, g++)
    endif
    else
    # linux defaults
    ifndef CC
    export CC = gcc
    endif
    ifndef CXX
    export CXX = g++
    endif
    endif

    Thanks

    Mi



    ------------------------------
    Mi Wang
    ------------------------------


  • 2.  RE: Issue with C / C++ Compiling during the PIP installation for Xgboost

    Posted Tue May 16, 2023 02:29 AM

    HI,

    What are the steps used for compiling ? Please share that. 
    Try setting below in env and then try.

    export OBJECT_MODE=64

    export CC='gcc  -maix64 -pthread'

    export CXX='g++ -maix64 -pthread'


    Thanks
    Ranjit



    ------------------------------
    Ranjit Ranjan
    ------------------------------



  • 3.  RE: Issue with C / C++ Compiling during the PIP installation for Xgboost

    Posted Wed May 17, 2023 12:17 PM

    Hi Ranjit,

    I add the settings and redo the PIP installation. Now I got the following errors. It seem the CC variable can only be pointing to some Unix executable. Can you help?

       Makefile:31: MAKE [make] - checked OK
        'g++ -maix64 -pthread' -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude   -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
        'g++ -maix64 -pthread' -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude   -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
        'g++ -maix64 -pthread' -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude   -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -fPIC -fopenmp -MM -MT build/c_api/c_api_error.o src/c_api/c_api_error.cc >build/c_api/c_api_error.d
        'g++ -maix64 -pthread' -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude   -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -fPIC -fopenmp -MM -MT build/c_api/c_api.o src/c_api/c_api.cc >build/c_api/c_api.d
        /bin/sh: g++ -maix64 -pthread:  not found
        /bin/sh: g++ -maix64 -pthread:  not found
        make: *** [Makefile:147: build/learner.o] Error 127
    ..........

    Thanks

    Mi



    ------------------------------
    Mi Wang
    ------------------------------



  • 4.  RE: Issue with C / C++ Compiling during the PIP installation for Xgboost

    Posted Thu May 18, 2023 12:57 PM

    Hi,

    • How did you run above export commands . 

              Did you run above export commands in shell session from where you are trying to compile   Or some other way ?

              You can run these from the shell session and verify using the echo command like below. It should work. 

    export OBJECT_MODE=64
    export CC="gcc  -maix64 -pthread"
    export CXX="g++ -maix64 -pthread"

    # echo $CC
    gcc -maix64 -pthread

    •  I thought to try and compile c++ shared library , so I used xgboost version 1.7.5.  I see below compilation failure. To fix this I made some code changes also . 

    /home/ranjit/xgboost/include/xgboost/collective/socket.h:294:41: error: 'SO_DOMAIN' was not declared in this scope; did you mean 'DOMAIN'?

      294 |         getsockopt(handle_, SOL_SOCKET, SO_DOMAIN, reinterpret_cast<char *>(&domain), &len), 0);

          |                                         ^~~~~~~~~

    /home/ranjit/xgboost/include/xgboost/base.h:73:54: note: in definition of macro 'XGBOOST_EXPECT'

       73 | #define XGBOOST_EXPECT(cond, ret)  __builtin_expect((cond), (ret))

          |                                                      ^~~~

    /home/ranjit/xgboost/include/xgboost/collective/socket.h:293:5: note: in expansion of macro 'xgboost_CHECK_SYS_CALL'

      293 |     xgboost_CHECK_SYS_CALL(

          |     ^~~~~~~~~~~~~~~~~~~~~~

    make[2]: *** [src/CMakeFiles/objxgboost.dir/build.make:118: src/CMakeFiles/objxgboost.dir/collective/socket.cc.o] Error 1

    make[1]: *** [CMakeFiles/Makefile2:264: src/CMakeFiles/objxgboost.dir/all] Error 2

    make: *** [Makefile:156: all] Error 2

     

    • To fix above compile error ,  I have modified include/xgboost/collective/socket.h. Below is git diff of this file.

    # git diff include/xgboost/collective/socket.h

    diff --git a/include/xgboost/collective/socket.h b/include/xgboost/collective/socket.h

    index 33d14fe8..4f9dc565 100644

    --- a/include/xgboost/collective/socket.h

    +++ b/include/xgboost/collective/socket.h

    @@ -287,7 +287,7 @@ class TCPSocket {

     #elif defined(__APPLE__)

         return domain_;

     #elif defined(__unix__)

    -#ifndef __PASE__

    +#if defined(__PASE__) && !defined(AIX)

         std::int32_t domain;

         socklen_t len = sizeof(domain);

         xgboost_CHECK_SYS_CALL

     

    Hope, you understand  above change and make out that which line is removed and which is added.  

    After this changes I am able to build shared object and executable .

     

    [ 93%] Building CXX object src/CMakeFiles/objxgboost.dir/__/rabit/src/allreduce_base.cc.o

    [ 95%] Building CXX object src/CMakeFiles/objxgboost.dir/__/rabit/src/rabit_c_api.cc.o

    [ 96%] Building CXX object src/CMakeFiles/objxgboost.dir/__/rabit/src/engine.cc.o

    [ 96%] Built target objxgboost

    [ 97%] Building CXX object CMakeFiles/runxgboost.dir/src/cli_main.cc.o

    [ 98%] Linking CXX executable xgboost

    [ 98%] Built target runxgboost

    [100%] Linking CXX shared library lib/libxgboost

    [100%] Built target xgboost

     

    Let me know if you need any info regarding above.


    Thanks
    Ranjit



    ------------------------------
    Ranjit Ranjan
    ------------------------------



  • 5.  RE: Issue with C / C++ Compiling during the PIP installation for Xgboost

    Posted Thu May 18, 2023 02:50 PM

    Hi Ranjit,

    1. I added the 3 export commands in the file xgboost/Makefile. I didn't run them in interactive session. Maybe I should run them in the shell session instead?
    2. I tried xgboost on an old version 0.82 on python 3.7 (AIX Toolbox one) by using python's pip tool. I am not which version Python you have tried this xgboost 1.7.5? Will it work on python 3.7 (I tried this before and got some strange errors)?

    Thanks

    Mi



    ------------------------------
    Mi Wang
    ------------------------------



  • 6.  RE: Issue with C / C++ Compiling during the PIP installation for Xgboost

    Posted Mon May 22, 2023 03:28 AM

    Hi

    As per https://pypi.org/project/xgboost/1.6.2/ , this version requirement is python 3.7. Try this version once and let us know. 



    ------------------------------
    Ranjit Ranjan
    ------------------------------



  • 7.  RE: Issue with C / C++ Compiling during the PIP installation for Xgboost

    Posted Tue May 23, 2023 02:59 PM

    Hi Ranjit,

    I have tried to install xgboost 1.6.2 on python 3.7 and got the following error. Is there a way to get around this 'convert_2to3_doctests' attribute?

    Thanks

    Mi

    copying xgboost/CMakeLists.txt -> build/lib.aix-7.1-3.7/xgboost
      Traceback (most recent call last):
        File "<string>", line 1, in <module>
        File "/tmp/pip-install-76li2fpj/xgboost/setup.py", line 370, in <module>
          url='https://github.com/dmlc/xgboost')
        File "/opt/freeware/lib/python3.7/site-packages/setuptools/__init__.py", line 144, in setup
          return distutils.core.setup(**attrs)
        File "/opt/freeware/lib64/python3.7/distutils/core.py", line 148, in setup
          dist.run_commands()
        File "/opt/freeware/lib64/python3.7/distutils/dist.py", line 966, in run_commands
          self.run_command(cmd)
        File "/opt/freeware/lib64/python3.7/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/opt/freeware/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 299, in run
          self.run_command('build')
        File "/opt/freeware/lib64/python3.7/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/opt/freeware/lib64/python3.7/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/opt/freeware/lib64/python3.7/distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
        File "/opt/freeware/lib64/python3.7/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/opt/freeware/lib64/python3.7/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/opt/freeware/lib/python3.7/site-packages/setuptools/command/build_py.py", line 58, in run
          self.build_package_data()
        File "/opt/freeware/lib/python3.7/site-packages/setuptools/command/build_py.py", line 132, in build_package_data
          srcfile in self.distribution.convert_2to3_doctests):
      AttributeError: 'Distribution' object has no attribute 'convert_2to3_doctests'
      ----------------------------------------
      ERROR: Failed building wheel for xgboost
      Running setup.py clean for xgboost
    Failed to build xgboost



    ------------------------------
    Mi Wang
    ------------------------------



  • 8.  RE: Issue with C / C++ Compiling during the PIP installation for Xgboost

    Posted Wed May 24, 2023 10:49 AM

    Hi Wang,

    Suspecting  some python configuration issue in your system .  
    I got below error while running 'pip3 install xgboost==1.6.2'


        File "/opt/freeware/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
          self.build_extensions()
        File "/opt/freeware/lib64/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-qjq6vaku/xgboost_b8e68e1351b843aab1cda2c4f932547c/setup.py", line 203, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-qjq6vaku/xgboost_b8e68e1351b843aab1cda2c4f932547c/setup.py", line 153, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-qjq6vaku/xgboost_b8e68e1351b843aab1cda2c4f932547c/setup.py", line 48, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

    Could you try from other system and confirm ? 



    ------------------------------
    Ranjit Ranjan
    ------------------------------



  • 9.  RE: Issue with C / C++ Compiling during the PIP installation for Xgboost

    Posted Wed May 24, 2023 05:23 PM

    Hi Ranjit,

    I see following in the build_py.py. Maybe the setuptools's version is too old in my system (it's 59.2.0). What's the setuptools version in your system? Is your setuptools package also from PYPI?

    try:
        from setuptools.lib2to3_ex import Mixin2to3
    except ImportError:

        class Mixin2to3:
            def run_2to3(self, files, doctests=True):
                "do nothing"

    Thanks

    Mi



    ------------------------------
    Mi Wang
    ------------------------------



  • 10.  RE: Issue with C / C++ Compiling during the PIP installation for Xgboost

    Posted Fri May 26, 2023 07:41 AM

    Hi Wang,
    Looks like something is wrong with your setuptools. 
    You can remove the existing setuptools and install it freshly. 
    rm -r /opt/freeware/lib/python3.7/site-packages/setuptools
    rm -r /opt/freeware/lib/python3.7/site-packages/setuptools*dist-info

    python3 -m ensurepip
    python3 -m pip install --upgrade setuptools

    Then you can try installing xgboost. 



    ------------------------------
    Ayappan P
    ------------------------------



  • 11.  RE: Issue with C / C++ Compiling during the PIP installation for Xgboost

    Posted Fri May 26, 2023 04:11 PM

    Hi Ranjit / Ayappan,

    After I reinstalled the default setuptools (47.1.0), now I got the similar error when installing xgboost as you did. Please help to resolve the issue:

      File "/opt/freeware/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 87, in run
            _build_ext.run(self)
          File "/opt/freeware/lib64/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
            _build_ext.build_ext.run(self)
          File "/opt/freeware/lib64/python3.7/distutils/command/build_ext.py", line 340, in run
            self.build_extensions()
          File "/opt/freeware/lib64/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
            _build_ext.build_ext.build_extensions(self)
          File "/opt/freeware/lib64/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
            self._build_extensions_serial()
          File "/opt/freeware/lib64/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
            self.build_extension(ext)
          File "/tmp/pip-install-jto30iwh/xgboost/setup.py", line 203, in build_extension
            self.build_cmake_extension()
          File "/tmp/pip-install-jto30iwh/xgboost/setup.py", line 153, in build_cmake_extension
            os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
          File "/tmp/pip-install-jto30iwh/xgboost/setup.py", line 48, in lib_name
            return name
        UnboundLocalError: local variable 'name' referenced before assignment

    Thanks

    Mi



    ------------------------------
    Mi Wang
    ------------------------------



  • 12.  RE: Issue with C / C++ Compiling during the PIP installation for Xgboost

    Posted Mon May 29, 2023 03:55 AM
    Edited by Ranjit Ranjan Mon May 29, 2023 03:58 AM

    Hi Wang,

    So direct pip installation will not work. We need to make some source code change to make it compatible with AIX.  
    Try below steps and see if it's working for you. 

    First download the source from below link
    https://files.pythonhosted.org/packages/25/e0/f00261d3357eafbe7c657de718abd3c0f6613633a38d05350f63ee099cde/xgboost-1.6.2.tar.gz

      1. Extract it : tar -xvf xgboost-1.6.2.tar.gz
      2. cd xgboost-1.6.2
      3. Export the env which I had shared previously.
      4. Run 'python3.7 setup.py install'. Same below error will be reported.

          self.build_cmake_extension()

        File "setup.py", line 153, in build_cmake_extension

          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))

        File "setup.py", line 48, in lib_name

          return name

      UnboundLocalError: local variable 'name' referenced before assignment.

       

      To fix this , we need to modify the source as explained. Make sure to use proper spacing for modifying below source file as these are python code and spacing matters. Use 4 spacing for modifying the code. 

      1. Take a backup of setup.py and modify setup.py with below changes.

      # diff -u setup.py.orig setup.py

      --- setup.py.orig       2023-05-29 01:40:36.295265022 +0000

      +++ setup.py    2023-05-29 01:43:15.045066128 +0000

      @@ -45,6 +45,8 @@

               name = 'libxgboost.dylib'

           elif system() == 'Windows':

               name = 'xgboost.dll'

      +    elif system() == 'AIX':

      +        name = 'libxgboost.a'

           return name

       

      1. Take a backup of xgboost/libpath.py  and modify  xgboost/libpath.py with below change

      (0) root @ pokndd3: /ranjit/xgboost-1.6.2/xgboost

      # diff -u libpath.py.orig libpath.py

      --- libpath.py.orig     2023-05-29 01:57:12.505062007 +0000

      +++ libpath.py  2023-05-29 02:16:33.611085204 +0000

      @@ -50,6 +50,8 @@

               dll_path = [os.path.join(p, 'libxgboost.dylib') for p in dll_path]

           elif sys.platform == 'cygwin':

               dll_path = [os.path.join(p, 'cygxgboost.dll') for p in dll_path]

      +    elif sys.platform.startswith('aix'):

      +        dll_path = [os.path.join(p, 'libxgboost.so') for p in dll_path]

           lib_path = [p for p in dll_path if os.path.exists(p) and os.path.isfile(p)]

      1. python3.7 setup.py install   (This should pass now)

      2. Execute below:

      cp /opt/freeware/lib64/python3.7/site-packages/xgboost/lib/libxgboost.a /opt/freeware/lib

      cd /opt/freeware/lib/

      ar -x libxgboost.a

       

      1. Now, xgboost can be imported.

      # python3

      Python 3.7.15 (default, Dec 15 2022, 03:08:16)

      [GCC 8.3.0] on aix7

      Type "help", "copyright", "credits" or "license" for more information.

      >>> import xgboost



      ------------------------------
      Ranjit Ranjan
      ------------------------------



    1. 13.  RE: Issue with C / C++ Compiling during the PIP installation for Xgboost

      Posted Mon May 29, 2023 03:47 PM

      Hi Ranjit,

      I have followed the steps you suggested. But got following errors when doing step 7: running 'python3.7 setup.py install'. Please help.

      Thanks

      Mi

      -- Check for working CXX compiler: /opt/freeware/bin/g++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /opt/freeware/bin/gcc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- CMake version 3.22.0
      -- Performing Test XGBOOST_MM_PREFETCH_PRESENT
      -- Performing Test XGBOOST_MM_PREFETCH_PRESENT - Failed
      -- Performing Test XGBOOST_BUILTIN_PREFETCH_PRESENT
      -- Performing Test XGBOOST_BUILTIN_PREFETCH_PRESENT - Success
      -- xgboost VERSION: 1.6.2
      -- Setting build type to 'Release' as none was specified.
      -- Looking for pthread.h
      -- Looking for pthread.h - not found
      CMake Error at /opt/freeware/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
        Could NOT find Threads (missing: Threads_FOUND)
      Call Stack (most recent call first):
        /opt/freeware/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
        /opt/freeware/share/cmake-3.22/Modules/FindThreads.cmake:238 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
        CMakeLists.txt:151 (find_package)


      -- Configuring incomplete, errors occurred!
      See also "/tmp/new-libs-1/xgboost-1.6.2/build/temp.aix-7.2-3.7/CMakeFiles/CMakeOutput.log".
      See also "/tmp/new-libs-1/xgboost-1.6.2/build/temp.aix-7.2-3.7/CMakeFiles/CMakeError.log".
      WARNING:XGBoost build_ext:Disabling OpenMP support.
      INFO:XGBoost build_ext:Run CMake command: ['cmake', 'xgboost', '-GUnix Makefiles', '-DUSE_OPENMP=0', '-DUSE_CUDA=0', '-DUSE_NCCL=0', '-DBUILD_WITH_SHARED_NCCL=0', '-DHIDE_CXX_SYMBOLS=1', '-DUSE_HDFS=0', '-DUSE_AZURE=0', '-DUSE_S3=0', '-DPLUGIN_DENSE_PARSER=0']
      -- CMake version 3.22.0
      -- xgboost VERSION: 1.6.2
      CMake Error at /opt/freeware/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
        Could NOT find Threads (missing: Threads_FOUND)
      Call Stack (most recent call first):
        /opt/freeware/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
        /opt/freeware/share/cmake-3.22/Modules/FindThreads.cmake:238 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
        CMakeLists.txt:151 (find_package)


      -- Configuring incomplete, errors occurred!
      See also "/tmp/new-libs-1/xgboost-1.6.2/build/temp.aix-7.2-3.7/CMakeFiles/CMakeOutput.log".
      See also "/tmp/new-libs-1/xgboost-1.6.2/build/temp.aix-7.2-3.7/CMakeFiles/CMakeError.log".
      Traceback (most recent call last):
        File "setup.py", line 197, in build_cmake_extension
          self.build(src_dir, build_dir, gen, build_tool, use_omp=1)
        File "setup.py", line 134, in build
          subprocess.check_call(cmake_cmd, cwd=build_dir)
        File "/opt/freeware/lib64/python3.7/subprocess.py", line 363, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['cmake', 'xgboost', '-GUnix Makefiles', '-DUSE_OPENMP=1', '-DUSE_CUDA=0', '-DUSE_NCCL=0', '-DBUILD_WITH_SHARED_NCCL=0', '-DHIDE_CXX_SYMBOLS=1', '-DUSE_HDFS=0', '-DUSE_AZURE=0', '-DUSE_S3=0', '-DPLUGIN_DENSE_PARSER=0']' returned non-zero exit status 1.

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "setup.py", line 372, in <module>
          url='https://github.com/dmlc/xgboost')
        File "/opt/freeware/lib/python3.7/site-packages/setuptools/__init__.py", line 144, in setup
          return distutils.core.setup(**attrs)
        File "/opt/freeware/lib64/python3.7/distutils/core.py", line 148, in setup
          dist.run_commands()
        File "/opt/freeware/lib64/python3.7/distutils/dist.py", line 966, in run_commands
          self.run_command(cmd)
        File "/opt/freeware/lib64/python3.7/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "setup.py", line 307, in run
          super().run()
        File "/opt/freeware/lib/python3.7/site-packages/setuptools/command/install.py", line 65, in run
          orig.install.run(self)
        File "/opt/freeware/lib64/python3.7/distutils/command/install.py", line 545, in run
          self.run_command('build')
        File "/opt/freeware/lib64/python3.7/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/opt/freeware/lib64/python3.7/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/opt/freeware/lib64/python3.7/distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
        File "/opt/freeware/lib64/python3.7/distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/opt/freeware/lib64/python3.7/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/opt/freeware/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 87, in run
          _build_ext.run(self)
        File "/opt/freeware/lib64/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
          _build_ext.build_ext.run(self)
        File "/opt/freeware/lib64/python3.7/distutils/command/build_ext.py", line 340, in run
          self.build_extensions()
        File "/opt/freeware/lib64/python3.7/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib64/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib64/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
          self.build_extension(ext)
        File "setup.py", line 205, in build_extension
          self.build_cmake_extension()
        File "setup.py", line 200, in build_cmake_extension
          self.build(src_dir, build_dir, gen, build_tool, use_omp=0)
        File "setup.py", line 134, in build
          subprocess.check_call(cmake_cmd, cwd=build_dir)
        File "/opt/freeware/lib64/python3.7/subprocess.py", line 363, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['cmake', 'xgboost', '-GUnix Makefiles', '-DUSE_OPENMP=0', '-DUSE_CUDA=0', '-DUSE_NCCL=0', '-DBUILD_WITH_SHARED_NCCL=0', '-DHIDE_CXX_SYMBOLS=1', '-DUSE_HDFS=0', '-DUSE_AZURE=0', '-DUSE_S3=0', '-DPLUGIN_DENSE_PARSER=0']' returned non-zero exit status 1.
      root [bastdctdv64] /tmp/new-libs-1/xgboost-1.6.2:

       



      ------------------------------
      Mi Wang
      ------------------------------



    2. 14.  RE: Issue with C / C++ Compiling during the PIP installation for Xgboost

      Posted Tue May 30, 2023 03:40 AM

      Can you try the below steps

      git clone https://github.com/dmlc/xgboost
      cd xgboost
      git checkout v1.6.2
      git submodule init
      git submodule update
      mkdir build
      cd build
      export CC="gcc -maix64 -pthread"
      export CXX="g++ -maix64 -pthread"
      export OBJECT_MODE=64
      cmake ..

      cd ../python-package

      Modify below three files

      # diff -u setup.py_orig setup.py
      --- setup.py_orig       2023-05-30 02:07:56.160027553 +0000
      +++ setup.py    2023-05-30 02:19:35.710912296 +0000
      @@ -45,6 +45,8 @@
               name = 'libxgboost.dylib'
           elif system() == 'Windows':
               name = 'xgboost.dll'
      +    elif system() == 'AIX':
      +        name = 'libxgboost.a'
           return name

      # diff -u xgboost/libpath.py_orig xgboost/libpath.py
      --- xgboost/libpath.py_orig     2023-05-30 02:07:56.190026737 +0000
      +++ xgboost/libpath.py  2023-05-30 02:21:00.651401005 +0000
      @@ -50,6 +50,8 @@
               dll_path = [os.path.join(p, 'libxgboost.dylib') for p in dll_path]
           elif sys.platform == 'cygwin':
               dll_path = [os.path.join(p, 'cygxgboost.dll') for p in dll_path]
      +    elif sys.platform.startswith('aix'):
      +        dll_path = [os.path.join(p, 'libxgboost.a') for p in dll_path]
       
           lib_path = [p for p in dll_path if os.path.exists(p) and os.path.isfile(p)]
       

      # diff -u xgboost/core.py_orig xgboost/core.py
      --- xgboost/core.py_orig        2023-05-30 02:07:56.180025453 +0000
      +++ xgboost/core.py     2023-05-30 02:21:22.261566474 +0000
      @@ -172,7 +172,7 @@
                   os.environ["PATH"] = os.pathsep.join(
                       pathBackup + [os.path.dirname(lib_path)]
                   )
      -            lib = ctypes.cdll.LoadLibrary(lib_path)
      +            lib = ctypes.cdll.LoadLibrary(lib_path + '(libxgboost.so)')
                   lib_success = True
               except OSError as e:
                   os_error_list.append(str(e))

      Then inside python-package directory , run python3 -m pip install . 

      I would recommend you to use a custom path to install it , python3 -m pip install . --root <custompath>
      Then do export PYTHONPATH=<custompath>/opt/freeware/lib64/python3.7/site-packages



      ------------------------------
      Ayappan P
      ------------------------------



    3. 15.  RE: Issue with C / C++ Compiling during the PIP installation for Xgboost

      Posted Tue May 30, 2023 12:00 PM

      Hi Ayappan,

      I am wondering if there is some workaround to avoid using those git commands? Our servers don't have internet connection due to the security control measures on them.

      Thanks

      Mi



      ------------------------------
      Mi Wang
      ------------------------------



    4. 16.  RE: Issue with C / C++ Compiling during the PIP installation for Xgboost

      Posted Wed May 31, 2023 03:40 AM

      You can run these steps in a machine with internet access, create a tar file and copy it to the AIX machine.

      git clone https://github.com/dmlc/xgboost
      cd xgboost
      git checkout v1.6.2
      git submodule init
      git submodule update

      Then you can create a tar file "tar -cf xgboost.tar xgboost" and copy the xgboost.tar to the AIX machine. Then you can untar it and do the remaining steps.



      ------------------------------
      Ayappan P
      ------------------------------