AIX Open Source

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

Hi Ranjit,

Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

mkdir build
cd build
export CC="gcc -maix64 -pthread"
export CXX="g++ -maix64 -pthread"
export OBJECT_MODE=64

............

Thanks

Mi

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

Okay, Wang.

Make the changes for below file also .  you can take the change from previous thread.

• setup.py,
• xgboost/libpath.py,
• xgboost/core.py
  
  

To build and install, run the command 'python3 -m pip install . --root /usr/local'.  

Before using it, export below two path.
export LIBPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/xgboost/lib
export PYTHONPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/

This should work and it worked for me.  :) 

Hi Ranjit,

Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

mkdir build
cd build
export CC="gcc -maix64 -pthread"
export CXX="g++ -maix64 -pthread"
export OBJECT_MODE=64

............

Thanks

Mi

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

Hi Ranjit,

As our Python 3.9 environment is temporarily not available for me, I have tried installing xgboost 1.6.2 on another Python 3.7 environment. I used the same steps from you and got the following error. Can you please help?

Thanks

Mi

Okay, Wang.

Make the changes for below file also .  you can take the change from previous thread.

• setup.py,
• xgboost/libpath.py,
• xgboost/core.py
  
  

To build and install, run the command 'python3 -m pip install . --root /usr/local'.  

Before using it, export below two path.
export LIBPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/xgboost/lib
export PYTHONPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/

This should work and it worked for me.  :) 

Hi Ranjit,

Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

mkdir build
cd build
export CC="gcc -maix64 -pthread"
export CXX="g++ -maix64 -pthread"
export OBJECT_MODE=64

............

Thanks

Mi

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

what are the env exports used ?

cmake is not able to find pthread.h that's why it's throwing above error. 

Thanks

Ranjit

Hi Ranjit,

As our Python 3.9 environment is temporarily not available for me, I have tried installing xgboost 1.6.2 on another Python 3.7 environment. I used the same steps from you and got the following error. Can you please help?

Thanks

Mi

Okay, Wang.

Make the changes for below file also .  you can take the change from previous thread.

• setup.py,
• xgboost/libpath.py,
• xgboost/core.py
  
  

To build and install, run the command 'python3 -m pip install . --root /usr/local'.  

Before using it, export below two path.
export LIBPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/xgboost/lib
export PYTHONPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/

This should work and it worked for me.  :) 

Hi Ranjit,

Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

mkdir build
cd build
export CC="gcc -maix64 -pthread"
export CXX="g++ -maix64 -pthread"
export OBJECT_MODE=64

............

Thanks

Mi

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

Hi Ranjit,

I use the following for defining env parameters. Should I also add export LIBPATH=/opt/freeware/lib64:$LIBPATH as well?

export OBJECT_MODE=64
export CC="gcc  -maix64 -pthread"
export CXX="g++ -maix64 -pthread"

Thanks

Mi

what are the env exports used ?

cmake is not able to find pthread.h that's why it's throwing above error. 

Thanks

Ranjit

Hi Ranjit,

As our Python 3.9 environment is temporarily not available for me, I have tried installing xgboost 1.6.2 on another Python 3.7 environment. I used the same steps from you and got the following error. Can you please help?

Thanks

Mi

Okay, Wang.

Make the changes for below file also .  you can take the change from previous thread.

• setup.py,
• xgboost/libpath.py,
• xgboost/core.py
  
  

To build and install, run the command 'python3 -m pip install . --root /usr/local'.  

Before using it, export below two path.
export LIBPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/xgboost/lib
export PYTHONPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/

This should work and it worked for me.  :) 

Hi Ranjit,

Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

mkdir build
cd build
export CC="gcc -maix64 -pthread"
export CXX="g++ -maix64 -pthread"
export OBJECT_MODE=64

............

Thanks

Mi

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

By the way, I used the following command to install the package:

python3 -m pip install . --root /

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 10:08 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

I use the following for defining env parameters. Should I also add export LIBPATH=/opt/freeware/lib64:$LIBPATH as well?

export OBJECT_MODE=64
export CC="gcc  -maix64 -pthread"
export CXX="g++ -maix64 -pthread"

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 02:48 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

what are the env exports used ?

cmake is not able to find pthread.h that's why it's throwing above error. 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Fri September 22, 2023 02:55 PM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

As our Python 3.9 environment is temporarily not available for me, I have tried installing xgboost 1.6.2 on another Python 3.7 environment. I used the same steps from you and got the following error. Can you please help?

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Wed September 20, 2023 08:37 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Okay, Wang.

Make the changes for below file also .  you can take the change from previous thread.

• setup.py,
• xgboost/libpath.py,
• xgboost/core.py
  
  

To build and install, run the command 'python3 -m pip install . --root /usr/local'.  

Before using it, export below two path.
export LIBPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/xgboost/lib
export PYTHONPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/

This should work and it worked for me.  :) 

------------------------------
Ranjit Ranjan

Original Message:
Sent: Tue September 19, 2023 10:51 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

mkdir build
cd build
export CC="gcc -maix64 -pthread"
export CXX="g++ -maix64 -pthread"
export OBJECT_MODE=64

............

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Tue September 19, 2023 01:29 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 18, 2023 10:22 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

------------------------------
Mi Wang
------------------------------

Above export env is okay.  No need to set any LIBPATH.

I don't have python3.7 env to try it out. But it should work. 

I Don't see any issue in python3.9. 

Try freshly from start ,by extracting the source code of 1.6.2 , applying the suggested patches and run the same command for installation.

you can add -v for verbose output 'python3 -m pip install . --root / -v' 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 25, 2023 10:35 AM
From: Mi Wang
Subject: Error during installing xgboost

By the way, I used the following command to install the package:

python3 -m pip install . --root /

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 10:08 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

I use the following for defining env parameters. Should I also add export LIBPATH=/opt/freeware/lib64:$LIBPATH as well?

export OBJECT_MODE=64
export CC="gcc  -maix64 -pthread"
export CXX="g++ -maix64 -pthread"

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 02:48 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

what are the env exports used ?

cmake is not able to find pthread.h that's why it's throwing above error. 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Fri September 22, 2023 02:55 PM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

As our Python 3.9 environment is temporarily not available for me, I have tried installing xgboost 1.6.2 on another Python 3.7 environment. I used the same steps from you and got the following error. Can you please help?

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Wed September 20, 2023 08:37 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Okay, Wang.

Make the changes for below file also .  you can take the change from previous thread.

• setup.py,
• xgboost/libpath.py,
• xgboost/core.py
  
  

To build and install, run the command 'python3 -m pip install . --root /usr/local'.  

Before using it, export below two path.
export LIBPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/xgboost/lib
export PYTHONPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/

This should work and it worked for me.  :) 

------------------------------
Ranjit Ranjan

Original Message:
Sent: Tue September 19, 2023 10:51 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

mkdir build
cd build
export CC="gcc -maix64 -pthread"
export CXX="g++ -maix64 -pthread"
export OBJECT_MODE=64

............

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Tue September 19, 2023 01:29 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 18, 2023 10:22 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

------------------------------
Mi Wang
------------------------------

Hi Ranjit,

I just tried he installation of xgboost-1.7.5 on our Python 3.9 environment. I did the same environment variable exports. Please see following modifications to the 3 py scripts. The installation failed (see below). Please help.

Thanks

Mi

Modification:

/tmp/xgboost/xgboost-1.7.5: diff setup.py_orig setup.py
46a47,48
>     elif system() == 'AIX':
>         name = 'libxgboost.a'

/tmp/xgboost/xgboost-1.7.5/xgboost: diff libpath.py_orig libpath.py
51a52,53
>     elif sys.platform.startswith('aix'):
>         dll_path = [os.path.join(p, 'libxgboost.a') for p in dll_path]

/tmp/xgboost/xgboost-1.7.5/xgboost: diff core.py_orig core.py
206c206
<             lib = ctypes.cdll.LoadLibrary(lib_path)
---
>             lib = ctypes.cdll.LoadLibrary(lib_path + '(libxgboost.so)')

Error message:

 [ 96%] Building CXX object src/CMakeFiles/objxgboost.dir/__/rabit/src/engine.cc.o
      In file included from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:55,
                       from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/src/collective/socket.cc:4:
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h: In member function 'xgboost::collective::SockDomain xgboost::collective::TCPSocket::Domain() const':
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:294:41: error: 'SO_DOMAIN' was not declared in this scope; did you mean 'DOMAIN'?
        294 |         getsockopt(handle_, SOL_SOCKET, SO_DOMAIN, reinterpret_cast<char *>(&domain), &len), 0);
            |                                         ^~~~~~~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/base.h:73:54: note: in definition of macro 'XGBOOST_EXPECT'
         73 | #define XGBOOST_EXPECT(cond, ret)  __builtin_expect((cond), (ret))
            |                                                      ^~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:293:5: note: in expansion of macro 'xgboost_CHECK_SYS_CALL'
        293 |     xgboost_CHECK_SYS_CALL(
            |     ^~~~~~~~~~~~~~~~~~~~~~

Above export env is okay.  No need to set any LIBPATH.

I don't have python3.7 env to try it out. But it should work. 

I Don't see any issue in python3.9. 

Try freshly from start ,by extracting the source code of 1.6.2 , applying the suggested patches and run the same command for installation.

you can add -v for verbose output 'python3 -m pip install . --root / -v' 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 25, 2023 10:35 AM
From: Mi Wang
Subject: Error during installing xgboost

By the way, I used the following command to install the package:

python3 -m pip install . --root /

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 10:08 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

I use the following for defining env parameters. Should I also add export LIBPATH=/opt/freeware/lib64:$LIBPATH as well?

export OBJECT_MODE=64
export CC="gcc  -maix64 -pthread"
export CXX="g++ -maix64 -pthread"

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 02:48 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

what are the env exports used ?

cmake is not able to find pthread.h that's why it's throwing above error. 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Fri September 22, 2023 02:55 PM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

As our Python 3.9 environment is temporarily not available for me, I have tried installing xgboost 1.6.2 on another Python 3.7 environment. I used the same steps from you and got the following error. Can you please help?

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Wed September 20, 2023 08:37 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Okay, Wang.

Make the changes for below file also .  you can take the change from previous thread.

• setup.py,
• xgboost/libpath.py,
• xgboost/core.py
  
  

To build and install, run the command 'python3 -m pip install . --root /usr/local'.  

Before using it, export below two path.
export LIBPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/xgboost/lib
export PYTHONPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/

This should work and it worked for me.  :) 

------------------------------
Ranjit Ranjan

Original Message:
Sent: Tue September 19, 2023 10:51 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

mkdir build
cd build
export CC="gcc -maix64 -pthread"
export CXX="g++ -maix64 -pthread"
export OBJECT_MODE=64

............

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Tue September 19, 2023 01:29 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 18, 2023 10:22 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

------------------------------
Mi Wang
------------------------------

For 1.7.5, you also need to apply the patch for socket.h . I think, you know the changes as you mentioned in the starting post message.

Apply that change and try it out.

Hi Ranjit,

I just tried he installation of xgboost-1.7.5 on our Python 3.9 environment. I did the same environment variable exports. Please see following modifications to the 3 py scripts. The installation failed (see below). Please help.

Thanks

Mi

Modification:

/tmp/xgboost/xgboost-1.7.5: diff setup.py_orig setup.py
46a47,48
>     elif system() == 'AIX':
>         name = 'libxgboost.a'

/tmp/xgboost/xgboost-1.7.5/xgboost: diff libpath.py_orig libpath.py
51a52,53
>     elif sys.platform.startswith('aix'):
>         dll_path = [os.path.join(p, 'libxgboost.a') for p in dll_path]

/tmp/xgboost/xgboost-1.7.5/xgboost: diff core.py_orig core.py
206c206
<             lib = ctypes.cdll.LoadLibrary(lib_path)
---
>             lib = ctypes.cdll.LoadLibrary(lib_path + '(libxgboost.so)')

Error message:

 [ 96%] Building CXX object src/CMakeFiles/objxgboost.dir/__/rabit/src/engine.cc.o
      In file included from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:55,
                       from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/src/collective/socket.cc:4:
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h: In member function 'xgboost::collective::SockDomain xgboost::collective::TCPSocket::Domain() const':
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:294:41: error: 'SO_DOMAIN' was not declared in this scope; did you mean 'DOMAIN'?
        294 |         getsockopt(handle_, SOL_SOCKET, SO_DOMAIN, reinterpret_cast<char *>(&domain), &len), 0);
            |                                         ^~~~~~~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/base.h:73:54: note: in definition of macro 'XGBOOST_EXPECT'
         73 | #define XGBOOST_EXPECT(cond, ret)  __builtin_expect((cond), (ret))
            |                                                      ^~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:293:5: note: in expansion of macro 'xgboost_CHECK_SYS_CALL'
        293 |     xgboost_CHECK_SYS_CALL(
            |     ^~~~~~~~~~~~~~~~~~~~~~

Above export env is okay.  No need to set any LIBPATH.

I don't have python3.7 env to try it out. But it should work. 

I Don't see any issue in python3.9. 

Try freshly from start ,by extracting the source code of 1.6.2 , applying the suggested patches and run the same command for installation.

you can add -v for verbose output 'python3 -m pip install . --root / -v' 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 25, 2023 10:35 AM
From: Mi Wang
Subject: Error during installing xgboost

By the way, I used the following command to install the package:

python3 -m pip install . --root /

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 10:08 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

I use the following for defining env parameters. Should I also add export LIBPATH=/opt/freeware/lib64:$LIBPATH as well?

export OBJECT_MODE=64
export CC="gcc  -maix64 -pthread"
export CXX="g++ -maix64 -pthread"

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 02:48 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

what are the env exports used ?

cmake is not able to find pthread.h that's why it's throwing above error. 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Fri September 22, 2023 02:55 PM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

As our Python 3.9 environment is temporarily not available for me, I have tried installing xgboost 1.6.2 on another Python 3.7 environment. I used the same steps from you and got the following error. Can you please help?

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Wed September 20, 2023 08:37 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Okay, Wang.

Make the changes for below file also .  you can take the change from previous thread.

• setup.py,
• xgboost/libpath.py,
• xgboost/core.py
  
  

To build and install, run the command 'python3 -m pip install . --root /usr/local'.  

Before using it, export below two path.
export LIBPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/xgboost/lib
export PYTHONPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/

This should work and it worked for me.  :) 

------------------------------
Ranjit Ranjan

Original Message:
Sent: Tue September 19, 2023 10:51 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

mkdir build
cd build
export CC="gcc -maix64 -pthread"
export CXX="g++ -maix64 -pthread"
export OBJECT_MODE=64

............

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Tue September 19, 2023 01:29 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 18, 2023 10:22 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

------------------------------
Mi Wang
------------------------------

Hi Ranjit,

After I mode the modification to the socket.h file, I am now getting below new error message. Please take a look.

Thanks

Mi

INFO:XGBoost build_ext:Run CMake command: ['cmake', 'xgboost', '-GUnix Makefiles', '-DUSE_OPENMP=1', '-DUSE_CUDA=0', '-DUSE_NCCL=0', '-DBUILD_WITH_SHARED_NCCL=0', '-DHIDE_CXX_SYMBOLS=1', '-DUSE_HDFS=0', '-DUSE_AZURE=0', '-DUSE_S3=0', '-DPLUGIN_DENSE_PARSER=0']
  Could not load program cmake:
  Symbol resolution failed for cmake because:
        Symbol _ZTINSt6thread6_StateE (number 351) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZTVNSt6thread6_StateE (number 352) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread4joinEv (number 381) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variableD1Ev (number 403) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variableC1Ev (number 404) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable4waitERSt11unique_lockISt5mutexE (number 448) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable10notify_oneEv (number 449) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable10notify_allEv (number 450) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread6_StateD2Ev (number 461) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread15_M_start_threadESt10unique_ptrINS_6_StateESt14default_deleteIS1_EEPFvvE (number 462) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread20hardware_concurrencyEv (number 463) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
  Examine .loader section symbols with the 'dump -Tv' command.

For 1.7.5, you also need to apply the patch for socket.h . I think, you know the changes as you mentioned in the starting post message.

Apply that change and try it out.

Hi Ranjit,

I just tried he installation of xgboost-1.7.5 on our Python 3.9 environment. I did the same environment variable exports. Please see following modifications to the 3 py scripts. The installation failed (see below). Please help.

Thanks

Mi

Modification:

/tmp/xgboost/xgboost-1.7.5: diff setup.py_orig setup.py
46a47,48
>     elif system() == 'AIX':
>         name = 'libxgboost.a'

/tmp/xgboost/xgboost-1.7.5/xgboost: diff libpath.py_orig libpath.py
51a52,53
>     elif sys.platform.startswith('aix'):
>         dll_path = [os.path.join(p, 'libxgboost.a') for p in dll_path]

/tmp/xgboost/xgboost-1.7.5/xgboost: diff core.py_orig core.py
206c206
<             lib = ctypes.cdll.LoadLibrary(lib_path)
---
>             lib = ctypes.cdll.LoadLibrary(lib_path + '(libxgboost.so)')

Error message:

 [ 96%] Building CXX object src/CMakeFiles/objxgboost.dir/__/rabit/src/engine.cc.o
      In file included from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:55,
                       from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/src/collective/socket.cc:4:
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h: In member function 'xgboost::collective::SockDomain xgboost::collective::TCPSocket::Domain() const':
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:294:41: error: 'SO_DOMAIN' was not declared in this scope; did you mean 'DOMAIN'?
        294 |         getsockopt(handle_, SOL_SOCKET, SO_DOMAIN, reinterpret_cast<char *>(&domain), &len), 0);
            |                                         ^~~~~~~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/base.h:73:54: note: in definition of macro 'XGBOOST_EXPECT'
         73 | #define XGBOOST_EXPECT(cond, ret)  __builtin_expect((cond), (ret))
            |                                                      ^~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:293:5: note: in expansion of macro 'xgboost_CHECK_SYS_CALL'
        293 |     xgboost_CHECK_SYS_CALL(
            |     ^~~~~~~~~~~~~~~~~~~~~~

Above export env is okay.  No need to set any LIBPATH.

I don't have python3.7 env to try it out. But it should work. 

I Don't see any issue in python3.9. 

Try freshly from start ,by extracting the source code of 1.6.2 , applying the suggested patches and run the same command for installation.

you can add -v for verbose output 'python3 -m pip install . --root / -v' 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 25, 2023 10:35 AM
From: Mi Wang
Subject: Error during installing xgboost

By the way, I used the following command to install the package:

python3 -m pip install . --root /

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 10:08 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

I use the following for defining env parameters. Should I also add export LIBPATH=/opt/freeware/lib64:$LIBPATH as well?

export OBJECT_MODE=64
export CC="gcc  -maix64 -pthread"
export CXX="g++ -maix64 -pthread"

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 02:48 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

what are the env exports used ?

cmake is not able to find pthread.h that's why it's throwing above error. 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Fri September 22, 2023 02:55 PM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

As our Python 3.9 environment is temporarily not available for me, I have tried installing xgboost 1.6.2 on another Python 3.7 environment. I used the same steps from you and got the following error. Can you please help?

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Wed September 20, 2023 08:37 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Okay, Wang.

Make the changes for below file also .  you can take the change from previous thread.

• setup.py,
• xgboost/libpath.py,
• xgboost/core.py
  
  

To build and install, run the command 'python3 -m pip install . --root /usr/local'.  

Before using it, export below two path.
export LIBPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/xgboost/lib
export PYTHONPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/

This should work and it worked for me.  :) 

------------------------------
Ranjit Ranjan

Original Message:
Sent: Tue September 19, 2023 10:51 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

mkdir build
cd build
export CC="gcc -maix64 -pthread"
export CXX="g++ -maix64 -pthread"
export OBJECT_MODE=64

............

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Tue September 19, 2023 01:29 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 18, 2023 10:22 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

------------------------------
Mi Wang
------------------------------

Hi Wang,

Looks ,  LIBPATH issue.

are you able to run 'cmake' from the shell session? just type 'cmake' and enter'.

are you setting any LIBPATH in your build session ? 

Try without setting any LIBPATH.

if you still see the above error message 'Could not load program cmake', then collect the output of below commands:

• rpm -qa | grep -i cmake
• ldd /opt/freeware/bin/cmake

Thanks

Ranjit

Hi Ranjit,

After I mode the modification to the socket.h file, I am now getting below new error message. Please take a look.

Thanks

Mi

INFO:XGBoost build_ext:Run CMake command: ['cmake', 'xgboost', '-GUnix Makefiles', '-DUSE_OPENMP=1', '-DUSE_CUDA=0', '-DUSE_NCCL=0', '-DBUILD_WITH_SHARED_NCCL=0', '-DHIDE_CXX_SYMBOLS=1', '-DUSE_HDFS=0', '-DUSE_AZURE=0', '-DUSE_S3=0', '-DPLUGIN_DENSE_PARSER=0']
  Could not load program cmake:
  Symbol resolution failed for cmake because:
        Symbol _ZTINSt6thread6_StateE (number 351) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZTVNSt6thread6_StateE (number 352) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread4joinEv (number 381) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variableD1Ev (number 403) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variableC1Ev (number 404) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable4waitERSt11unique_lockISt5mutexE (number 448) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable10notify_oneEv (number 449) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable10notify_allEv (number 450) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread6_StateD2Ev (number 461) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread15_M_start_threadESt10unique_ptrINS_6_StateESt14default_deleteIS1_EEPFvvE (number 462) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread20hardware_concurrencyEv (number 463) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
  Examine .loader section symbols with the 'dump -Tv' command.

For 1.7.5, you also need to apply the patch for socket.h . I think, you know the changes as you mentioned in the starting post message.

Apply that change and try it out.

Hi Ranjit,

I just tried he installation of xgboost-1.7.5 on our Python 3.9 environment. I did the same environment variable exports. Please see following modifications to the 3 py scripts. The installation failed (see below). Please help.

Thanks

Mi

Modification:

/tmp/xgboost/xgboost-1.7.5: diff setup.py_orig setup.py
46a47,48
>     elif system() == 'AIX':
>         name = 'libxgboost.a'

/tmp/xgboost/xgboost-1.7.5/xgboost: diff libpath.py_orig libpath.py
51a52,53
>     elif sys.platform.startswith('aix'):
>         dll_path = [os.path.join(p, 'libxgboost.a') for p in dll_path]

/tmp/xgboost/xgboost-1.7.5/xgboost: diff core.py_orig core.py
206c206
<             lib = ctypes.cdll.LoadLibrary(lib_path)
---
>             lib = ctypes.cdll.LoadLibrary(lib_path + '(libxgboost.so)')

Error message:

 [ 96%] Building CXX object src/CMakeFiles/objxgboost.dir/__/rabit/src/engine.cc.o
      In file included from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:55,
                       from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/src/collective/socket.cc:4:
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h: In member function 'xgboost::collective::SockDomain xgboost::collective::TCPSocket::Domain() const':
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:294:41: error: 'SO_DOMAIN' was not declared in this scope; did you mean 'DOMAIN'?
        294 |         getsockopt(handle_, SOL_SOCKET, SO_DOMAIN, reinterpret_cast<char *>(&domain), &len), 0);
            |                                         ^~~~~~~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/base.h:73:54: note: in definition of macro 'XGBOOST_EXPECT'
         73 | #define XGBOOST_EXPECT(cond, ret)  __builtin_expect((cond), (ret))
            |                                                      ^~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:293:5: note: in expansion of macro 'xgboost_CHECK_SYS_CALL'
        293 |     xgboost_CHECK_SYS_CALL(
            |     ^~~~~~~~~~~~~~~~~~~~~~

Above export env is okay.  No need to set any LIBPATH.

I don't have python3.7 env to try it out. But it should work. 

I Don't see any issue in python3.9. 

Try freshly from start ,by extracting the source code of 1.6.2 , applying the suggested patches and run the same command for installation.

you can add -v for verbose output 'python3 -m pip install . --root / -v' 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 25, 2023 10:35 AM
From: Mi Wang
Subject: Error during installing xgboost

By the way, I used the following command to install the package:

python3 -m pip install . --root /

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 10:08 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

I use the following for defining env parameters. Should I also add export LIBPATH=/opt/freeware/lib64:$LIBPATH as well?

export OBJECT_MODE=64
export CC="gcc  -maix64 -pthread"
export CXX="g++ -maix64 -pthread"

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 02:48 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

what are the env exports used ?

cmake is not able to find pthread.h that's why it's throwing above error. 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Fri September 22, 2023 02:55 PM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

As our Python 3.9 environment is temporarily not available for me, I have tried installing xgboost 1.6.2 on another Python 3.7 environment. I used the same steps from you and got the following error. Can you please help?

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Wed September 20, 2023 08:37 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Okay, Wang.

Make the changes for below file also .  you can take the change from previous thread.

• setup.py,
• xgboost/libpath.py,
• xgboost/core.py
  
  

To build and install, run the command 'python3 -m pip install . --root /usr/local'.  

Before using it, export below two path.
export LIBPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/xgboost/lib
export PYTHONPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/

This should work and it worked for me.  :) 

------------------------------
Ranjit Ranjan

Original Message:
Sent: Tue September 19, 2023 10:51 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

mkdir build
cd build
export CC="gcc -maix64 -pthread"
export CXX="g++ -maix64 -pthread"
export OBJECT_MODE=64

............

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Tue September 19, 2023 01:29 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 18, 2023 10:22 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

------------------------------
Mi Wang
------------------------------

Hi Ranjit,

I have followed your recommendation and the installation has got into normal 'building CXX...' stage. Then it hit with the error (see below for error messages) related to space. I have tried to increase the space for / and /opt filesystems (the /tmp has about 10 GB free space), but still got the same error. Can you please help to find out where this 'not enough space' complaint is for? 

Thanks

Mi

......

g++: fatal error: Killed signal terminated program cc1plus
  compilation terminated.
  /bin/sh[3]: 11206954 Killed
  error: [Errno 12] Not enough space
  /bin/sh[3]: 11862358 Killed
  /bin/sh[3]: 13435344 Killed

Hi Wang,

Looks ,  LIBPATH issue.

are you able to run 'cmake' from the shell session? just type 'cmake' and enter'.

are you setting any LIBPATH in your build session ? 

Try without setting any LIBPATH.

if you still see the above error message 'Could not load program cmake', then collect the output of below commands:

• rpm -qa | grep -i cmake
• ldd /opt/freeware/bin/cmake

Thanks

Ranjit

Hi Ranjit,

After I mode the modification to the socket.h file, I am now getting below new error message. Please take a look.

Thanks

Mi

INFO:XGBoost build_ext:Run CMake command: ['cmake', 'xgboost', '-GUnix Makefiles', '-DUSE_OPENMP=1', '-DUSE_CUDA=0', '-DUSE_NCCL=0', '-DBUILD_WITH_SHARED_NCCL=0', '-DHIDE_CXX_SYMBOLS=1', '-DUSE_HDFS=0', '-DUSE_AZURE=0', '-DUSE_S3=0', '-DPLUGIN_DENSE_PARSER=0']
  Could not load program cmake:
  Symbol resolution failed for cmake because:
        Symbol _ZTINSt6thread6_StateE (number 351) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZTVNSt6thread6_StateE (number 352) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread4joinEv (number 381) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variableD1Ev (number 403) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variableC1Ev (number 404) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable4waitERSt11unique_lockISt5mutexE (number 448) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable10notify_oneEv (number 449) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable10notify_allEv (number 450) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread6_StateD2Ev (number 461) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread15_M_start_threadESt10unique_ptrINS_6_StateESt14default_deleteIS1_EEPFvvE (number 462) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread20hardware_concurrencyEv (number 463) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
  Examine .loader section symbols with the 'dump -Tv' command.

For 1.7.5, you also need to apply the patch for socket.h . I think, you know the changes as you mentioned in the starting post message.

Apply that change and try it out.

Hi Ranjit,

I just tried he installation of xgboost-1.7.5 on our Python 3.9 environment. I did the same environment variable exports. Please see following modifications to the 3 py scripts. The installation failed (see below). Please help.

Thanks

Mi

Modification:

/tmp/xgboost/xgboost-1.7.5: diff setup.py_orig setup.py
46a47,48
>     elif system() == 'AIX':
>         name = 'libxgboost.a'

/tmp/xgboost/xgboost-1.7.5/xgboost: diff libpath.py_orig libpath.py
51a52,53
>     elif sys.platform.startswith('aix'):
>         dll_path = [os.path.join(p, 'libxgboost.a') for p in dll_path]

/tmp/xgboost/xgboost-1.7.5/xgboost: diff core.py_orig core.py
206c206
<             lib = ctypes.cdll.LoadLibrary(lib_path)
---
>             lib = ctypes.cdll.LoadLibrary(lib_path + '(libxgboost.so)')

Error message:

 [ 96%] Building CXX object src/CMakeFiles/objxgboost.dir/__/rabit/src/engine.cc.o
      In file included from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:55,
                       from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/src/collective/socket.cc:4:
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h: In member function 'xgboost::collective::SockDomain xgboost::collective::TCPSocket::Domain() const':
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:294:41: error: 'SO_DOMAIN' was not declared in this scope; did you mean 'DOMAIN'?
        294 |         getsockopt(handle_, SOL_SOCKET, SO_DOMAIN, reinterpret_cast<char *>(&domain), &len), 0);
            |                                         ^~~~~~~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/base.h:73:54: note: in definition of macro 'XGBOOST_EXPECT'
         73 | #define XGBOOST_EXPECT(cond, ret)  __builtin_expect((cond), (ret))
            |                                                      ^~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:293:5: note: in expansion of macro 'xgboost_CHECK_SYS_CALL'
        293 |     xgboost_CHECK_SYS_CALL(
            |     ^~~~~~~~~~~~~~~~~~~~~~

Above export env is okay.  No need to set any LIBPATH.

I don't have python3.7 env to try it out. But it should work. 

I Don't see any issue in python3.9. 

Try freshly from start ,by extracting the source code of 1.6.2 , applying the suggested patches and run the same command for installation.

you can add -v for verbose output 'python3 -m pip install . --root / -v' 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 25, 2023 10:35 AM
From: Mi Wang
Subject: Error during installing xgboost

By the way, I used the following command to install the package:

python3 -m pip install . --root /

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 10:08 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

I use the following for defining env parameters. Should I also add export LIBPATH=/opt/freeware/lib64:$LIBPATH as well?

export OBJECT_MODE=64
export CC="gcc  -maix64 -pthread"
export CXX="g++ -maix64 -pthread"

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 02:48 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

what are the env exports used ?

cmake is not able to find pthread.h that's why it's throwing above error. 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Fri September 22, 2023 02:55 PM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

As our Python 3.9 environment is temporarily not available for me, I have tried installing xgboost 1.6.2 on another Python 3.7 environment. I used the same steps from you and got the following error. Can you please help?

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Wed September 20, 2023 08:37 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Okay, Wang.

Make the changes for below file also .  you can take the change from previous thread.

• setup.py,
• xgboost/libpath.py,
• xgboost/core.py
  
  

To build and install, run the command 'python3 -m pip install . --root /usr/local'.  

Before using it, export below two path.
export LIBPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/xgboost/lib
export PYTHONPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/

This should work and it worked for me.  :) 

------------------------------
Ranjit Ranjan

Original Message:
Sent: Tue September 19, 2023 10:51 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

mkdir build
cd build
export CC="gcc -maix64 -pthread"
export CXX="g++ -maix64 -pthread"
export OBJECT_MODE=64

............

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Tue September 19, 2023 01:29 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 18, 2023 10:22 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

------------------------------
Mi Wang
------------------------------

Hi, 

is the disk partition under which you extracted the source code , having the enough disk space ?

check this also.

once you hit this error, capture 'df -k' also.

Thanks

Ranjit

Hi Ranjit,

I have followed your recommendation and the installation has got into normal 'building CXX...' stage. Then it hit with the error (see below for error messages) related to space. I have tried to increase the space for / and /opt filesystems (the /tmp has about 10 GB free space), but still got the same error. Can you please help to find out where this 'not enough space' complaint is for? 

Thanks

Mi

......

g++: fatal error: Killed signal terminated program cc1plus
  compilation terminated.
  /bin/sh[3]: 11206954 Killed
  error: [Errno 12] Not enough space
  /bin/sh[3]: 11862358 Killed
  /bin/sh[3]: 13435344 Killed

Hi Wang,

Looks ,  LIBPATH issue.

are you able to run 'cmake' from the shell session? just type 'cmake' and enter'.

are you setting any LIBPATH in your build session ? 

Try without setting any LIBPATH.

if you still see the above error message 'Could not load program cmake', then collect the output of below commands:

• rpm -qa | grep -i cmake
• ldd /opt/freeware/bin/cmake

Thanks

Ranjit

Hi Ranjit,

After I mode the modification to the socket.h file, I am now getting below new error message. Please take a look.

Thanks

Mi

INFO:XGBoost build_ext:Run CMake command: ['cmake', 'xgboost', '-GUnix Makefiles', '-DUSE_OPENMP=1', '-DUSE_CUDA=0', '-DUSE_NCCL=0', '-DBUILD_WITH_SHARED_NCCL=0', '-DHIDE_CXX_SYMBOLS=1', '-DUSE_HDFS=0', '-DUSE_AZURE=0', '-DUSE_S3=0', '-DPLUGIN_DENSE_PARSER=0']
  Could not load program cmake:
  Symbol resolution failed for cmake because:
        Symbol _ZTINSt6thread6_StateE (number 351) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZTVNSt6thread6_StateE (number 352) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread4joinEv (number 381) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variableD1Ev (number 403) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variableC1Ev (number 404) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable4waitERSt11unique_lockISt5mutexE (number 448) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable10notify_oneEv (number 449) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable10notify_allEv (number 450) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread6_StateD2Ev (number 461) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread15_M_start_threadESt10unique_ptrINS_6_StateESt14default_deleteIS1_EEPFvvE (number 462) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread20hardware_concurrencyEv (number 463) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
  Examine .loader section symbols with the 'dump -Tv' command.

For 1.7.5, you also need to apply the patch for socket.h . I think, you know the changes as you mentioned in the starting post message.

Apply that change and try it out.

Hi Ranjit,

I just tried he installation of xgboost-1.7.5 on our Python 3.9 environment. I did the same environment variable exports. Please see following modifications to the 3 py scripts. The installation failed (see below). Please help.

Thanks

Mi

Modification:

/tmp/xgboost/xgboost-1.7.5: diff setup.py_orig setup.py
46a47,48
>     elif system() == 'AIX':
>         name = 'libxgboost.a'

/tmp/xgboost/xgboost-1.7.5/xgboost: diff libpath.py_orig libpath.py
51a52,53
>     elif sys.platform.startswith('aix'):
>         dll_path = [os.path.join(p, 'libxgboost.a') for p in dll_path]

/tmp/xgboost/xgboost-1.7.5/xgboost: diff core.py_orig core.py
206c206
<             lib = ctypes.cdll.LoadLibrary(lib_path)
---
>             lib = ctypes.cdll.LoadLibrary(lib_path + '(libxgboost.so)')

Error message:

 [ 96%] Building CXX object src/CMakeFiles/objxgboost.dir/__/rabit/src/engine.cc.o
      In file included from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:55,
                       from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/src/collective/socket.cc:4:
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h: In member function 'xgboost::collective::SockDomain xgboost::collective::TCPSocket::Domain() const':
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:294:41: error: 'SO_DOMAIN' was not declared in this scope; did you mean 'DOMAIN'?
        294 |         getsockopt(handle_, SOL_SOCKET, SO_DOMAIN, reinterpret_cast<char *>(&domain), &len), 0);
            |                                         ^~~~~~~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/base.h:73:54: note: in definition of macro 'XGBOOST_EXPECT'
         73 | #define XGBOOST_EXPECT(cond, ret)  __builtin_expect((cond), (ret))
            |                                                      ^~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:293:5: note: in expansion of macro 'xgboost_CHECK_SYS_CALL'
        293 |     xgboost_CHECK_SYS_CALL(
            |     ^~~~~~~~~~~~~~~~~~~~~~

Above export env is okay.  No need to set any LIBPATH.

I don't have python3.7 env to try it out. But it should work. 

I Don't see any issue in python3.9. 

Try freshly from start ,by extracting the source code of 1.6.2 , applying the suggested patches and run the same command for installation.

you can add -v for verbose output 'python3 -m pip install . --root / -v' 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 25, 2023 10:35 AM
From: Mi Wang
Subject: Error during installing xgboost

By the way, I used the following command to install the package:

python3 -m pip install . --root /

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 10:08 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

I use the following for defining env parameters. Should I also add export LIBPATH=/opt/freeware/lib64:$LIBPATH as well?

export OBJECT_MODE=64
export CC="gcc  -maix64 -pthread"
export CXX="g++ -maix64 -pthread"

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 02:48 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

what are the env exports used ?

cmake is not able to find pthread.h that's why it's throwing above error. 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Fri September 22, 2023 02:55 PM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

As our Python 3.9 environment is temporarily not available for me, I have tried installing xgboost 1.6.2 on another Python 3.7 environment. I used the same steps from you and got the following error. Can you please help?

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Wed September 20, 2023 08:37 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Okay, Wang.

Make the changes for below file also .  you can take the change from previous thread.

• setup.py,
• xgboost/libpath.py,
• xgboost/core.py
  
  

To build and install, run the command 'python3 -m pip install . --root /usr/local'.  

Before using it, export below two path.
export LIBPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/xgboost/lib
export PYTHONPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/

This should work and it worked for me.  :) 

------------------------------
Ranjit Ranjan

Original Message:
Sent: Tue September 19, 2023 10:51 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

mkdir build
cd build
export CC="gcc -maix64 -pthread"
export CXX="g++ -maix64 -pthread"
export OBJECT_MODE=64

............

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Tue September 19, 2023 01:29 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 18, 2023 10:22 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

------------------------------
Mi Wang
------------------------------

Hi Ranjit,

That filesystem (for extracting the source code) has 74 GB free space. All other system related filesytems' space usage doesn't exceed 65 % except /usr, which has > 1GB free space. I am wondering could there be other type storage causing the error (I checked swap space and it's also fine)?

Thanks

Mi

Hi, 

is the disk partition under which you extracted the source code , having the enough disk space ?

check this also.

once you hit this error, capture 'df -k' also.

Thanks

Ranjit

Hi Ranjit,

I have followed your recommendation and the installation has got into normal 'building CXX...' stage. Then it hit with the error (see below for error messages) related to space. I have tried to increase the space for / and /opt filesystems (the /tmp has about 10 GB free space), but still got the same error. Can you please help to find out where this 'not enough space' complaint is for? 

Thanks

Mi

......

g++: fatal error: Killed signal terminated program cc1plus
  compilation terminated.
  /bin/sh[3]: 11206954 Killed
  error: [Errno 12] Not enough space
  /bin/sh[3]: 11862358 Killed
  /bin/sh[3]: 13435344 Killed

Hi Wang,

Looks ,  LIBPATH issue.

are you able to run 'cmake' from the shell session? just type 'cmake' and enter'.

are you setting any LIBPATH in your build session ? 

Try without setting any LIBPATH.

if you still see the above error message 'Could not load program cmake', then collect the output of below commands:

• rpm -qa | grep -i cmake
• ldd /opt/freeware/bin/cmake

Thanks

Ranjit

Hi Ranjit,

After I mode the modification to the socket.h file, I am now getting below new error message. Please take a look.

Thanks

Mi

INFO:XGBoost build_ext:Run CMake command: ['cmake', 'xgboost', '-GUnix Makefiles', '-DUSE_OPENMP=1', '-DUSE_CUDA=0', '-DUSE_NCCL=0', '-DBUILD_WITH_SHARED_NCCL=0', '-DHIDE_CXX_SYMBOLS=1', '-DUSE_HDFS=0', '-DUSE_AZURE=0', '-DUSE_S3=0', '-DPLUGIN_DENSE_PARSER=0']
  Could not load program cmake:
  Symbol resolution failed for cmake because:
        Symbol _ZTINSt6thread6_StateE (number 351) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZTVNSt6thread6_StateE (number 352) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread4joinEv (number 381) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variableD1Ev (number 403) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variableC1Ev (number 404) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable4waitERSt11unique_lockISt5mutexE (number 448) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable10notify_oneEv (number 449) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable10notify_allEv (number 450) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread6_StateD2Ev (number 461) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread15_M_start_threadESt10unique_ptrINS_6_StateESt14default_deleteIS1_EEPFvvE (number 462) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread20hardware_concurrencyEv (number 463) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
  Examine .loader section symbols with the 'dump -Tv' command.

For 1.7.5, you also need to apply the patch for socket.h . I think, you know the changes as you mentioned in the starting post message.

Apply that change and try it out.

Hi Ranjit,

I just tried he installation of xgboost-1.7.5 on our Python 3.9 environment. I did the same environment variable exports. Please see following modifications to the 3 py scripts. The installation failed (see below). Please help.

Thanks

Mi

Modification:

/tmp/xgboost/xgboost-1.7.5: diff setup.py_orig setup.py
46a47,48
>     elif system() == 'AIX':
>         name = 'libxgboost.a'

/tmp/xgboost/xgboost-1.7.5/xgboost: diff libpath.py_orig libpath.py
51a52,53
>     elif sys.platform.startswith('aix'):
>         dll_path = [os.path.join(p, 'libxgboost.a') for p in dll_path]

/tmp/xgboost/xgboost-1.7.5/xgboost: diff core.py_orig core.py
206c206
<             lib = ctypes.cdll.LoadLibrary(lib_path)
---
>             lib = ctypes.cdll.LoadLibrary(lib_path + '(libxgboost.so)')

Error message:

 [ 96%] Building CXX object src/CMakeFiles/objxgboost.dir/__/rabit/src/engine.cc.o
      In file included from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:55,
                       from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/src/collective/socket.cc:4:
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h: In member function 'xgboost::collective::SockDomain xgboost::collective::TCPSocket::Domain() const':
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:294:41: error: 'SO_DOMAIN' was not declared in this scope; did you mean 'DOMAIN'?
        294 |         getsockopt(handle_, SOL_SOCKET, SO_DOMAIN, reinterpret_cast<char *>(&domain), &len), 0);
            |                                         ^~~~~~~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/base.h:73:54: note: in definition of macro 'XGBOOST_EXPECT'
         73 | #define XGBOOST_EXPECT(cond, ret)  __builtin_expect((cond), (ret))
            |                                                      ^~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:293:5: note: in expansion of macro 'xgboost_CHECK_SYS_CALL'
        293 |     xgboost_CHECK_SYS_CALL(
            |     ^~~~~~~~~~~~~~~~~~~~~~

Above export env is okay.  No need to set any LIBPATH.

I don't have python3.7 env to try it out. But it should work. 

I Don't see any issue in python3.9. 

Try freshly from start ,by extracting the source code of 1.6.2 , applying the suggested patches and run the same command for installation.

you can add -v for verbose output 'python3 -m pip install . --root / -v' 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 25, 2023 10:35 AM
From: Mi Wang
Subject: Error during installing xgboost

By the way, I used the following command to install the package:

python3 -m pip install . --root /

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 10:08 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

I use the following for defining env parameters. Should I also add export LIBPATH=/opt/freeware/lib64:$LIBPATH as well?

export OBJECT_MODE=64
export CC="gcc  -maix64 -pthread"
export CXX="g++ -maix64 -pthread"

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 02:48 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

what are the env exports used ?

cmake is not able to find pthread.h that's why it's throwing above error. 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Fri September 22, 2023 02:55 PM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

As our Python 3.9 environment is temporarily not available for me, I have tried installing xgboost 1.6.2 on another Python 3.7 environment. I used the same steps from you and got the following error. Can you please help?

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Wed September 20, 2023 08:37 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Okay, Wang.

Make the changes for below file also .  you can take the change from previous thread.

• setup.py,
• xgboost/libpath.py,
• xgboost/core.py
  
  

To build and install, run the command 'python3 -m pip install . --root /usr/local'.  

Before using it, export below two path.
export LIBPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/xgboost/lib
export PYTHONPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/

This should work and it worked for me.  :) 

------------------------------
Ranjit Ranjan

Original Message:
Sent: Tue September 19, 2023 10:51 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

mkdir build
cd build
export CC="gcc -maix64 -pthread"
export CXX="g++ -maix64 -pthread"
export OBJECT_MODE=64

............

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Tue September 19, 2023 01:29 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 18, 2023 10:22 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

------------------------------
Mi Wang
------------------------------

Hi Ranjit,

I just tried installing the xgboost using a different filesystem's storage for extracting the source code. I got exact the same result: the building process failed when building CXX objects reached 96 %. SO this is not related to the filesystem holding the building contents. It must be something else.

Thanks

Mi

Hi Ranjit,

That filesystem (for extracting the source code) has 74 GB free space. All other system related filesytems' space usage doesn't exceed 65 % except /usr, which has > 1GB free space. I am wondering could there be other type storage causing the error (I checked swap space and it's also fine)?

Thanks

Mi

Hi, 

is the disk partition under which you extracted the source code , having the enough disk space ?

check this also.

once you hit this error, capture 'df -k' also.

Thanks

Ranjit

Hi Ranjit,

I have followed your recommendation and the installation has got into normal 'building CXX...' stage. Then it hit with the error (see below for error messages) related to space. I have tried to increase the space for / and /opt filesystems (the /tmp has about 10 GB free space), but still got the same error. Can you please help to find out where this 'not enough space' complaint is for? 

Thanks

Mi

......

g++: fatal error: Killed signal terminated program cc1plus
  compilation terminated.
  /bin/sh[3]: 11206954 Killed
  error: [Errno 12] Not enough space
  /bin/sh[3]: 11862358 Killed
  /bin/sh[3]: 13435344 Killed

Hi Wang,

Looks ,  LIBPATH issue.

are you able to run 'cmake' from the shell session? just type 'cmake' and enter'.

are you setting any LIBPATH in your build session ? 

Try without setting any LIBPATH.

if you still see the above error message 'Could not load program cmake', then collect the output of below commands:

• rpm -qa | grep -i cmake
• ldd /opt/freeware/bin/cmake

Thanks

Ranjit

Hi Ranjit,

After I mode the modification to the socket.h file, I am now getting below new error message. Please take a look.

Thanks

Mi

INFO:XGBoost build_ext:Run CMake command: ['cmake', 'xgboost', '-GUnix Makefiles', '-DUSE_OPENMP=1', '-DUSE_CUDA=0', '-DUSE_NCCL=0', '-DBUILD_WITH_SHARED_NCCL=0', '-DHIDE_CXX_SYMBOLS=1', '-DUSE_HDFS=0', '-DUSE_AZURE=0', '-DUSE_S3=0', '-DPLUGIN_DENSE_PARSER=0']
  Could not load program cmake:
  Symbol resolution failed for cmake because:
        Symbol _ZTINSt6thread6_StateE (number 351) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZTVNSt6thread6_StateE (number 352) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread4joinEv (number 381) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variableD1Ev (number 403) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variableC1Ev (number 404) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable4waitERSt11unique_lockISt5mutexE (number 448) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable10notify_oneEv (number 449) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable10notify_allEv (number 450) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread6_StateD2Ev (number 461) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread15_M_start_threadESt10unique_ptrINS_6_StateESt14default_deleteIS1_EEPFvvE (number 462) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread20hardware_concurrencyEv (number 463) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
  Examine .loader section symbols with the 'dump -Tv' command.

For 1.7.5, you also need to apply the patch for socket.h . I think, you know the changes as you mentioned in the starting post message.

Apply that change and try it out.

Hi Ranjit,

I just tried he installation of xgboost-1.7.5 on our Python 3.9 environment. I did the same environment variable exports. Please see following modifications to the 3 py scripts. The installation failed (see below). Please help.

Thanks

Mi

Modification:

/tmp/xgboost/xgboost-1.7.5: diff setup.py_orig setup.py
46a47,48
>     elif system() == 'AIX':
>         name = 'libxgboost.a'

/tmp/xgboost/xgboost-1.7.5/xgboost: diff libpath.py_orig libpath.py
51a52,53
>     elif sys.platform.startswith('aix'):
>         dll_path = [os.path.join(p, 'libxgboost.a') for p in dll_path]

/tmp/xgboost/xgboost-1.7.5/xgboost: diff core.py_orig core.py
206c206
<             lib = ctypes.cdll.LoadLibrary(lib_path)
---
>             lib = ctypes.cdll.LoadLibrary(lib_path + '(libxgboost.so)')

Error message:

 [ 96%] Building CXX object src/CMakeFiles/objxgboost.dir/__/rabit/src/engine.cc.o
      In file included from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:55,
                       from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/src/collective/socket.cc:4:
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h: In member function 'xgboost::collective::SockDomain xgboost::collective::TCPSocket::Domain() const':
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:294:41: error: 'SO_DOMAIN' was not declared in this scope; did you mean 'DOMAIN'?
        294 |         getsockopt(handle_, SOL_SOCKET, SO_DOMAIN, reinterpret_cast<char *>(&domain), &len), 0);
            |                                         ^~~~~~~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/base.h:73:54: note: in definition of macro 'XGBOOST_EXPECT'
         73 | #define XGBOOST_EXPECT(cond, ret)  __builtin_expect((cond), (ret))
            |                                                      ^~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:293:5: note: in expansion of macro 'xgboost_CHECK_SYS_CALL'
        293 |     xgboost_CHECK_SYS_CALL(
            |     ^~~~~~~~~~~~~~~~~~~~~~

Above export env is okay.  No need to set any LIBPATH.

I don't have python3.7 env to try it out. But it should work. 

I Don't see any issue in python3.9. 

Try freshly from start ,by extracting the source code of 1.6.2 , applying the suggested patches and run the same command for installation.

you can add -v for verbose output 'python3 -m pip install . --root / -v' 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 25, 2023 10:35 AM
From: Mi Wang
Subject: Error during installing xgboost

By the way, I used the following command to install the package:

python3 -m pip install . --root /

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 10:08 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

I use the following for defining env parameters. Should I also add export LIBPATH=/opt/freeware/lib64:$LIBPATH as well?

export OBJECT_MODE=64
export CC="gcc  -maix64 -pthread"
export CXX="g++ -maix64 -pthread"

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 02:48 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

what are the env exports used ?

cmake is not able to find pthread.h that's why it's throwing above error. 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Fri September 22, 2023 02:55 PM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

As our Python 3.9 environment is temporarily not available for me, I have tried installing xgboost 1.6.2 on another Python 3.7 environment. I used the same steps from you and got the following error. Can you please help?

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Wed September 20, 2023 08:37 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Okay, Wang.

Make the changes for below file also .  you can take the change from previous thread.

• setup.py,
• xgboost/libpath.py,
• xgboost/core.py
  
  

To build and install, run the command 'python3 -m pip install . --root /usr/local'.  

Before using it, export below two path.
export LIBPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/xgboost/lib
export PYTHONPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/

This should work and it worked for me.  :) 

------------------------------
Ranjit Ranjan

Original Message:
Sent: Tue September 19, 2023 10:51 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

mkdir build
cd build
export CC="gcc -maix64 -pthread"
export CXX="g++ -maix64 -pthread"
export OBJECT_MODE=64

............

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Tue September 19, 2023 01:29 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 18, 2023 10:22 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

------------------------------
Mi Wang
------------------------------

Try increasing below ulimit values. 
ulimit -d unlimited
ulimit -m unlimited
ulimit -n 20000

Hi Ranjit,

I just tried installing the xgboost using a different filesystem's storage for extracting the source code. I got exact the same result: the building process failed when building CXX objects reached 96 %. SO this is not related to the filesystem holding the building contents. It must be something else.

Thanks

Mi

Hi Ranjit,

That filesystem (for extracting the source code) has 74 GB free space. All other system related filesytems' space usage doesn't exceed 65 % except /usr, which has > 1GB free space. I am wondering could there be other type storage causing the error (I checked swap space and it's also fine)?

Thanks

Mi

Hi, 

is the disk partition under which you extracted the source code , having the enough disk space ?

check this also.

once you hit this error, capture 'df -k' also.

Thanks

Ranjit

Hi Ranjit,

I have followed your recommendation and the installation has got into normal 'building CXX...' stage. Then it hit with the error (see below for error messages) related to space. I have tried to increase the space for / and /opt filesystems (the /tmp has about 10 GB free space), but still got the same error. Can you please help to find out where this 'not enough space' complaint is for? 

Thanks

Mi

......

g++: fatal error: Killed signal terminated program cc1plus
  compilation terminated.
  /bin/sh[3]: 11206954 Killed
  error: [Errno 12] Not enough space
  /bin/sh[3]: 11862358 Killed
  /bin/sh[3]: 13435344 Killed

Hi Wang,

Looks ,  LIBPATH issue.

are you able to run 'cmake' from the shell session? just type 'cmake' and enter'.

are you setting any LIBPATH in your build session ? 

Try without setting any LIBPATH.

if you still see the above error message 'Could not load program cmake', then collect the output of below commands:

• rpm -qa | grep -i cmake
• ldd /opt/freeware/bin/cmake

Thanks

Ranjit

Hi Ranjit,

After I mode the modification to the socket.h file, I am now getting below new error message. Please take a look.

Thanks

Mi

INFO:XGBoost build_ext:Run CMake command: ['cmake', 'xgboost', '-GUnix Makefiles', '-DUSE_OPENMP=1', '-DUSE_CUDA=0', '-DUSE_NCCL=0', '-DBUILD_WITH_SHARED_NCCL=0', '-DHIDE_CXX_SYMBOLS=1', '-DUSE_HDFS=0', '-DUSE_AZURE=0', '-DUSE_S3=0', '-DPLUGIN_DENSE_PARSER=0']
  Could not load program cmake:
  Symbol resolution failed for cmake because:
        Symbol _ZTINSt6thread6_StateE (number 351) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZTVNSt6thread6_StateE (number 352) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread4joinEv (number 381) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variableD1Ev (number 403) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variableC1Ev (number 404) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable4waitERSt11unique_lockISt5mutexE (number 448) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable10notify_oneEv (number 449) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt18condition_variable10notify_allEv (number 450) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread6_StateD2Ev (number 461) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread15_M_start_threadESt10unique_ptrINS_6_StateESt14default_deleteIS1_EEPFvvE (number 462) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
        Symbol _ZNSt6thread20hardware_concurrencyEv (number 463) is not exported from dependent
          module /opt/freeware/lib/libstdc++.a[libstdc++.so.6].
  Examine .loader section symbols with the 'dump -Tv' command.

For 1.7.5, you also need to apply the patch for socket.h . I think, you know the changes as you mentioned in the starting post message.

Apply that change and try it out.

Hi Ranjit,

I just tried he installation of xgboost-1.7.5 on our Python 3.9 environment. I did the same environment variable exports. Please see following modifications to the 3 py scripts. The installation failed (see below). Please help.

Thanks

Mi

Modification:

/tmp/xgboost/xgboost-1.7.5: diff setup.py_orig setup.py
46a47,48
>     elif system() == 'AIX':
>         name = 'libxgboost.a'

/tmp/xgboost/xgboost-1.7.5/xgboost: diff libpath.py_orig libpath.py
51a52,53
>     elif sys.platform.startswith('aix'):
>         dll_path = [os.path.join(p, 'libxgboost.a') for p in dll_path]

/tmp/xgboost/xgboost-1.7.5/xgboost: diff core.py_orig core.py
206c206
<             lib = ctypes.cdll.LoadLibrary(lib_path)
---
>             lib = ctypes.cdll.LoadLibrary(lib_path + '(libxgboost.so)')

Error message:

 [ 96%] Building CXX object src/CMakeFiles/objxgboost.dir/__/rabit/src/engine.cc.o
      In file included from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:55,
                       from /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/src/collective/socket.cc:4:
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h: In member function 'xgboost::collective::SockDomain xgboost::collective::TCPSocket::Domain() const':
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:294:41: error: 'SO_DOMAIN' was not declared in this scope; did you mean 'DOMAIN'?
        294 |         getsockopt(handle_, SOL_SOCKET, SO_DOMAIN, reinterpret_cast<char *>(&domain), &len), 0);
            |                                         ^~~~~~~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/base.h:73:54: note: in definition of macro 'XGBOOST_EXPECT'
         73 | #define XGBOOST_EXPECT(cond, ret)  __builtin_expect((cond), (ret))
            |                                                      ^~~~
      /usr/opt/sas/python3a/xgboost-1.7.5/build/temp.aix-7205-2319-64-cpython-39/xgboost/include/xgboost/collective/socket.h:293:5: note: in expansion of macro 'xgboost_CHECK_SYS_CALL'
        293 |     xgboost_CHECK_SYS_CALL(
            |     ^~~~~~~~~~~~~~~~~~~~~~

Above export env is okay.  No need to set any LIBPATH.

I don't have python3.7 env to try it out. But it should work. 

I Don't see any issue in python3.9. 

Try freshly from start ,by extracting the source code of 1.6.2 , applying the suggested patches and run the same command for installation.

you can add -v for verbose output 'python3 -m pip install . --root / -v' 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 25, 2023 10:35 AM
From: Mi Wang
Subject: Error during installing xgboost

By the way, I used the following command to install the package:

python3 -m pip install . --root /

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 10:08 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

I use the following for defining env parameters. Should I also add export LIBPATH=/opt/freeware/lib64:$LIBPATH as well?

export OBJECT_MODE=64
export CC="gcc  -maix64 -pthread"
export CXX="g++ -maix64 -pthread"

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Mon September 25, 2023 02:48 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

what are the env exports used ?

cmake is not able to find pthread.h that's why it's throwing above error. 

Thanks

Ranjit

------------------------------
Ranjit Ranjan

Original Message:
Sent: Fri September 22, 2023 02:55 PM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

As our Python 3.9 environment is temporarily not available for me, I have tried installing xgboost 1.6.2 on another Python 3.7 environment. I used the same steps from you and got the following error. Can you please help?

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Wed September 20, 2023 08:37 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Okay, Wang.

Make the changes for below file also .  you can take the change from previous thread.

• setup.py,
• xgboost/libpath.py,
• xgboost/core.py
  
  

To build and install, run the command 'python3 -m pip install . --root /usr/local'.  

Before using it, export below two path.
export LIBPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/xgboost/lib
export PYTHONPATH=/usr/local/opt/freeware/lib64/python3.9/site-packages/

This should work and it worked for me.  :) 

------------------------------
Ranjit Ranjan

Original Message:
Sent: Tue September 19, 2023 10:51 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjit,

Our AIX server / other machine don't have git installed. I am wondering if I can do the following instead:

1. download the Xgboost package from PYPI site, upload it into a location on our AIX server, e.g.: /tmp/test and ungzip / untar the source code (folders / files) there. A subfolder called 'xgboost-1.7.5' will be created there.

2. Under the /tmp/test/xgboost-1.7.5 folder, continue the following steps:

mkdir build
cd build
export CC="gcc -maix64 -pthread"
export CXX="g++ -maix64 -pthread"
export OBJECT_MODE=64

............

Thanks

Mi

------------------------------
Mi Wang

Original Message:
Sent: Tue September 19, 2023 01:29 AM
From: Ranjit Ranjan
Subject: Error during installing xgboost

Hi Wang,

This is also discussed in previous thread https://community.ibm.com/community/user/power/discussion/issue-with-c-c-compiling-during-the-pip-installation-for-xgboost.

Could you try the steps provided by Ayappan and let us know if it's working or not.

Thanks

------------------------------
Ranjit Ranjan

Original Message:
Sent: Mon September 18, 2023 10:22 AM
From: Mi Wang
Subject: Error during installing xgboost

Hi Ranjan,

This is actually the continued discussion regarding issues with xgboost package installation (re: previous thread "Issue with C / C++ Compiling during the PIP installation for Xgboost"). We have now upgraded to Python 3.9 eco system. I have tried the installation of xgboost 1.7.5 the way you did the installation: 1) set the environment variables OBJECT_MODE, CC and CXX; 2) modified the include/xgboost/collective/socket.h file, replacing "#ifndef __PASE__" with "#if defined(__PASE__) && !defined(AIX)". But I got the following error message. Can you please help?

Thanks

Mi

        ..........

        File "/opt/freeware/lib64/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
          _build_ext.build_ext.build_extensions(self)
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/opt/freeware/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 206, in build_extension
          self.build_cmake_extension()
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 154, in build_cmake_extension
          os.path.join(CURRENT_DIR, os.path.pardir, 'lib', lib_name()))
        File "/tmp/pip-install-m1nm1aqx/xgboost_fde5d35a81024fff9cce8bb9ca0d86c5/setup.py", line 49, in lib_name
          return name
      UnboundLocalError: local variable 'name' referenced before assignment
      [end of output]

------------------------------
Mi Wang
------------------------------