Open Source Development

Power Open Source Development

Explore the open source tools and capabilities for building and deploying modern applications on IBM Power platforms including AIX, IBM i, and Linux.


#Power


#Power


#Servers
#Opensource
 View Only

  • 1.  Unable to install numpy with default Python 3.9 installation on AIX 7.3

    Posted 04/23/24 10:00 AM

    I am using Python 3.9 out of the box from AIX 7.3 :
    [root@CUNIM01:/] # which python3
    /usr/bin/python3
    [root@CUNIM01:/] # ll /usr/bin/python3
    0 lrwxrwxrwx 1 root system 30 Dec 11 15:10 /usr/bin/python3 -> /usr/opt/python3/bin/python3.9
    [root@CUNIM01:/] # lslpp -w /usr/opt/python3/bin/python3.9
    File Fileset Type
    ----------------------------------------------------------------------------
    /usr/opt/python3/bin/python3.9 python3.9.base File
    [root@CUNIM01:/] # lslpp -l python3\*
    Fileset Level State Description
    ----------------------------------------------------------------------------
    Path: /usr/lib/objrepos
    python3.9.base 3.9.17.1 COMMITTED Python 3.9 64-bit binary
    distribution

    Path: /etc/objrepos
    python3.9.base 3.9.17.1 COMMITTED Python 3.9 64-bit binary
    distribution

    I've created a new virtual python environment :
    [root@CUMPBSV1:/dwhouse/python] # python3 -m venv $PWD
    [root@CUMPBSV1:/dwhouse/python] # . bin/activate
    (python) [root@CUMPBSV1:/dwhouse/python] # python --version
    Python 3.9.18

    When trying to install numpy, I have the following error :
    (python) [root@CUMPBSV1:/dwhouse/python] # python -m pip install --proxy=http://proxysrvd:3128 numpy
    Collecting numpy
    Using cached numpy-1.26.4.tar.gz (15.8 MB)
    Installing build dependencies ... error
    error: subprocess-exited-with-error

    pip subprocess to install build dependencies did not run successfully.
    exit code: 1

    [10 lines of output]
    WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0xa00000001567fa0>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/cython/
    WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0xa000000015700d0>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/cython/
    WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0xa00000001570fd0>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/cython/
    WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0xa00000001570f10>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/cython/
    WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0xa000000015a3310>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/cython/
    ERROR: Could not find a version that satisfies the requirement Cython<3.1,>=0.29.34 (from versions: none)
    ERROR: No matching distribution found for Cython<3.1,>=0.29.34

    [notice] A new release of pip is available: 23.3.1 -> 24.0
    [notice] To update, run: pip install --upgrade pip
    [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
    error: subprocess-exited-with-error

    pip subprocess to install build dependencies did not run successfully.
    exit code: 1

    See above for output.

    note: This error originates from a subprocess, and is likely not a problem with pip.

    [notice] A new release of pip is available: 23.3.1 -> 24.0
    [notice] To update, run: pip install --upgrade pip
    (python) [root@CUMPBSV1:/dwhouse/python] # python -m pip list --proxy=http://proxysrvd:3128 Cython
    Package Version
    ---------- -------
    pip 23.3.1
    setuptools 58.1.0

    [notice] A new release of pip is available: 23.3.1 -> 24.0
    [notice] To update, run: pip install --upgrade pip

    I installed both rpms python3-Cython and python3.9-Cython but the result is still the same.

    Do you have and idea on how to install the Cython dependency for numpy in my environment ?

    Regards,

    Marc Baguelin



    ------------------------------
    Marc Baguelin
    ------------------------------

    #AIXOpenSource


  • 2.  RE: Unable to install numpy with default Python 3.9 installation on AIX 7.3

    Posted 04/24/24 01:24 AM

    Hi Marc,

    I see you are using /usr/bin/python. 

    Any reason to not use /opt/freeware/bin/python which is the tool box python that is built with gcc?

    I would recommend you use the toolbox python. If you have dnf installed, you should see /opt/freeware/bin/python3. 

    We have numpy 1.26.2 in the toolbox and you can install it using command "dnf install python3-numpy"

    If you wish to build and install numpy on your own (The latest version - 1.26.4) then here is how you can do it:-

    export CXX="g++ -pthread"
    export CXXFLAGS=-maix64
    export OBJECT_MODE=64
    export LDFLAGS=-maix64
    export CC="gcc -pthread"
    export CFLAGS=-maix64
    pip3.9 install numpy --root=$PWD/install_dir -v --ignore-installed

    I succeeded in doing so. 

    Successfully built numpy
    Installing collected packages: numpy
      Creating /numpy-1.26.4/install_dir/opt/freeware/bin
      changing mode of /numpy-1.26.4/install_dir/opt/freeware/bin/f2py to 755
      WARNING: The script f2py is installed in '/numpy-1.26.4/install_dir/opt/freeware/bin' which is not on PATH.
      Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
    Successfully installed numpy-1.26.4

    Try it and let us know if something goes wrong or if it works. Please change the install directory in your command to wherever you want to. 



    ------------------------------
    Aditya Kamath
    ------------------------------



  • 3.  RE: Unable to install numpy with default Python 3.9 installation on AIX 7.3

    Posted 04/24/24 03:52 AM

    Hi Marc,

    first of all you have standard Python3 3.9.17 on AIX 7.3 but when you created virtual environment, you have Python3 3.9.18. It means you have another Python installation on your box and you use it instead of the standard AIX Python. It is not bad and I agree with Aditya, you shoud use /opt/freeware/bin/python3 for what you are doing. Otherwise you must have IBM XL C/C++ compiler on your box to compile numpy.

    The second and the most important problem with pip doesn't have anything to do with Python but with your environment. The error is "connection timed out" and as for me it is your proxy server (which definitely want to check for viruses in each downloaded file) the reason for the failure. Check with your network guys and the system administrator what they could do.



    ------------------------------
    Andrey Klyachkin

    https://www.power-devops.com
    ------------------------------



  • 4.  RE: Unable to install numpy with default Python 3.9 installation on AIX 7.3

    Posted 04/24/24 09:16 AM

    Thank you Aditya and Andrey for your answers.

    I thought using the Python 3.9 published with AIX 7.3 would be better.

    I managed to install numpy exporting the HTTPS_PROXY system variable to the same proxy specified with the pip command and it worked.

    I'm now facing a problem installing pandas :

    (python) [root@CUMPBSV1:/dwhouse/python] # python -m pip --proxy http://proxysrvd:3128 install pandas
    Collecting pandas
      Using cached pandas-2.2.2.tar.gz (4.4 MB)
      Installing build dependencies ... done
      Getting requirements to build wheel ... done
      Preparing metadata (pyproject.toml) ... error
      error: subprocess-exited-with-error

      Preparing metadata (pyproject.toml) did not run successfully.
      exit code: 2

      [55 lines of output]
      + meson setup /tmp/pip-install-ysh9iza3/pandas_76ea6c2ff584440c823836fdfa261c0c /tmp/pip-install-ysh9iza3/pandas_76ea6c2ff584440c823836fdfa261c0c/.mesonpy-sb2eir_b/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --vsenv --native-file=/tmp/pip-install-ysh9iza3/pandas_76ea6c2ff584440c823836fdfa261c0c/.mesonpy-sb2eir_b/build/meson-python-native-file.ini
      Traceback (most recent call last):
        File "/tmp/pip-build-env-li2gcnpk/overlay/lib/python3.9/site-packages/mesonbuild/mesonmain.py", line 194, in run
          return options.run_func(options)
        File "/tmp/pip-build-env-li2gcnpk/overlay/lib/python3.9/site-packages/mesonbuild/msetup.py", line 358, in run
          app.generate()
        File "/tmp/pip-build-env-li2gcnpk/overlay/lib/python3.9/site-packages/mesonbuild/msetup.py", line 183, in generate
          return self._generate(env, capture, vslite_ctx)
        File "/tmp/pip-build-env-li2gcnpk/overlay/lib/python3.9/site-packages/mesonbuild/msetup.py", line 253, in _generate
          captured_compile_args = intr.backend.generate(capture, vslite_ctx)
        File "/tmp/pip-build-env-li2gcnpk/overlay/lib/python3.9/site-packages/mesonbuild/backend/ninjabackend.py", line 616, in generate
          self.generate_rules()
        File "/tmp/pip-build-env-li2gcnpk/overlay/lib/python3.9/site-packages/mesonbuild/backend/ninjabackend.py", line 1308, in generate_rules
          self.generate_dynamic_link_rules()
        File "/tmp/pip-build-env-li2gcnpk/overlay/lib/python3.9/site-packages/mesonbuild/backend/ninjabackend.py", line 2390, in generate_dynamic_link_rules
          cmdlist = compiler.get_command_to_archive_shlib()
        File "/tmp/pip-build-env-li2gcnpk/overlay/lib/python3.9/site-packages/mesonbuild/compilers/compilers.py", line 948, in get_command_to_archive_shlib
          return self.linker.get_command_to_archive_shlib()
      AttributeError: 'NoneType' object has no attribute 'get_command_to_archive_shlib'
      The Meson build system
      Version: 1.2.1
      Source dir: /tmp/pip-install-ysh9iza3/pandas_76ea6c2ff584440c823836fdfa261c0c
      Build dir: /tmp/pip-install-ysh9iza3/pandas_76ea6c2ff584440c823836fdfa261c0c/.mesonpy-sb2eir_b/build
      Build type: native build
      Project name: pandas
      Project version: 2.2.2
      C compiler for the host machine: gcc -pthread (gcc 10.3.0 "gcc (GCC) 10.3.0")
      C linker for the host machine: gcc -pthread ld.aix 7.3.2
      C++ compiler for the host machine: g++ -pthread (gcc 10.3.0 "g++ (GCC) 10.3.0")
      C++ linker for the host machine: g++ -pthread ld.aix 7.3.2
      Cython compiler for the host machine: cython (cython 3.0.5)
      Host machine cpu family: ppc
      Host machine cpu: powerpc
      Program python found: YES (/dwhouse/python/bin/python)
      Found pkg-config: /usr/bin/pkg-config (0.29.2)
      Run-time dependency python found: YES 3.9
      Build targets in project: 53

      pandas 2.2.2

        User defined options
          Native files: /tmp/pip-install-ysh9iza3/pandas_76ea6c2ff584440c823836fdfa261c0c/.mesonpy-sb2eir_b/build/meson-python-native-file.ini
          buildtype   : release
          vsenv       : True
          b_ndebug    : if-release
          b_vscrt     : md

      Found ninja-1.11.1 at /opt/freeware/bin/ninja

      Visual Studio environment is needed to run Ninja. It is recommended to use Meson wrapper:
      /tmp/pip-build-env-li2gcnpk/overlay/bin/meson compile -C .

      ERROR: Unhandled python exception

          This is a Meson bug and should be reported!
      [end of output]

      note: This error originates from a subprocess, and is likely not a problem with pip.
    error: metadata-generation-failed

    Encountered error while generating package metadata.

    See above for output.

    note: This is an issue with the package mentioned above, not pip.
    hint: See above for details.

    Regards,

    Marc Baguelin



    ------------------------------
    Marc Baguelin
    ------------------------------

    Original Message


  • 5.  RE: Unable to install numpy with default Python 3.9 installation on AIX 7.3

    Posted 04/24/24 09:29 AM
    Edited by Aditya Kamath 04/24/24 09:34 AM

    Hi Marc,

    Yeah this is a bug we have fixed in meson and will be resolved if pandas upgrade meson. 

    But we have a work around. 

    By the way we will release toolbox pandas soon. 

    Having said that here is how you can do it. 

    install meson using command "dnf install meson"

    Then run

    export CXX="g++ -pthread"
    export CXXFLAGS=-maix64
    export OBJECT_MODE=64
    export CC="gcc -pthread"
    export CFLAGS=-maix64
    export LDFLAGS="-lm"
    pip3.9 install pandas --root=$PWD/install_dir -I --no-deps --no-build-isolation

    This should install. Let me know if any problems



    ------------------------------
    Aditya Kamath
    ------------------------------

    Original Message


  • 6.  RE: Unable to install numpy with default Python 3.9 installation on AIX 7.3

    Posted 04/24/24 10:04 AM

    Hi Aditya,

    I've installed meson using dnf, the error is not the same now :

    (python) [root@CUMPBSV1:/dwhouse/python] # rpm -q meson
    meson-1.2.1-1.noarch

    (python) [root@CUMPBSV1:/dwhouse/python] # export CXX="g++ -pthread"
    (python) [root@CUMPBSV1:/dwhouse/python] # export CXXFLAGS=-maix64
    (python) [root@CUMPBSV1:/dwhouse/python] # export OBJECT_MODE=64
    (python) [root@CUMPBSV1:/dwhouse/python] # export CC="gcc -pthread"
    (python) [root@CUMPBSV1:/dwhouse/python] # export CFLAGS=-maix64
    (python) [root@CUMPBSV1:/dwhouse/python] # export LDFLAGS="-lm"

    (python) [root@CUMPBSV1:/dwhouse/python] # python -m pip --proxy http://proxysrvd:3128 install . --root=$PWD/install_dir -I --no-deps --no-build-isolation pandas
    ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.
    (python) [root@CUMPBSV1:/dwhouse/python] # python -m pip --proxy http://proxysrvd:3128 install pandas --root=$PWD/install_dir -I --no-deps --no-build-isolation
    Collecting pandas
      Using cached pandas-2.2.2.tar.gz (4.4 MB)
      Preparing metadata (pyproject.toml) ... error
      error: subprocess-exited-with-error

      Preparing metadata (pyproject.toml) did not run successfully.
      exit code: 1

      [10 lines of output]
      + meson setup /tmp/pip-install-cat_hzrp/pandas_f899a0e1c65140358388264ce675f672 /tmp/pip-install-cat_hzrp/pandas_f899a0e1c65140358388264ce675f672/.mesonpy-t8nk21s5/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --vsenv --native-file=/tmp/pip-install-cat_hzrp/pandas_f899a0e1c65140358388264ce675f672/.mesonpy-t8nk21s5/build/meson-python-native-file.ini
      The Meson build system
      Version: 1.4.0
      Source dir: /tmp/pip-install-cat_hzrp/pandas_f899a0e1c65140358388264ce675f672
      Build dir: /tmp/pip-install-cat_hzrp/pandas_f899a0e1c65140358388264ce675f672/.mesonpy-t8nk21s5/build
      Build type: native build

      ../../meson.build:5:13: ERROR: Command `/dwhouse/python/bin/python /tmp/pip-install-cat_hzrp/pandas_f899a0e1c65140358388264ce675f672/generate_version.py --print` failed with status 1.

      A full log can be found at /tmp/pip-install-cat_hzrp/pandas_f899a0e1c65140358388264ce675f672/.mesonpy-t8nk21s5/build/meson-logs/meson-log.txt
      [end of output]

      note: This error originates from a subprocess, and is likely not a problem with pip.
    error: metadata-generation-failed

    Encountered error while generating package metadata.

    See above for output.

    note: This is an issue with the package mentioned above, not pip.
    hint: See above for details.

    Regards,

    Marc Baguelin



    ------------------------------
    Marc Baguelin
    ------------------------------

    Original Message


  • 7.  RE: Unable to install numpy with default Python 3.9 installation on AIX 7.3

    Posted 04/29/24 08:19 AM

    Hi Marc, 

    We will release a new version of pandas to the tool box soon. These bugs require pandas community to update their meson. We will inform you once we have done so.

    In the mean time, can you try the existing version using the command 'dnf install python3-pandas' and let me know if it works. 



    ------------------------------
    Aditya Kamath
    ------------------------------

    Original Message


Related Content