Original Message:
Sent: Tue December 10, 2024 05:15 AM
From: Manish Anand
Subject: Numpy Failing to import with python3.9
Thanks a lot Aditya, finally it worked :).
Clearing the cache and reinstalling from download tar worked. Thanks again for all your help.
------------------------------
Manish Anand
Original Message:
Sent: Fri December 06, 2024 05:01 AM
From: Aditya Kamath
Subject: Numpy Failing to import with python3.9
Hi Manish ,
Before I could think about other stuff, can you do one more try? In my environment, I am able to install and use Pandas and Numpy.
> Collecting numpy>=2.0
> Using cached numpy-2.0.2-cp39-cp39-aix_7302_2345_64.whl
> Collecting versioneer[toml]
I believe your Pandas has been compiled with the 32-bit cache numpy.
I think we must first clear your cache so things can be clean.
pip3.9 cache dir
pip3.9 cache purge
Then you can uninstall Pandas via pip3.9 uninstall pandas.
Then try to install pandas with the same steps [version 2.2.3 with meson >= 1.5]
You should succeed.
Created wheel for pandas: filename=pandas-2.2.3-cp39-cp39-aix_7205_2419_64.whl size=54606068 sha256=9d8b7880762200278475a632296bdaf723dbb8714006ae8a0064eec83b0312cd
Stored in directory: /.cache/pip/wheels/f6/13/f4/6214eee3bb6b1fea00898ed8825a1210d9be65f9c587752c38
Successfully built pandas
Installing collected packages: pandas
# python3
Python 3.9.20 (main, Sep 27 2024, 03:16:26)
[GCC 10.3.0] on aix
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> import pandas
>>>
------------------------------
Aditya Kamath
Original Message:
Sent: Fri December 06, 2024 03:53 AM
From: Manish Anand
Subject: Numpy Failing to import with python3.9
Thanks Aditya. Unfortunately this also throws error (below). I installed meson version 1.5.1 and also changed in pyproject.toml but again its failed.
Can we have solution for this meson bug please? No way its getting fixed because of which Pandas not getting installed/working :(.
#/opt/freeware/bin/python3 -m pip install . -v
Using pip 24.3.1 from /opt/freeware/lib/python3.9/site-packages/pip (python 3.9)
Looking in indexes: https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/simple
Processing /root/pandas-2.2.3
Running command pip subprocess to install build dependencies
Using pip 24.3.1 from /opt/freeware/lib/python3.9/site-packages/pip (python 3.9)
Looking in indexes: https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/simple
Collecting meson-python==0.13.1
Using cached https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/packages/meson-python/0.13.1/meson_python-0.13.1-py3-none-any.whl (79 kB)
Collecting meson==1.5.1
Using cached https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/packages/meson/1.5.1/meson-1.5.1-py3-none-any.whl (966 kB)
Collecting wheel
Using cached https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/packages/wheel/0.45.1/wheel-0.45.1-py3-none-any.whl (72 kB)
Collecting Cython~=3.0.5
Using cached https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/packages/cython/3.0.11/Cython-3.0.11-py2.py3-none-any.whl (1.2 MB)
Link requires a different Python (3.9.17 not in: '>=3.10'): https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/packages/numpy/2.1.0/numpy-2.1.0.tar.gz#sha256=7dc90da0081f7e1da49ec4e398ede6a8e9cc4f5ebe5f9e06b443ed889ee9aaa2 (from https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/simple/numpy/) (requires-python:>=3.10)
Link requires a different Python (3.9.17 not in: '>=3.10'): https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/packages/numpy/2.1.1/numpy-2.1.1.tar.gz#sha256=d0cf7d55b1051387807405b3898efafa862997b4cba8aa5dbe657be794afeafd (from https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/simple/numpy/) (requires-python:>=3.10)
Link requires a different Python (3.9.17 not in: '>=3.10'): https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/packages/numpy/2.1.2/numpy-2.1.2.tar.gz#sha256=13532a088217fa624c99b843eeb54640de23b3414b14aa66d023805eb731066c (from https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/simple/numpy/) (requires-python:>=3.10)
Link requires a different Python (3.9.17 not in: '>=3.10'): https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/packages/numpy/2.1.3/numpy-2.1.3.tar.gz#sha256=aa08e04e08aaf974d4458def539dece0d28146d866a39da5639596f4921fd761 (from https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/simple/numpy/) (requires-python:>=3.10)
Collecting numpy>=2.0
Using cached numpy-2.0.2-cp39-cp39-aix_7302_2345_64.whl
Collecting versioneer[toml]
Using cached https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/packages/versioneer/0.29/versioneer-0.29-py3-none-any.whl (46 kB)
Collecting pyproject-metadata>=0.7.1 (from meson-python==0.13.1)
Using cached https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/packages/pyproject-metadata/0.8.0/pyproject_metadata-0.8.0-py3-none-any.whl (7.5 kB)
Collecting tomli>=1.0.0 (from meson-python==0.13.1)
Using cached https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/packages/tomli/2.2.1/tomli-2.2.1-py3-none-any.whl (14 kB)
Collecting packaging>=19.0 (from pyproject-metadata>=0.7.1->meson-python==0.13.1)
Using cached https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/packages/packaging/24.2/packaging-24.2-py3-none-any.whl (65 kB)
Installing collected packages: wheel, versioneer, tomli, packaging, numpy, meson, Cython, pyproject-metadata, meson-python
Creating /tmp/pip-build-env-bltwv3d5/overlay/bin
changing mode of /tmp/pip-build-env-bltwv3d5/overlay/bin/wheel to 755
changing mode of /tmp/pip-build-env-bltwv3d5/overlay/bin/versioneer to 755
changing mode of /tmp/pip-build-env-bltwv3d5/overlay/bin/f2py to 755
changing mode of /tmp/pip-build-env-bltwv3d5/overlay/bin/numpy-config to 755
changing mode of /tmp/pip-build-env-bltwv3d5/overlay/bin/meson to 755
changing mode of /tmp/pip-build-env-bltwv3d5/overlay/bin/cygdb to 755
changing mode of /tmp/pip-build-env-bltwv3d5/overlay/bin/cython to 755
changing mode of /tmp/pip-build-env-bltwv3d5/overlay/bin/cythonize to 755
Successfully installed Cython-3.0.11 meson-1.5.1 meson-python-0.13.1 numpy-2.0.2 packaging-24.2 pyproject-metadata-0.8.0 tomli-2.2.1 versioneer-0.29 wheel-0.45.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
Installing build dependencies ... done
Running command Getting requirements to build wheel
Getting requirements to build wheel ... done
Running command Preparing metadata (pyproject.toml)
+ meson setup /root/pandas-2.2.3 /root/pandas-2.2.3/.mesonpy-8f8_661y/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --vsenv --native-file=/root/pandas-2.2.3/.mesonpy-8f8_661y/build/meson-python-native-file.ini
The Meson build system
Version: 1.5.1
Source dir: /root/pandas-2.2.3
Build dir: /root/pandas-2.2.3/.mesonpy-8f8_661y/build
Build type: native build
Project name: pandas
Project version: 2.2.3
C compiler for the host machine: gcc (gcc 10.3.0 "gcc (GCC) 10.3.0")
C linker for the host machine: gcc ld.aix 7.3.2
C++ compiler for the host machine: c++ (gcc 10.3.0 "c++ (GCC) 10.3.0")
C++ linker for the host machine: c++ ld.aix 7.3.2
Cython compiler for the host machine: cython (cython 3.0.11)
Host machine cpu family: ppc
Host machine cpu: powerpc
Program python found: YES (/opt/freeware/bin/python3)
../../pandas/meson.build:1:15: ERROR: Command `/opt/freeware/bin/python3 -c '
import os
import numpy as np
try:
# Check if include directory is inside the pandas dir
# e.g. a venv created inside the pandas dir
# If so, convert it to a relative path
incdir = os.path.relpath(np.get_include())
except Exception:
incdir = np.get_include()
print(incdir)
'` failed with status 1.
A full log can be found at /root/pandas-2.2.3/.mesonpy-8f8_661y/build/meson-logs/meson-log.txt
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /opt/freeware/bin/python3 /opt/freeware/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp2855os_t
cwd: /root/pandas-2.2.3
Preparing metadata (pyproject.toml) ... error
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
#
#/opt/freeware/bin/python3 -m pip list | grep -i meson
meson 1.5.1
meson-python 0.15.0
#
I also tried installing lower version of Pandas which got installed successfully but while importing it again had error:
#/opt/freeware/bin/python3 -m pip install pandas===2.0.2
Looking in indexes: https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/simple
Collecting pandas===2.0.2
Using cached https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/packages/pandas/2.0.2/pandas-2.0.2.tar.gz (5.3 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy>=1.20.3 in /opt/freeware/lib64/python3.9/site-packages (from pandas===2.0.2) (2.0.2)
Requirement already satisfied: python-dateutil>=2.8.2 in /opt/freeware/lib/python3.9/site-packages (from pandas===2.0.2) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /opt/freeware/lib/python3.9/site-packages (from pandas===2.0.2) (2024.2)
Collecting tzdata>=2022.1 (from pandas===2.0.2)
Using cached https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/packages/tzdata/2024.2/tzdata-2024.2-py2.py3-none-any.whl (346 kB)
Requirement already satisfied: six>=1.5 in /opt/freeware/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pandas===2.0.2) (1.13.0)
Building wheels for collected packages: pandas
Building wheel for pandas (pyproject.toml) ... done
Created wheel for pandas: filename=pandas-2.0.2-cp39-cp39-aix_7302_2345_64.whl size=28743557 sha256=948f0d7353976223952a0c547815a2d65222ed2f80a2f9cc8215c0a6397fb43b
Stored in directory: /root/.cache/pip/wheels/21/94/53/20a8a17b7ba2b085a255b38953cf26cced6ee1fd822151e621
Successfully built pandas
Installing collected packages: tzdata, pandas
Successfully installed pandas-2.0.2 tzdata-2024.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
#
#/opt/freeware/bin/python3
Python 3.9.17 (main, Jul 5 2023, 06:17:22)
[GCC 10.3.0] on aix
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import numpy
>>> import pandas
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/freeware/lib64/python3.9/site-packages/pandas/__init__.py", line 22, in <module>
from pandas.compat import is_numpy_dev as _is_numpy_dev # pyright: ignore # noqa:F401
File "/opt/freeware/lib64/python3.9/site-packages/pandas/compat/__init__.py", line 25, in <module>
from pandas.compat.numpy import (
File "/opt/freeware/lib64/python3.9/site-packages/pandas/compat/numpy/__init__.py", line 4, in <module>
from pandas.util.version import Version
File "/opt/freeware/lib64/python3.9/site-packages/pandas/util/__init__.py", line 2, in <module>
from pandas.util._decorators import ( # noqa:F401
File "/opt/freeware/lib64/python3.9/site-packages/pandas/util/_decorators.py", line 14, in <module>
from pandas._libs.properties import cache_readonly
File "/opt/freeware/lib64/python3.9/site-packages/pandas/_libs/__init__.py", line 13, in <module>
from pandas._libs.interval import Interval
File "pandas/_libs/interval.pyx", line 1, in init pandas._libs.interval
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
>>>
Thanks for your help.
------------------------------
Manish Anand
Original Message:
Sent: Thu December 05, 2024 07:11 AM
From: Aditya Kamath
Subject: Numpy Failing to import with python3.9
You can change 1.6.0 to 1.5.1 and try. It will work since that version also has the fix.
------------------------------
Aditya Kamath
Original Message:
Sent: Thu December 05, 2024 05:35 AM
From: Manish Anand
Subject: Numpy Failing to import with python3.9
Hi Aditya,
Many Thanks for helping out. I removed the RPM version of 'meson' version 1.2.1 and installed from artifactory using pip, downloaded pandas-2.2.3.tar.gz and made the changes to file vi pyproject.toml, changing "meson==1.2.1" to "meson==1.6.0" but now during install it complains that its not able to find version 1.6.0.
Please advise how to get this meson this sorted please.
#/opt/freeware/bin/python3 -m pip install . -v
Using pip 24.3.1 from /opt/freeware/lib/python3.9/site-packages/pip (python 3.9)
Looking in indexes: https:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/api/pypi/pypi/simple
Processing /root/pandas-2.2.3
Running command pip subprocess to install build dependencies
Using pip 24.3.1 from /opt/freeware/lib/python3.9/site-packages/pip (python 3.9)
Looking in indexes: https://xxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/simple
Collecting meson-python==0.13.1
Using cached https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/packages/meson-python/0.13.1/meson_python-0.13.1-py3-none-any.whl (79 kB)
ERROR: Could not find a version that satisfies the requirement meson==1.6.0 (from versions: 0.55.3, 0.62.2, 0.63.3, 1.1.0, 1.2.1, 1.4.0, 1.5.0, 1.5.1)
ERROR: No matching distribution found for meson==1.6.0
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /opt/freeware/bin/python3 /opt/freeware/lib/python3.9/site-packages/pip/__pip-runner__.py install --ignore-installed --no-user --prefix /tmp/pip-build-env-ases3hiv/overlay --no-warn-script-location --disable-pip-version-check --target '' -v --no-binary :none: --only-binary :none: -i https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxx/artifactory/api/pypi/pypi/simple -- meson-python==0.13.1 meson==1.6.0 wheel 'Cython~=3.0.5' 'numpy>=2.0' 'versioneer[toml]'
cwd: [inherit]
Installing build dependencies ... error
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
#
------------------------------
Manish Anand
Original Message:
Sent: Thu December 05, 2024 03:57 AM
From: Aditya Kamath
Subject: Numpy Failing to import with python3.9
Hi Manish,
Glad that Numpy works.
So this bug came from Meson and it has been fixed. But Pandas downloads the buggy meson and builds.
I will work with the Pandas community to help them upgrade meson in Pandas.
Since your requirement is to build a wheel, we can do a workaround.
wget https://github.com/pandas-dev/pandas/releases/download/v2.2.3/pandas-2.2.3.tar.gz
tar -xf pandas-2.2.3.tar.gz
cd pandas-2.2.3
Here in pyproject.toml file line number 6, kindly change "meson==1.2.1" to "meson==1.6.0"
vi pyproject.toml
Then it is the regular steps,
export CXX="g++ -pthread"export CXXFLAGS=-maix64export OBJECT_MODE=64export CC="gcc -pthread"export CFLAGS=-maix64export LDFLAGS="-lm -Wl,-blibpath:/opt/freeware/lib/pthread:/opt/freeware/lib64:/opt/freeware/lib:/usr/lib:/lib"
pip3.9 install . -v
And this should install pandas.
------------------------------
Aditya Kamath
Original Message:
Sent: Thu December 05, 2024 01:24 AM
From: Manish Anand
Subject: Numpy Failing to import with python3.9
Many Thanks Aditya, it worked. I too didn't noticed that it was getting installed from cache. Thanks again for the help :).
One more thing if you are aware and can help please. I am getting the below error while installing 'pandas'. Pandas has numpy dependency and was thinking pandas install should be smooth. I have installed meson RPM from IBM OSS as well.
#/opt/freeware/bin/python3
Python 3.9.17 (main, Jul 5 2023, 06:17:22)
[GCC 10.3.0] on aix
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import numpy
>>>
#/opt/freeware/bin/python3 -m pip install pandas --no-cache-dir
Looking in indexes: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Collecting pandas
Downloading https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/packages/pandas/2.2.3/pandas-2.2.3.tar.gz (4.4 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.4/4.4 MB 12.6 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... error
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [31 lines of output]
+ meson setup /tmp/pip-install-3rm3e3d6/pandas_5dbb6ffdb19e4e30a7d88ce44cd258a0 /tmp/pip-install-3rm3e3d6/pandas_5dbb6ffdb19e4e30a7d88ce44cd258a0/.mesonpy-94hewn5d/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --vsenv --native-file=/tmp/pip-install-3rm3e3d6/pandas_5dbb6ffdb19e4e30a7d88ce44cd258a0/.mesonpy-94hewn5d/build/meson-python-native-file.ini
The Meson build system
Version: 1.2.1
Source dir: /tmp/pip-install-3rm3e3d6/pandas_5dbb6ffdb19e4e30a7d88ce44cd258a0
Build dir: /tmp/pip-install-3rm3e3d6/pandas_5dbb6ffdb19e4e30a7d88ce44cd258a0/.mesonpy-94hewn5d/build
Build type: native build
Project name: pandas
Project version: 2.2.3
C compiler for the host machine: gcc -pthread (gcc 10.3.0 "gcc (GCC) 10.3.0")
C linker for the host machine: gcc -pthread ld.aix 7.3.2
C++ compiler for the host machine: g++ -pthread (gcc 10.3.0 "g++ (GCC) 10.3.0")
C++ linker for the host machine: g++ -pthread ld.aix 7.3.2
Cython compiler for the host machine: cython (cython 3.0.11)
Host machine cpu family: ppc
Host machine cpu: powerpc
Program python found: YES (/opt/freeware/bin/python3)
../../pandas/meson.build:1:15: ERROR: Command `/opt/freeware/bin/python3 -c '
import os
import numpy as np
try:
# Check if include directory is inside the pandas dir
# e.g. a venv created inside the pandas dir
# If so, convert it to a relative path
incdir = os.path.relpath(np.get_include())
except Exception:
incdir = np.get_include()
print(incdir)
'` failed with status 1.
A full log can be found at /tmp/pip-install-3rm3e3d6/pandas_5dbb6ffdb19e4e30a7d88ce44cd258a0/.mesonpy-94hewn5d/build/meson-logs/meson-log.txt
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
#
#rpm -qa | grep -i meson
meson-1.2.1-2.noarch
#
------------------------------
Manish Anand
Original Message:
Sent: Wed December 04, 2024 10:00 PM
From: Aditya Kamath
Subject: Numpy Failing to import with python3.9
Okay Manish.
So this is interesting. I spent sometime trying to understand what might have happened.
>Looking in indexes: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>Collecting numpy
> Using cached numpy-2.0.2-cp39-cp39-aix_7302_2345_64.whl
>Installing collected packages: numpy
>Successfully installed numpy-2.0.2
Checkout this 3rd message. It says using cached numpy wheel. :)
So even though you tried to install 64 bit numpy freshly pip took your 32 bit numpy you tried first time from cache and installed.
Hence you saw this:
#file /opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so
/opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so: executable (RISC System/6000) or object module not stripped
#
This is indeed not 64 bit.
So what we can do is
/opt/freeware/bin/python3 -m pip uninstall numpy
export CXX="g++ -pthread"
export CXXFLAGS=-maix64
export OBJECT_MODE=64
export LDFLAGS=-maix64
export CC="gcc -pthread"
export CFLAGS=-maix64
/opt/freeware/bin/python3 -m pip install numpy --no-cache-dir
This --no-cache-dir will ignore the wheel in your cache and install 64 bit numpy freshly.
In fact, I reproduced your error as well and this fix will resolve your error for sure :)
Have a nice day ahead.
------------------------------
Aditya Kamath
Original Message:
Sent: Wed December 04, 2024 01:49 AM
From: Manish Anand
Subject: Numpy Failing to import with python3.9
Hi Aditya,
Thanks for looking into this, now I get the issue. But I tried the same steps you gave again today and still facing the same issue (pasting full steps and output I got below). Do you think any other RPM installed on this host can cause issue or is it the numpy package itself which is an issue?
Output below:
#/opt/freeware/bin/python3 -m pip uninstall numpy
Found existing installation: numpy 2.0.2
Uninstalling numpy-2.0.2:
Would remove:
/opt/freeware/bin/f2py
/opt/freeware/bin/numpy-config
/opt/freeware/lib64/python3.9/site-packages/numpy-2.0.2.dist-info/*
/opt/freeware/lib64/python3.9/site-packages/numpy/*
Proceed (Y/n)? y
Successfully uninstalled numpy-2.0.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
# ls /opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so
/opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so not found
#export CXX="g++ -pthread"
#export CXXFLAGS=-maix64
#export OBJECT_MODE=64
#export LDFLAGS=-maix64
#export CC="gcc -pthread"
#export CFLAGS=-maix64
#/opt/freeware/bin/python3 -m pip install numpy
Looking in indexes: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Collecting numpy
Using cached numpy-2.0.2-cp39-cp39-aix_7302_2345_64.whl
Installing collected packages: numpy
Successfully installed numpy-2.0.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
#/opt/freeware/bin/python3
Python 3.9.17 (main, Jul 5 2023, 06:17:22)
[GCC 10.3.0] on aix
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
File "/opt/freeware/lib64/python3.9/site-packages/numpy/_core/__init__.py", line 23, in <module>
from . import multiarray
File "/opt/freeware/lib64/python3.9/site-packages/numpy/_core/multiarray.py", line 10, in <module>
from . import overrides
File "/opt/freeware/lib64/python3.9/site-packages/numpy/_core/overrides.py", line 8, in <module>
from numpy._core._multiarray_umath import (
ImportError: Could not load module /opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so.
System error: Exec format error
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/freeware/lib64/python3.9/site-packages/numpy/__init__.py", line 114, in <module>
from numpy.__config__ import show as show_config
File "/opt/freeware/lib64/python3.9/site-packages/numpy/__config__.py", line 4, in <module>
from numpy._core._multiarray_umath import (
File "/opt/freeware/lib64/python3.9/site-packages/numpy/_core/__init__.py", line 49, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.9 from "/opt/freeware/bin/python3"
* The NumPy version is: "2.0.2"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: Could not load module /opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so.
System error: Exec format error
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/freeware/lib64/python3.9/site-packages/numpy/__init__.py", line 119, in <module>
raise ImportError(msg) from e
ImportError: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python interpreter from there.
>>>
# ls /opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so
/opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so
#file /opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so
/opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so: executable (RISC System/6000) or object module not stripped
#
------------------------------
Manish Anand
Original Message:
Sent: Tue December 03, 2024 07:10 AM
From: Aditya Kamath
Subject: Numpy Failing to import with python3.9
Yeah, so that is the error. Python is 64-bit, but you have a 32-bit shared object. So while importing your system is complaining that your exec is not in the correct format.
We should ideally see something like this below where it clearly says the shared object is a 64-bit one.
file /opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so
/opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so: 64-bit XCOFF executable or object module not stripped
For example, grep, in 32 it just says executable; but for 64-bit, it says a 64-bit executable.
# file /opt/freeware/bin/grep_32
/opt/freeware/bin/grep_32: executable (RISC System/6000 V3.1) or obj module
# file /opt/freeware/bin/grep_64
/opt/freeware/bin/grep_64: 64-bit XCOFF executable or object module
Are you sure that you have uninstalled and then installed?
Can you give it a try again??
/opt/freeware/bin/python3 -m pip uninstall numpy
Here all your files should have gone. If you do ls /opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so it should not be there.
Then do,
export CXX="g++ -pthread"
export CXXFLAGS=-maix64
export OBJECT_MODE=64
export LDFLAGS=-maix64
export CC="gcc -pthread"
export CFLAGS=-maix64
/opt/freeware/bin/python3 -m pip install numpy.
I was able to do import post that.
Successfully installed numpy-2.0.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
# python3
Python 3.11.9 (main, Apr 28 2024, 11:03:03) [GCC 10.3.0] on aix
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>>
Kindly check and let me know what happens. Once installed, please check if the below output is the same.
file /opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so
/opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so: 64-bit XCOFF executable or object module not stripped
------------------------------
Aditya Kamath
Original Message:
Sent: Tue December 03, 2024 06:24 AM
From: Manish Anand
Subject: Numpy Failing to import with python3.9
Hi Aditya,
Here is the output:
#file /opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so
/opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so: executable (RISC System/6000) or object module not stripped
#
Thanks for the help.
------------------------------
Manish Anand
Original Message:
Sent: Tue December 03, 2024 06:07 AM
From: Aditya Kamath
Subject: Numpy Failing to import with python3.9
What is the output of "file /opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so" ?
> /opt/freeware/bin/python3 -m pip install numpy
This should be okay if you have used the exports before.
------------------------------
Aditya Kamath
Original Message:
Sent: Tue December 03, 2024 05:31 AM
From: Manish Anand
Subject: Numpy Failing to import with python3.9
Thanks for the reply Aditya. I tried the exports you have mentioned, uninstalled numpy and reinstalled which is fine but import numpy is still failing with same error.
I am using the below to install numpy. I am using python3 installed from IBM OSS, if I use 'pip3.9 install numpy' it uses defauly python3 which come with AIX 73 OS which we are not using.
# /opt/freeware/bin/python3 -m pip install numpy
Please let me know if this is ok because I am still getting the same error. I tried using 'pip3.9 install numpy' as well but import numpy still fails.
Numpy installed via 'dnf' works fine but we are looking to install using 'pip' and use the .whl package on other servers as well.
Thanks.
------------------------------
Manish Anand
Original Message:
Sent: Tue December 03, 2024 04:44 AM
From: Aditya Kamath
Subject: Numpy Failing to import with python3.9
Hi Manish,
There are several reasons why this could have happened.
Did you compile numpy for a 64 bit build??
Any way you can uninstall numpy using "pip3.9 uninstall numpy" and then,
use the below instructions to build numpy,
export CXX="g++ -pthread"
export CXXFLAGS=-maix64
export OBJECT_MODE=64
export LDFLAGS=-maix64
export CC="gcc -pthread"
export CFLAGS=-maix64
pip3.9 install numpy
One more thing is you can also install numpy via DNF. "dnf install python3-numpy python3.9-numpy*"
------------------------------
Aditya Kamath
Original Message:
Sent: Tue December 03, 2024 04:13 AM
From: Manish Anand
Subject: Numpy Failing to import with python3.9
Hi All,
I have installed python3.9 (on AIX 73) recently from the IBM OSS and using python3.9 installed numpy via 'pip' which worked fine. But when trying to import numpy it fails complaining "multiarray_umath.cpython-39" issue. I used below to install numpy using 'pip'. I have even tried install lower version on numpy but the error is same.
# /opt/freeware/bin/python3 -m pip install numpy
Pasting the full error I am getting while importing, if someone can help please. Thanks.
#/opt/freeware/bin/python3
Python 3.9.17 (main, Jul 5 2023, 06:17:22)
[GCC 10.3.0] on aix
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import numpy
Traceback (most recent call last):
File "/opt/freeware/lib64/python3.9/site-packages/numpy/_core/__init__.py", line 23, in <module>
from . import multiarray
File "/opt/freeware/lib64/python3.9/site-packages/numpy/_core/multiarray.py", line 10, in <module>
from . import overrides
File "/opt/freeware/lib64/python3.9/site-packages/numpy/_core/overrides.py", line 8, in <module>
from numpy._core._multiarray_umath import (
ImportError: Could not load module /opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so.
System error: Exec format error
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/freeware/lib64/python3.9/site-packages/numpy/__init__.py", line 114, in <module>
from numpy.__config__ import show as show_config
File "/opt/freeware/lib64/python3.9/site-packages/numpy/__config__.py", line 4, in <module>
from numpy._core._multiarray_umath import (
File "/opt/freeware/lib64/python3.9/site-packages/numpy/_core/__init__.py", line 49, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.9 from "/opt/freeware/bin/python3"
* The NumPy version is: "2.0.2"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: Could not load module /opt/freeware/lib64/python3.9/site-packages/numpy/_core/_multiarray_umath.cpython-39.so.
System error: Exec format error
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/freeware/lib64/python3.9/site-packages/numpy/__init__.py", line 119, in <module>
raise ImportError(msg) from e
ImportError: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python interpreter from there.
>>>
#gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/opt/freeware/libexec/gcc/powerpc-ibm-aix7.3.0.0/10/lto-wrapper
Target: powerpc-ibm-aix7.3.0.0
Configured with: ../gcc-10.3.0/configure --prefix=/opt/freeware --mandir=/opt/freeware/man --infodir=/opt/freeware/info --with-local-prefix=/opt/freeware --with-as=/usr/bin/as --with-ld=/usr/bin/ld --enable-languages=c,c++,fortran,go --enable-version-specific-runtime-libs --disable-nls --disable-libstdcxx-pch --disable-werror --enable-libstdcxx-filesystem-ts --with-gcc-major-version-only --program-suffix=-10 --with-cpu=default32 --host=powerpc-ibm-aix7.3.0.0
Thread model: aix
Supported LTO compression algorithms: zlib
gcc version 10.3.0 (GCC)
------------------------------
Manish Anand
------------------------------
#AIXOpenSource