Greetings team: I pass this on in case someone else hits upon the same issue as we had.

Recently the Development team raised a ticket as they were seeing the following "error" on an AIX 7.3.3.0 LPAR.

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):  File "<stdin>", line 1, in <module>ModuleNotFoundError: No module named 'pandas'>>>

Easy, just install "pandas" and we're all done.  Wrong!  Or so it was initially :-)

So, first things first, get "pandas" installed.

# dnf install python3.9-pandas. . . .Installed:  python3.9-pandas-2.2.3-1.ppc    openblas-0.3.28-1.ppc    python3.9-numpy-2.0.2-1.ppc    python3.9-pytz-2023.3-1.noarch    python3.9-tzdata-2023.3-1.noarch    libgfortran-1:10-2.ppc  libgfortran10-10.3.0-6.ppc

Now retry importing "pandas"

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):  File "<stdin>", line 1, in <module>ModuleNotFoundError: No module named 'pandas'>>>

Still no joy.  Upon further analysis it was necessary to set the PYTHONPATH environment variable to point to where "pandas" had been installed.

# rpm -ql python3.9-pandas/opt/freeware/doc/python3.9-pandas-2.2.3/opt/freeware/doc/python3.9-pandas-2.2.3/README.md/opt/freeware/lib64/python3.9/site-packages/pandas. . .# export PYTHONPATH=/opt/freeware/lib64/python3.9/site-packages# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/opt/freeware/lib64/python3.9/site-packages/pandas/__init__.py", line 19, in <module>    raise ImportError(ImportError: Unable to import required dependencies:pytz: No module named 'pytz'dateutil: No module named 'dateutil'>>>

Ok, so where are "pytz" and "dateutil" to be found?

# rpm -ql python3.9-pytz/opt/freeware/doc/python3.9-pytz-2023.3/opt/freeware/doc/python3.9-pytz-2023.3/README.rst/opt/freeware/lib/python3.9/site-packages/pytz. . .# rpm -ql python3.9-dateutil/opt/freeware/doc/python3.9-dateutil-2.8.2/opt/freeware/doc/python3.9-dateutil-2.8.2/README.rst/opt/freeware/lib/python3.9/site-packages/dateutil. . .# ls -la /opt/freeware/lib/python3.9/site-packages | grep pytz# ls -la /opt/freeware/lib/python3.9/site-packages | grep dateutildrwxr-xr-x    6 root     system         4096 Oct 24 2024  dateutildrwxr-xr-x    2 root     system         4096 Oct 24 2024  python_dateutil-2.8.2.dist-info#

Err, even though both pytz and dateutil are "found" within the RPM DB (rpm -ql <package> returns details) when checking within the directory structure we find that pytz is, in fact, missing!!  Ok, let's re-install just "pytz" and see if that straightens things out.

# dnf reinstall python3.9-pytz. . .Reinstalled:  python3.9-pytz-2023.3-1.noarchComplete!# ls -la /opt/freeware/lib/python3.9/site-packages | grep pytzdrwxr-xr-x    4 root     system         4096 Aug 05 04:25 pytzdrwxr-xr-x    2 root     system         4096 Aug 05 04:25 pytz-2023.3-py3.9.egg-info#

Hurrah!!  We now have the "missing" package show up in the directory structure.  Time to import pandas!!

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/opt/freeware/lib64/python3.9/site-packages/pandas/__init__.py", line 19, in <module>    raise ImportError(ImportError: Unable to import required dependencies:pytz: No module named 'pytz'dateutil: No module named 'dateutil'>>>

Ah!  We need to revisit our PYTHONPATH environment variable as the "pytz" and "dateutil" packages are placed in a different directory structure.

# echo $PYTHONPATH/opt/freeware/lib64/python3.9/site-packages# export PYTHONPATH=/opt/freeware/lib64/python3.9/site-packages:/opt/freeware/lib/python3.9/site-packages# echo $PYTHONPATH/opt/freeware/lib64/python3.9/site-packages:/opt/freeware/lib/python3.9/site-packages#

Right, we're all set now, time to import pandas!

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandas>>>

SUCCESS :-)

So, all in all, an odd one whereby "rpm -ql" was correctly returning the fact that the packages (pytz, dateutil) were installed and displayed the directory where they had been installed however, upon closer examination, the "pytz" package was missing.  A "dnf reinstall" resolved that issue, then it was down to updating the PYTHONPATH environment variable before we could finally import pandas!!

All the best, Steve

Hi @Steve Munday

While your analysis is partially right the root cause of the issue is the customer is using AIX-base Python built with the openXL compiler.

If they use /opt/freeware/bin/python3, there is no need to export anything. Just importing pandas will work.

Either export PATH=/opt/freeware/bin:$PATH or give use the full binary path.

See the description between the two python in their compilers.

Thanks.

# python3Python 3.9.20 (main, Sep 27 2024, 03:16:26) [GCC 10.3.0] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandas>>> 

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandas

Greetings team: I pass this on in case someone else hits upon the same issue as we had.

Recently the Development team raised a ticket as they were seeing the following "error" on an AIX 7.3.3.0 LPAR.

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):  File "<stdin>", line 1, in <module>ModuleNotFoundError: No module named 'pandas'>>>

Easy, just install "pandas" and we're all done.  Wrong!  Or so it was initially :-)

So, first things first, get "pandas" installed.

# dnf install python3.9-pandas. . . .Installed:  python3.9-pandas-2.2.3-1.ppc    openblas-0.3.28-1.ppc    python3.9-numpy-2.0.2-1.ppc    python3.9-pytz-2023.3-1.noarch    python3.9-tzdata-2023.3-1.noarch    libgfortran-1:10-2.ppc  libgfortran10-10.3.0-6.ppc

Now retry importing "pandas"

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):  File "<stdin>", line 1, in <module>ModuleNotFoundError: No module named 'pandas'>>>

Still no joy.  Upon further analysis it was necessary to set the PYTHONPATH environment variable to point to where "pandas" had been installed.

# rpm -ql python3.9-pandas/opt/freeware/doc/python3.9-pandas-2.2.3/opt/freeware/doc/python3.9-pandas-2.2.3/README.md/opt/freeware/lib64/python3.9/site-packages/pandas. . .# export PYTHONPATH=/opt/freeware/lib64/python3.9/site-packages# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/opt/freeware/lib64/python3.9/site-packages/pandas/__init__.py", line 19, in <module>    raise ImportError(ImportError: Unable to import required dependencies:pytz: No module named 'pytz'dateutil: No module named 'dateutil'>>>

Ok, so where are "pytz" and "dateutil" to be found?

# rpm -ql python3.9-pytz/opt/freeware/doc/python3.9-pytz-2023.3/opt/freeware/doc/python3.9-pytz-2023.3/README.rst/opt/freeware/lib/python3.9/site-packages/pytz. . .# rpm -ql python3.9-dateutil/opt/freeware/doc/python3.9-dateutil-2.8.2/opt/freeware/doc/python3.9-dateutil-2.8.2/README.rst/opt/freeware/lib/python3.9/site-packages/dateutil. . .# ls -la /opt/freeware/lib/python3.9/site-packages | grep pytz# ls -la /opt/freeware/lib/python3.9/site-packages | grep dateutildrwxr-xr-x    6 root     system         4096 Oct 24 2024  dateutildrwxr-xr-x    2 root     system         4096 Oct 24 2024  python_dateutil-2.8.2.dist-info#

Err, even though both pytz and dateutil are "found" within the RPM DB (rpm -ql <package> returns details) when checking within the directory structure we find that pytz is, in fact, missing!!  Ok, let's re-install just "pytz" and see if that straightens things out.

# dnf reinstall python3.9-pytz. . .Reinstalled:  python3.9-pytz-2023.3-1.noarchComplete!# ls -la /opt/freeware/lib/python3.9/site-packages | grep pytzdrwxr-xr-x    4 root     system         4096 Aug 05 04:25 pytzdrwxr-xr-x    2 root     system         4096 Aug 05 04:25 pytz-2023.3-py3.9.egg-info#

Hurrah!!  We now have the "missing" package show up in the directory structure.  Time to import pandas!!

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/opt/freeware/lib64/python3.9/site-packages/pandas/__init__.py", line 19, in <module>    raise ImportError(ImportError: Unable to import required dependencies:pytz: No module named 'pytz'dateutil: No module named 'dateutil'>>>

Ah!  We need to revisit our PYTHONPATH environment variable as the "pytz" and "dateutil" packages are placed in a different directory structure.

# echo $PYTHONPATH/opt/freeware/lib64/python3.9/site-packages# export PYTHONPATH=/opt/freeware/lib64/python3.9/site-packages:/opt/freeware/lib/python3.9/site-packages# echo $PYTHONPATH/opt/freeware/lib64/python3.9/site-packages:/opt/freeware/lib/python3.9/site-packages#

Right, we're all set now, time to import pandas!

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandas>>>

SUCCESS :-)

So, all in all, an odd one whereby "rpm -ql" was correctly returning the fact that the packages (pytz, dateutil) were installed and displayed the directory where they had been installed however, upon closer examination, the "pytz" package was missing.  A "dnf reinstall" resolved that issue, then it was down to updating the PYTHONPATH environment variable before we could finally import pandas!!

All the best, Steve

A little background.

AIX has two pythons.

1: In AIX base built with the openXL compiler for users to run Python in the AIX base.
2: The toolbox Python built using the GCC compiler for AIX users to try toolbox packages.

The case above is where a customer installed pandas from the toolbox but is trying to import it using the AIX-based openXL compiler-built Python, which will fail. 

That is why whenever any AIX user uses toolbox packages, best practice is to keep export PATH=/opt/freeware/bin:$PATH in the profile so that the toolbox binaries like python arev used.

Hi @Steve Munday

While your analysis is partially right the root cause of the issue is the customer is using AIX-base Python built with the openXL compiler.

If they use /opt/freeware/bin/python3, there is no need to export anything. Just importing pandas will work.

Either export PATH=/opt/freeware/bin:$PATH or give use the full binary path.

See the description between the two python in their compilers.

Thanks.

# python3Python 3.9.20 (main, Sep 27 2024, 03:16:26) [GCC 10.3.0] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandas>>> 

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandas

Greetings team: I pass this on in case someone else hits upon the same issue as we had.

Recently the Development team raised a ticket as they were seeing the following "error" on an AIX 7.3.3.0 LPAR.

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):  File "<stdin>", line 1, in <module>ModuleNotFoundError: No module named 'pandas'>>>

Easy, just install "pandas" and we're all done.  Wrong!  Or so it was initially :-)

So, first things first, get "pandas" installed.

# dnf install python3.9-pandas. . . .Installed:  python3.9-pandas-2.2.3-1.ppc    openblas-0.3.28-1.ppc    python3.9-numpy-2.0.2-1.ppc    python3.9-pytz-2023.3-1.noarch    python3.9-tzdata-2023.3-1.noarch    libgfortran-1:10-2.ppc  libgfortran10-10.3.0-6.ppc

Now retry importing "pandas"

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):  File "<stdin>", line 1, in <module>ModuleNotFoundError: No module named 'pandas'>>>

Still no joy.  Upon further analysis it was necessary to set the PYTHONPATH environment variable to point to where "pandas" had been installed.

# rpm -ql python3.9-pandas/opt/freeware/doc/python3.9-pandas-2.2.3/opt/freeware/doc/python3.9-pandas-2.2.3/README.md/opt/freeware/lib64/python3.9/site-packages/pandas. . .# export PYTHONPATH=/opt/freeware/lib64/python3.9/site-packages# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/opt/freeware/lib64/python3.9/site-packages/pandas/__init__.py", line 19, in <module>    raise ImportError(ImportError: Unable to import required dependencies:pytz: No module named 'pytz'dateutil: No module named 'dateutil'>>>

Ok, so where are "pytz" and "dateutil" to be found?

# rpm -ql python3.9-pytz/opt/freeware/doc/python3.9-pytz-2023.3/opt/freeware/doc/python3.9-pytz-2023.3/README.rst/opt/freeware/lib/python3.9/site-packages/pytz. . .# rpm -ql python3.9-dateutil/opt/freeware/doc/python3.9-dateutil-2.8.2/opt/freeware/doc/python3.9-dateutil-2.8.2/README.rst/opt/freeware/lib/python3.9/site-packages/dateutil. . .# ls -la /opt/freeware/lib/python3.9/site-packages | grep pytz# ls -la /opt/freeware/lib/python3.9/site-packages | grep dateutildrwxr-xr-x    6 root     system         4096 Oct 24 2024  dateutildrwxr-xr-x    2 root     system         4096 Oct 24 2024  python_dateutil-2.8.2.dist-info#

Err, even though both pytz and dateutil are "found" within the RPM DB (rpm -ql <package> returns details) when checking within the directory structure we find that pytz is, in fact, missing!!  Ok, let's re-install just "pytz" and see if that straightens things out.

# dnf reinstall python3.9-pytz. . .Reinstalled:  python3.9-pytz-2023.3-1.noarchComplete!# ls -la /opt/freeware/lib/python3.9/site-packages | grep pytzdrwxr-xr-x    4 root     system         4096 Aug 05 04:25 pytzdrwxr-xr-x    2 root     system         4096 Aug 05 04:25 pytz-2023.3-py3.9.egg-info#

Hurrah!!  We now have the "missing" package show up in the directory structure.  Time to import pandas!!

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/opt/freeware/lib64/python3.9/site-packages/pandas/__init__.py", line 19, in <module>    raise ImportError(ImportError: Unable to import required dependencies:pytz: No module named 'pytz'dateutil: No module named 'dateutil'>>>

Ah!  We need to revisit our PYTHONPATH environment variable as the "pytz" and "dateutil" packages are placed in a different directory structure.

# echo $PYTHONPATH/opt/freeware/lib64/python3.9/site-packages# export PYTHONPATH=/opt/freeware/lib64/python3.9/site-packages:/opt/freeware/lib/python3.9/site-packages# echo $PYTHONPATH/opt/freeware/lib64/python3.9/site-packages:/opt/freeware/lib/python3.9/site-packages#

Right, we're all set now, time to import pandas!

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandas>>>

SUCCESS :-)

So, all in all, an odd one whereby "rpm -ql" was correctly returning the fact that the packages (pytz, dateutil) were installed and displayed the directory where they had been installed however, upon closer examination, the "pytz" package was missing.  A "dnf reinstall" resolved that issue, then it was down to updating the PYTHONPATH environment variable before we could finally import pandas!!

All the best, Steve

Hi @Steve Munday

While your analysis is partially right the root cause of the issue is the customer is using AIX-base Python built with the openXL compiler.

If they use /opt/freeware/bin/python3, there is no need to export anything. Just importing pandas will work.

Either export PATH=/opt/freeware/bin:$PATH or give use the full binary path.

See the description between the two python in their compilers.

Thanks.

# python3Python 3.9.20 (main, Sep 27 2024, 03:16:26) [GCC 10.3.0] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandas>>> 

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandas

Greetings team: I pass this on in case someone else hits upon the same issue as we had.

Recently the Development team raised a ticket as they were seeing the following "error" on an AIX 7.3.3.0 LPAR.

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):  File "<stdin>", line 1, in <module>ModuleNotFoundError: No module named 'pandas'>>>

Easy, just install "pandas" and we're all done.  Wrong!  Or so it was initially :-)

So, first things first, get "pandas" installed.

# dnf install python3.9-pandas. . . .Installed:  python3.9-pandas-2.2.3-1.ppc    openblas-0.3.28-1.ppc    python3.9-numpy-2.0.2-1.ppc    python3.9-pytz-2023.3-1.noarch    python3.9-tzdata-2023.3-1.noarch    libgfortran-1:10-2.ppc  libgfortran10-10.3.0-6.ppc

Now retry importing "pandas"

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):  File "<stdin>", line 1, in <module>ModuleNotFoundError: No module named 'pandas'>>>

Still no joy.  Upon further analysis it was necessary to set the PYTHONPATH environment variable to point to where "pandas" had been installed.

# rpm -ql python3.9-pandas/opt/freeware/doc/python3.9-pandas-2.2.3/opt/freeware/doc/python3.9-pandas-2.2.3/README.md/opt/freeware/lib64/python3.9/site-packages/pandas. . .# export PYTHONPATH=/opt/freeware/lib64/python3.9/site-packages# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/opt/freeware/lib64/python3.9/site-packages/pandas/__init__.py", line 19, in <module>    raise ImportError(ImportError: Unable to import required dependencies:pytz: No module named 'pytz'dateutil: No module named 'dateutil'>>>

Ok, so where are "pytz" and "dateutil" to be found?

# rpm -ql python3.9-pytz/opt/freeware/doc/python3.9-pytz-2023.3/opt/freeware/doc/python3.9-pytz-2023.3/README.rst/opt/freeware/lib/python3.9/site-packages/pytz. . .# rpm -ql python3.9-dateutil/opt/freeware/doc/python3.9-dateutil-2.8.2/opt/freeware/doc/python3.9-dateutil-2.8.2/README.rst/opt/freeware/lib/python3.9/site-packages/dateutil. . .# ls -la /opt/freeware/lib/python3.9/site-packages | grep pytz# ls -la /opt/freeware/lib/python3.9/site-packages | grep dateutildrwxr-xr-x    6 root     system         4096 Oct 24 2024  dateutildrwxr-xr-x    2 root     system         4096 Oct 24 2024  python_dateutil-2.8.2.dist-info#

Err, even though both pytz and dateutil are "found" within the RPM DB (rpm -ql <package> returns details) when checking within the directory structure we find that pytz is, in fact, missing!!  Ok, let's re-install just "pytz" and see if that straightens things out.

# dnf reinstall python3.9-pytz. . .Reinstalled:  python3.9-pytz-2023.3-1.noarchComplete!# ls -la /opt/freeware/lib/python3.9/site-packages | grep pytzdrwxr-xr-x    4 root     system         4096 Aug 05 04:25 pytzdrwxr-xr-x    2 root     system         4096 Aug 05 04:25 pytz-2023.3-py3.9.egg-info#

Hurrah!!  We now have the "missing" package show up in the directory structure.  Time to import pandas!!

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/opt/freeware/lib64/python3.9/site-packages/pandas/__init__.py", line 19, in <module>    raise ImportError(ImportError: Unable to import required dependencies:pytz: No module named 'pytz'dateutil: No module named 'dateutil'>>>

Ah!  We need to revisit our PYTHONPATH environment variable as the "pytz" and "dateutil" packages are placed in a different directory structure.

# echo $PYTHONPATH/opt/freeware/lib64/python3.9/site-packages# export PYTHONPATH=/opt/freeware/lib64/python3.9/site-packages:/opt/freeware/lib/python3.9/site-packages# echo $PYTHONPATH/opt/freeware/lib64/python3.9/site-packages:/opt/freeware/lib/python3.9/site-packages#

Right, we're all set now, time to import pandas!

# python3Python 3.9.20 (main, Oct 18 2024, 06:15:28)[IBM XL C/C++ for AIX 16.1.0.14] on aixType "help", "copyright", "credits" or "license" for more information.>>> import pandas>>>

SUCCESS :-)

So, all in all, an odd one whereby "rpm -ql" was correctly returning the fact that the packages (pytz, dateutil) were installed and displayed the directory where they had been installed however, upon closer examination, the "pytz" package was missing.  A "dnf reinstall" resolved that issue, then it was down to updating the PYTHONPATH environment variable before we could finally import pandas!!

All the best, Steve