AIX Open Source

AIX Open Source

Share your experiences and connect with fellow developers to discover how to build and manage open source software for the AIX operating system

 View Only
Expand all | Collapse all

How to install python 3 external libraries for python 3.9.6

  • 1.  How to install python 3 external libraries for python 3.9.6

    Posted Thu March 09, 2023 12:55 PM

    Hi AIX Experts,

    We have installed dnf and then python 3.9.6 on our AIX boxes. We have then installed external libraries like pandas and scikit-learn by using dnf tool. But I found the installed external libraries are only associated with the python 3.7 that comes with dnf. I tried to run command like 'python3.9 -m pip install python3-pandas-1.3.3-1.aix7.1.ppc.rpm' to install the libraries for python 3.9.6. But I got the following error. My question is what is the correct way to install external libraries for python 3.9.6?

    "

    ERROR: Exception:
    Traceback (most recent call last):
      File "/opt/freeware/lib/python3.9/site-packages/pip/_vendor/packaging/requirements.py", line 113, in __init__
        req = REQUIREMENT.parseString(requirement_string)
      File "/opt/freeware/lib/python3.9/site-packages/pip/_vendor/pyparsing.py", line 1955, in parseString
        raise exc
      File "/opt/freeware/lib/python3.9/site-packages/pip/_vendor/pyparsing.py", line 3250, in parseImpl
        raise ParseException(instring, loc, self.errmsg, self)
    pip._vendor.pyparsing.ParseException: Expected W:(abcd...), found '/'  (at char 0), (line:1, col:1)

     

    During handling of the above exception, another exception occurred: .........."

    Thanks

    Mi



    ------------------------------
    Mi Wang
    ------------------------------


  • 2.  RE: How to install python 3 external libraries for python 3.9.6

    Posted Fri March 10, 2023 01:26 AM

    It seems that you are trying to install an RPM package using pip, which is not the correct method for installing external libraries for Python. To install external libraries for Python 3.9.6 on AIX, you should use the pip3.9 command instead of pip.

    First, make sure that pip3.9 is installed on your system. You can do this by running the following command:

    python3.9 -m ensurepip

    This will install pip3.9 if it's not already installed. Once pip3.9 is installed, you can use it to install external libraries for Python 3.9.6 like this:

    pip3.9 install pandas pip3.9 install scikit-learn

    This will install the pandas and sci-kit-learn libraries for Python 3.9.6 on your AIX system.

    If you still encounter issues, ensure that the libraries you are trying to install are compatible with Python 3.9.6 and that their dependencies are also installed. You can use the pip3.9 command to install dependencies as well.

    Micheal

    CEO; Jackpot Results



    ------------------------------
    Micheal Jakson
    ------------------------------



  • 3.  RE: How to install python 3 external libraries for python 3.9.6

    Posted Fri March 10, 2023 03:41 AM

    From python3.9-3.9.16 onwards, pip and setuptools are not part of the main python3.9 rpm.
    So users have to install python3.9-pip & python3.9-setuptools rpm to get them. 

    "python3.9 -m ensurepip" is not recommended as it will extract pip & setuptools bundle from the main rpm. This will create problems if users are installing python3.9-pip & python3.9-setuptools since the bundle has lower versions of pip & setuptools. The environment then becomes a mix of files from different versions. 



    ------------------------------
    Ayappan P
    ------------------------------



  • 4.  RE: How to install python 3 external libraries for python 3.9.6

    Posted Fri March 10, 2023 10:32 AM

    We have tried running the following two commands and both are not successful. Can you please help?

     

    1. Running '/opt/freeware/bin/pip3.9 install pandas'. Although we have install AIX Toolbox repository with dnf, pip3.9 seems not being aware of that. I am wondering how we can set up the environment parameters to correct this?

      /opt/freeware/bin/pip3.9 install pandas

      WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError("<pip._vendor.urllib3.connection.HTTPSConnection object at 0xa000000011bfdf0>: Failed to establish a new connection: [Errno 68] Can't assign requested address")': /simple/pandas/
      ......

    2. Running '/opt/freeware/bin/pip3.9 install {file location folder}/python3-pandas-1.3.3-1.aix7.1.ppc.rpm'. How to resolve this?

      ERROR: Exception:

    Traceback (most recent call last):

      File "/opt/freeware/lib/python3.9/site-packages/pip/_vendor/packaging/requirements.py", line 113, in __init__

        req = REQUIREMENT.parseString(requirement_string)

      File "/opt/freeware/lib/python3.9/site-packages/pip/_vendor/pyparsing.py", line 1955, in parseString

        raise exc

      File "/opt/freeware/lib/python3.9/site-packages/pip/_vendor/pyparsing.py", line 3250, in parseImpl

        raise ParseException(instring, loc, self.errmsg, self)

    pip._vendor.pyparsing.ParseException: Expected W:(abcd...), found '/'  (at char 0), (line:1, col:1) During handling of the above exception, another exception occurred: 
    ........

     

    Thanks

     

    Mi


    Internal



    If you wish to unsubscribe from receiving commercial electronic messages from TD Bank Group, please click here or go to the following web address: www.td.com/tdoptout
    Si vous souhaitez vous désabonner des messages électroniques de nature commerciale envoyés par Groupe Banque TD veuillez cliquer ici ou vous rendre à l'adresse www.td.com/tddesab


    NOTICE: Confidential message which may be privileged. Unauthorized use/disclosure prohibited. If received in error, please go to www.td.com/legal for instructions.
    AVIS : Message confidentiel dont le contenu peut être privilégié. Utilisation/divulgation interdites sans permission. Si reçu par erreur, prière d'aller au www.td.com/francais/avis_juridique pour des instructions.






  • 5.  RE: How to install python 3 external libraries for python 3.9.6

    Posted Mon March 13, 2023 07:28 AM

    When you do pip to install any packages, it downloads the package source code from pypi site and tries to build & install it. pip & dnf/repo are two different things.
    The "python3" in python3-pandas rpm in Toolbox refers to the default python3 which is now 3.7. We are in the process of changing the default python3 to python3.9. As part of it, we will also be updating the python3 modules (like python3-pandas). 
    If you want to use pip3.9 to install pandas right now, then you need to make sure all the dependencies are installed. (like gcc, blas libraries., etc). Also the machine needs internet access to get the things from pypi site. 



    ------------------------------
    Ayappan P
    ------------------------------



  • 6.  RE: How to install python 3 external libraries for python 3.9.6

    Posted Mon March 13, 2023 10:59 AM

    Hi Ayappan,

    Thanks for the information. I now have the 2 questions:

    1. Do you have any rough date when IBM will finish the work to change the default python3 for AIX Toolbox to python3.9.16?
    2. If we want to  use pip3.9 to install python external libraries, we need to use all of the libraries (e,g,: those dependency libraries) downloaded from pypi? Can we download all related libraries and then do installation locally?

    Thanks

    Mi



    ------------------------------
    Mi Wang
    ------------------------------



  • 7.  RE: How to install python 3 external libraries for python 3.9.6

    Posted Tue March 14, 2023 05:56 AM

    1) We are targeting this month (March 2023) end for the python3 movement. 
    2) I see pandas has dependency on numpy, python-dateutil, pytz. So you need download the source code of these dependencies, compile them and install them. (These also may be having other dependencies). So it's not an easy thing. 



    ------------------------------
    Ayappan P
    ------------------------------



  • 8.  RE: How to install python 3 external libraries for python 3.9.6

    Posted Tue March 14, 2023 11:49 AM

    Hi Ayappan,

    For #2, is it doable that we do pip3.9 install using the numpy / python-dateutil / pytz packages from AIXToolbox instead?

    Thanks

    Mi



    ------------------------------
    Mi Wang
    ------------------------------



  • 9.  RE: How to install python 3 external libraries for python 3.9.6

    Posted Wed March 15, 2023 05:19 AM

    Hi Ayappan,

    For point number one referring to the movement to python3.9.16, do you have a list of packages that will be ready for March 23?

    Is paramiko in the radar for March?.



    ------------------------------
    LUIS ABDEL AGUILAR JURADO
    ------------------------------



  • 10.  RE: How to install python 3 external libraries for python 3.9.6

    Posted Wed March 15, 2023 05:22 AM

    All the python3 modules in Toolbox will be moved to python3.9. So yes , paramiko is also part of it. 



    ------------------------------
    Ayappan P
    ------------------------------



  • 11.  RE: How to install python 3 external libraries for python 3.9.6

    Posted Wed March 15, 2023 04:55 PM

    Hi Ayappan,

    The ultimate goal for us to build this python 3.9.6 and install external libraries is to implement machine learning / other advanced packages like Matplotlib, Xgboost, Mixtend. I see the following possible approaches for us to achieve our goals:

    1. Following instructions in this IBM link. I am wondering if we can install python v3.10 for AIX (non AIX Toolbox version) and then install external libraries / packages from PYPI only (this solution would need server to have internet connection)
      https://developer.ibm.com/tutorials/machine-learning-with-python-on-aix/
    2. Maybe the python 3.9.16 with AIX Toolbox can provide another path for us to achieve our goal? I noticed in the current AIX repository, there is no Matplotlib, Xgboost, Mixtend packages. Will the new AIX toolbox with python 3.9.16 provide those packages?
    3. Is there a third way that will allow us to combine above two methods that allow us to eventually achieve our goal?

    Thanks

    Mi



    ------------------------------
    Mi Wang
    ------------------------------



  • 12.  RE: How to install python 3 external libraries for python 3.9.6

    Posted Thu March 16, 2023 05:51 AM

    Matplotlib was/in our plan for quite sometime but because of other works we are not able to provide it through Toolbox. So to make it available in Toolbox, it will take some more time. Xgboost & Mixtend is not in our plans. 
    You can always try pip install these packages using python3.9 in some test setup and see how things goes. 



    ------------------------------
    Ayappan P
    ------------------------------



  • 13.  RE: How to install python 3 external libraries for python 3.9.6

    Posted Thu March 16, 2023 05:19 PM

    Hi Ayappan,

    How is the compatibility between AIX Toolbox packages / libraries and those from PYPI? I am thinking we may proceed with the steps outlined in the following IBM link page, first using packages from AIX Toolbox up until the step to install Matplotlib 'python3 -m pip install matplotlib'. we then use the PYPI's Matplotlib package to do this last step. We will use python 3.7 for now.

    https://developer.ibm.com/tutorials/machine-learning-with-python-on-aix/

    Thanks

    Mi



    ------------------------------
    Mi Wang
    ------------------------------



  • 14.  RE: How to install python 3 external libraries for python 3.9.6

    Posted Fri March 17, 2023 02:07 AM

    Yes you can try this on Python 3.7 as of now for experiment. But I would suggest to use Python 3.9 when it is available.

    We will add more modules (Matplotlib, Jupyter Notebook and others) to AIX Toolbox going forward. 



    ------------------------------
    SANKET RATHI
    ------------------------------



  • 15.  RE: How to install python 3 external libraries for python 3.9.6

    Posted Thu March 30, 2023 02:09 PM

    Hi Ayappan,

    Is there any news about the movement of python3 modules to python3.9? Do the movement will be for end of March?



    ------------------------------
    LUIS ABDEL AGUILAR JURADO
    ------------------------------



  • 16.  RE: How to install python 3 external libraries for python 3.9.6

    Posted Fri March 31, 2023 01:01 AM

    We are still working on it. Probably it requires couple more weeks at the max. 



    ------------------------------
    Ayappan P
    ------------------------------



  • 17.  RE: How to install python 3 external libraries for python 3.9.6

    Posted Tue April 04, 2023 12:44 PM

    Hi Ayappan,

    Do you have an ETA for the migration, this will help me on planning the migration to python3.9.

    Thanks in advance.



    ------------------------------
    LUIS ABDEL AGUILAR JURADO
    ------------------------------