AIX Open Source

power-server-operatingsystems.png

Operating Systems

The core of your mission-critical workloads: AIX, IBM i, and Enterprise Linux

power-ISV-Solutions1.png

ISV Solutions

Stay up to date with key capabilities running on POWER such as SAP

Business Continuity

Learn how to keep critical processes running and adapt quickly with PowerHA

power-infrastructure-security.png

Power Security

Protect your most sensitive data anywhere in your hybrid cloud

Virtualization

Check here for virtualization and management needs: HMC & CMC, PowerVC, and PowerVM

Open Source

Join our open source efforts within the IBM Power Systems portfolio

Enterprise Infrastructure as a Service

Achieve business growth with agility and flexibility with our enterprise IaaS in Power Virtual Server

Programming Languages

The hub for Programming Languages

Expand all | Collapse all

Issue When Installing lxml for Python3 using pip

Jump to Best Answer
  • 1.  Issue When Installing lxml for Python3 using pip

    Posted Fri June 18, 2021 10:10 AM
    Hello all;
    I originally posted this in the AIX community, but, thankfully was directed to the more appropriate community here. Our PeopleSoft folks need lxml for Python3. We have been unable to locate Python3 with lxml for AIX so we are trying to use pip to install it using a download from the Python Package Index. Our AIX servers do not have internet access so pip cannot download it directly. AIX is at 7100-05-08-2114. We believe that we have installed the necessary prereqs, but the pip installation fails with a message to make sure they are installed. Also, where are the logs that we are to check for more info?

    # rpm -qa | grep python3
    libxml2-python3-2.9.10-1.ppc
    python3-tools-3.7.10-2.ppc
    python3-3.7.10-2.ppc
    python3-devel-3.7.10-2.ppc

    # rpm -qa | grep libxml2
    libxml2-python3-2.9.10-1.ppc
    libxml2-2.9.10-1.ppc
    libxml2-devel-2.9.10-1.ppc

    # rpm -qa | grep libxslt
    libxslt-1.1.34-1.ppc
    libxslt-devel-1.1.34-1.ppc

    # python -m pip install -v lxml-4.6.3.tar.gz --no-cache-dir
    Non-user install because site-packages writeable
    Created temporary directory: /tmp/pip-ephem-wheel-cache-1wk563bc
    Created temporary directory: /tmp/pip-req-tracker-ocf41tog
    Initialized build tracking at /tmp/pip-req-tracker-ocf41tog
    Created build tracker: /tmp/pip-req-tracker-ocf41tog
    Entered build tracker: /tmp/pip-req-tracker-ocf41tog
    Created temporary directory: /tmp/pip-install-206dmolc
    Processing ./lxml-4.6.3.tar.gz
    Created temporary directory: /tmp/pip-req-build-mynenchm
    Added file:///home/knighted/python3/lxml/lxml-4.6.3.tar.gz to build tracker '/tmp/pip-req-tracker-ocf41tog'
    Running setup.py (path:/tmp/pip-req-build-mynenchm/setup.py) egg_info for package from file:///<file path>/lxml/lxml-4.6.3.tar.gz
    Created temporary directory: /tmp/pip-pip-egg-info-kpbhtg7w
    Running command python setup.py egg_info
    Building lxml version 4.6.3.
    Building without Cython.
    Error: Please make sure the libxml2 and libxslt development packages are installed.
    ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
    Exception information:
    Traceback (most recent call last):
    File "/opt/freeware/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 188, in _main
    status = self.run(options, args)
    File "/opt/freeware/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 185, in wrapper
    return func(self, options, args)
    File "/opt/freeware/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 333, in run
    reqs, check_supported_wheels=not options.target_dir
    File "/opt/freeware/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 179, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
    File "/opt/freeware/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 362, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
    File "/opt/freeware/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 314, in _get_abstract_dist_for
    abstract_dist = self.preparer.prepare_linked_requirement(req)
    File "/opt/freeware/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 488, in prepare_linked_requirement
    req, self.req_tracker, self.finder, self.build_isolation,
    File "/opt/freeware/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 91, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
    File "/opt/freeware/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 40, in prepare_distribution_metadata
    self.req.prepare_metadata()
    File "/opt/freeware/lib/python3.7/site-packages/pip/_internal/req/req_install.py", line 550, in prepare_metadata
    self.metadata_directory = self._generate_metadata()
    File "/opt/freeware/lib/python3.7/site-packages/pip/_internal/req/req_install.py", line 530, in _generate_metadata
    details=self.name or "from {}".format(self.link)
    File "/opt/freeware/lib/python3.7/site-packages/pip/_internal/operations/build/metadata_legacy.py", line 73, in generate_metadata
    command_desc='python setup.py egg_info',
    File "/opt/freeware/lib/python3.7/site-packages/pip/_internal/utils/subprocess.py", line 241, in call_subprocess
    raise InstallationError(exc_msg)
    pip._internal.exceptions.InstallationError: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
    1 location(s) to search for versions of pip:
    * https://pypi.org/simple/pip/
    Fetching project page and analyzing links: https://pypi.org/simple/pip/
    Getting page https://pypi.org/simple/pip/
    Found index url https://pypi.org/simple
    Starting new HTTPS connection (1): pypi.org:443
    Could not fetch URL https://pypi.org/simple/pip/: connection error: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0xa0000000105ad88>: Failed to establish a new connection: [Errno 79] A remote host refused an attempted connect operation.')) - skipping
    Given no hashes to check 0 links for project 'pip': discarding no candidates
    Removed file:///home/knighted/python3/lxml/lxml-4.6.3.tar.gz from build tracker '/tmp/pip-req-tracker-ocf41tog'
    Removed build tracker: '/tmp/pip-req-tracker-ocf41tog'



    ------------------------------
    Ed Stuart
    ------------------------------


  • 2.  RE: Issue When Installing lxml for Python3 using pip

    Posted Fri June 18, 2021 12:33 PM
    Could it be that it cannot find the header files at /opt/freeware/include even though that is in the PATH? If so, how do we explicitly tell pip to look in that directory?

    ------------------------------
    Ed Stuart
    ------------------------------



  • 3.  RE: Issue When Installing lxml for Python3 using pip

    Posted Mon June 21, 2021 02:30 AM
    The libxslt installed is not from Toolbox. 
    rpm -qi libxslt ?
    Can you install libxslt & libxslt-devel from Toolbox and then try pip ?

    ------------------------------
    Ayappan P
    ------------------------------



  • 4.  RE: Issue When Installing lxml for Python3 using pip

    Posted Mon June 21, 2021 10:34 AM
    Thank-you Ayappan P; we installed the ones from the AIX Toolbox and are still getting the same error.

    # rpm -qi libxslt-devel-1.1.32-1.ppc
    Name : libxslt-devel
    Version : 1.1.32
    Release : 1
    Architecture: ppc
    Install Date: Mon Jun 21 09:12:31 CDT 2021
    Group : Development/Libraries
    Size : 2724077
    License : MIT
    Signature : (none)
    Source RPM : libxslt-1.1.32-1.src.rpm
    Build Date : Tue Mar 20 02:18:36 CDT 2018
    Build Host : pokndd5.pok.stglabs.ibm.com
    URL : http://xmlsoft.org/XSLT/
    Summary : Libraries, includes, etc. to embed the GNOME XSLT engine
    Description :
    This C library allows to transform XML files into other XML files
    (or HTML, text, ...) using the standard XSLT stylesheet transformation
    mechanism. To use it you need to have a version of libxml2 >= 2.9.5
    installed.

    If you are compiling a 32-bit program, no special compiler options are
    needed.

    If you are compiling a 64-bit program, you have to compile and link your
    application with "xlc_r -q64" or "gcc -maix64".

    # rpm -qi libxml2-devel-2.9.10-1.ppc
    Name : libxml2-devel
    Version : 2.9.10
    Release : 1
    Architecture: ppc
    Install Date: Mon Jun 21 09:04:16 CDT 2021
    Group : Development/Libraries
    Size : 9603061
    License : MIT
    Signature : (none)
    Source RPM : libxml2-2.9.10-1.src.rpm
    Build Date : Thu Nov 19 07:38:04 CST 2020
    Build Host : pokndd10.pok.stglabs.ibm.com
    Packager : IBM AIX Toolbox <https://ibm.biz/AIXToolbox>
    URL : http://xmlsoft.org/
    Bug URL : https://ibm.biz/aixoss_forum
    Summary : Libraries, includes, etc. to develop XML and HTML applications
    Description :
    Libraries, include files, etc you can use to develop XML applications.
    This library allows to manipulate XML files. It includes support
    to read, modify and write XML and HTML files. There is DTDs support
    this includes parsing and validation even with complex DtDs, either
    at parse time or later once the document has been modified. The output
    can be a simple SAX stream or and in-memory DOM like representations.
    In this case one can use the built-in XPath and XPointer implementation
    to select subnodes or ranges. A flexible Input/Output mechanism is
    available, with existing HTTP and FTP modules and combined to an
    URI library.

    If you are compiling a 32-bit program, no special compiler options are
    needed.

    If you are compiling a 64-bit program, you have to compile and link your
    application with "xlc -q64" or "gcc -maix64".

    # rpm -qi python3-devel-3.7.10-2.ppc
    Name : python3-devel
    Version : 3.7.10
    Release : 2
    Architecture: ppc
    Install Date: Mon Jun 21 09:16:26 CDT 2021
    Group : Development/Libraries
    Size : 783172
    License : Modified CNRI Open Source License
    Signature : (none)
    Source RPM : python3-3.7.10-2.src.rpm
    Build Date : Tue Jun 1 05:01:22 CDT 2021
    Build Host : pokndd10.pok.stglabs.ibm.com
    Relocations : /opt/freeware
    Packager : IBM AIX Toolbox <https://ibm.biz/AIXToolbox>
    URL : http://www.python.org/
    Bug URL : https://ibm.biz/aixoss_forum
    Summary : The libraries and header files needed for Python development.
    Description :
    The Python programming language s interpreter can be extended with
    dynamically loaded extensions and can be embedded in other programs.
    This package contains the header files and libraries needed to do
    these types of tasks.

    Install python-devel if you want to develop Python extensions. The
    python package will also need to be installed. You ll probably also
    want to install the python-docs package, which contains Python
    documentation.

    This version has been compiled with GCC.

    # sudo python -m pip install /home/knighted/python3/lxml/lxml-4.6.3 --no-cache-dir
    Password:
    WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
    Processing /home/knighted/python3/lxml/lxml-4.6.3
    ERROR: Command errored out with exit status 1:
    command: /usr/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-b1ir5xps/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-b1ir5xps/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-e4ynlnhx
    cwd: /tmp/pip-req-build-b1ir5xps/
    Complete output (3 lines):
    Building lxml version 4.6.3.
    Building without Cython.
    Error: Please make sure the libxml2 and libxslt development packages are installed.
    ----------------------------------------
    ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
    WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
    Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping










    ------------------------------
    Ed Stuart
    ------------------------------



  • 5.  RE: Issue When Installing lxml for Python3 using pip
    Best Answer

    Posted Mon June 21, 2021 02:40 PM
    We were able to get around the pip issue by using the setup.py file with switches that point to the xslt and xml2 config files.
    sudo python setup.py install --with-xslt-config=/opt/freeware/bin/xslt-config  --with-xml2-config=/opt/freeware/bin/xml2-config

    It would be great if an lxml package for python3 would get added to the AIX Toolbox.

    ------------------------------
    Ed Stuart
    ------------------------------



  • 6.  RE: Issue When Installing lxml for Python3 using pip

    Posted Tue June 22, 2021 04:14 AM
    I think you haven't set your PATH to use "/opt/freeware/bin". 
    Otherwise this would have been smooth.

    ------------------------------
    Ayappan P
    ------------------------------