What is the best way to package on z/OS

6. RE: What is the best way to package on z/OS

Like

Colin Paice

IBM Champion

Posted Thu January 06, 2022 07:39 AM

Hi Steven,

Thank you for your very valuable input. I've managed to package my component.

I had to install wheel, and then add import setuptools into my setup.sh and then it worked.

I had to learn how to install wheel on my standalone machine.

I looked into trying to get one .whl package for both python 3.8 and 3.10 - but it looks like the .whl format has the Python version in the file name. If I had pure Python it can have PY2 and PY3.

I'm blogging all the lessons I learned, so I hope people following behind me will have an easier time!

Thanks again for your help

Colin

I'll update

------------------------------
Colin Paice
------------------------------

Original Message

Original Message:
Sent: Wed January 05, 2022 04:59 PM
From: Steven Pitman
Subject: What is the best way to package on z/OS

Hi Colin,

I think the wheels binary format is what you are looking for. You can install it into a venv (see previous message) using pip3 install wheel, then build your package using python3 setup.py bdist_wheel (assuming it uses setuptools). It'll generate a binary file with both your Python + shared libs already compiled. Then this file can be installed by doing a pip3 install <file.whl>, even for users that do not have wheel installed, so it can still be a single command to install. This should only be used for Python 3.10, not lower versions of IBM Open Enterprise SDK for Python. To answer your question though, the reason it gives you the error is that the bdist distribution format is meant so that you extract/copy it yourself, pip does not handle that. When you use bdist_wheel, it will, which will fix your error.

There's been two main ways Python has typically done distribution of packages - distutils and setuptools. As of Python 3.10, distutils has now been marked as deprecated (to be removed in 3.12), so should no longer be used. That said, there is some documentation still available for it here (and this can be swapped to use setuptools by simply changing the import statements). Setuptools has replaced distutils, and largely provides the same experience / options, and you can find some examples on how to get started with it here - there isn't anything z/OS specific, so any general guide on distutils/setuptools can be used. You may find articles like this useful for simple examples on using setuptools with c extensions.

When it comes to the actual distribution, here's a quick explanation of the commands for creating the archive files (pip will prefer wheels when installing, i.e any dependency):

python3 setup.py sdist - Creates a source distribution, i.e. contains the raw source files, when the installing the user will compile any requirements
python3 setup.py bdist - Creates a binary distribution, i.e contains the compiled shared libs + Python files. The user must extract/copy the files to the destination
python3 setup.py bdist_wheel - Creates a binary distribution, similar to bdist but it will install directly with pip

Regards,

Steven

------------------------------
Steven Pitman

Original Message:
Sent: Wed January 05, 2022 12:00 PM
From: Colin Paice
Subject: What is the best way to package on z/OS

Steven,

I'm struggling with packaging and installing it.
I'm using python3 setup.py bdist to build my file, and in a virtual environment I'm using
python3 -m pip install -v -v -v --user --no-cache-dir /u/pymqi/dist/pymqi-1.12.0.os390-27.00-1090.tar.gz 1>a 2>b
This gives me
pip._internal.exceptions.InstallationError: file:///u/pymqi/dist/pymqi-1.12.0.os390-27.00-1090.tar.gz
does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.

and it is correct - setup.py is not in the package.

I read that pip does not handle bdist files, and I should use bdist_wheel. I could not get this to work, and it looks wheel is not installed by default on z/OS 3.10.

I also saw that setup.py from distutils is deprecated; but I could not find a way of building a C extension... I can use setuptools for pure python - but could not find how to compile things.

Is there any documentation of how to build an extension on z/OS?

regards

Colin

------------------------------
Colin Paice

Original Message:
Sent: Tue January 04, 2022 11:09 AM
From: Steven Pitman
Subject: What is the best way to package on z/OS

Hi Colin,

For storing the package, there's a few options you can take:

1) If the user has access to IBM z/OS XL C/C++ and it's open source, you could submit it to PyPI and have the user install the package with pip, like any other package
2) If they have access to Rocket Git, you could also only store it on GitHub, and install it with pip directly from there
3) If it's not open source, you could store the binary (the pax & tar formats can support file tagging) on Github or any other file management system (ex: Artifactory) and have the user download it

For installing IBM Open Enterprise SDK for Python without a sysprog, you can download the pax edition and follow the Installation documentation. If you already have Python installed with SMP/E or it was installed into a directory you do not have write access to, Python comes with the module venv (sample usage found here). This module can create a local environment of Python, which you can then use to install packages into for any testing required.

Steven

------------------------------
Steven Pitman

Original Message:
Sent: Tue January 04, 2022 07:35 AM
From: Colin Paice
Subject: What is the best way to package on z/OS

I am looking at porting the PYMQI package to z/OS.

What is the best way to package it and make it available? There are some .py files, and a C program which gets compiled and bound into pymqe.cpython-310.so .
I could create a zip package and store it on GitHub, so you would have to download, it, upload it and install it.

What is the right way of doing it?

Also in many customers, Python is on a read only zfs system.

What is the best way of installing it in my local system and running with it, (think of it as evaluation mode) rather than get the sysprog to update production.

Colin

------------------------------
Colin Paice
------------------------------

7. RE: What is the best way to package on z/OS

Like

Steven Pitman

Posted Thu January 06, 2022 09:14 AM

Hi Colin,

One thing to note - in general for wheels, it's safe to use the package built for a given version on other minor releases. For example, building it for 3.10.0, it's generally safe to use it for 3.10.1 (PTFs) and onwards. But for major releases it is not, so it would not be guaranteed to be compatible with 3.8 or 3.9. The same is true even for building on earlier versions, e.g. build it on 3.8 and trying to use it on 3.9/3.10.

For a wheel package, you'll need to build it for all major versions and cannot just use one. Note that there were some issues in 3.8/3.9 with wheels that have been resolved in 3.10, so it's recommended you to use 3.10.

Regards,

Steven

------------------------------
Steven Pitman
------------------------------

Original Message

Original Message:
Sent: Thu January 06, 2022 07:39 AM
From: Colin Paice
Subject: What is the best way to package on z/OS

Hi Steven,

Thank you for your very valuable input. I've managed to package my component.

I had to install wheel, and then add import setuptools into my setup.sh and then it worked.

I had to learn how to install wheel on my standalone machine.

I looked into trying to get one .whl package for both python 3.8 and 3.10 - but it looks like the .whl format has the Python version in the file name. If I had pure Python it can have PY2 and PY3.

I'm blogging all the lessons I learned, so I hope people following behind me will have an easier time!

Thanks again for your help

Colin

I'll update

------------------------------
Colin Paice

Original Message:
Sent: Wed January 05, 2022 04:59 PM
From: Steven Pitman
Subject: What is the best way to package on z/OS

Hi Colin,

I think the wheels binary format is what you are looking for. You can install it into a venv (see previous message) using pip3 install wheel, then build your package using python3 setup.py bdist_wheel (assuming it uses setuptools). It'll generate a binary file with both your Python + shared libs already compiled. Then this file can be installed by doing a pip3 install <file.whl>, even for users that do not have wheel installed, so it can still be a single command to install. This should only be used for Python 3.10, not lower versions of IBM Open Enterprise SDK for Python. To answer your question though, the reason it gives you the error is that the bdist distribution format is meant so that you extract/copy it yourself, pip does not handle that. When you use bdist_wheel, it will, which will fix your error.

There's been two main ways Python has typically done distribution of packages - distutils and setuptools. As of Python 3.10, distutils has now been marked as deprecated (to be removed in 3.12), so should no longer be used. That said, there is some documentation still available for it here (and this can be swapped to use setuptools by simply changing the import statements). Setuptools has replaced distutils, and largely provides the same experience / options, and you can find some examples on how to get started with it here - there isn't anything z/OS specific, so any general guide on distutils/setuptools can be used. You may find articles like this useful for simple examples on using setuptools with c extensions.

When it comes to the actual distribution, here's a quick explanation of the commands for creating the archive files (pip will prefer wheels when installing, i.e any dependency):

python3 setup.py sdist - Creates a source distribution, i.e. contains the raw source files, when the installing the user will compile any requirements
python3 setup.py bdist - Creates a binary distribution, i.e contains the compiled shared libs + Python files. The user must extract/copy the files to the destination
python3 setup.py bdist_wheel - Creates a binary distribution, similar to bdist but it will install directly with pip

Regards,

Steven

------------------------------
Steven Pitman

Original Message:
Sent: Wed January 05, 2022 12:00 PM
From: Colin Paice
Subject: What is the best way to package on z/OS

Steven,

I'm struggling with packaging and installing it.
I'm using python3 setup.py bdist to build my file, and in a virtual environment I'm using
python3 -m pip install -v -v -v --user --no-cache-dir /u/pymqi/dist/pymqi-1.12.0.os390-27.00-1090.tar.gz 1>a 2>b
This gives me
pip._internal.exceptions.InstallationError: file:///u/pymqi/dist/pymqi-1.12.0.os390-27.00-1090.tar.gz
does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.

and it is correct - setup.py is not in the package.

I read that pip does not handle bdist files, and I should use bdist_wheel. I could not get this to work, and it looks wheel is not installed by default on z/OS 3.10.

I also saw that setup.py from distutils is deprecated; but I could not find a way of building a C extension... I can use setuptools for pure python - but could not find how to compile things.

Is there any documentation of how to build an extension on z/OS?

regards

Colin

------------------------------
Colin Paice

Original Message:
Sent: Tue January 04, 2022 11:09 AM
From: Steven Pitman
Subject: What is the best way to package on z/OS

Hi Colin,

For storing the package, there's a few options you can take:

1) If the user has access to IBM z/OS XL C/C++ and it's open source, you could submit it to PyPI and have the user install the package with pip, like any other package
2) If they have access to Rocket Git, you could also only store it on GitHub, and install it with pip directly from there
3) If it's not open source, you could store the binary (the pax & tar formats can support file tagging) on Github or any other file management system (ex: Artifactory) and have the user download it

For installing IBM Open Enterprise SDK for Python without a sysprog, you can download the pax edition and follow the Installation documentation. If you already have Python installed with SMP/E or it was installed into a directory you do not have write access to, Python comes with the module venv (sample usage found here). This module can create a local environment of Python, which you can then use to install packages into for any testing required.

Steven

------------------------------
Steven Pitman

Original Message:
Sent: Tue January 04, 2022 07:35 AM
From: Colin Paice
Subject: What is the best way to package on z/OS

I am looking at porting the PYMQI package to z/OS.

What is the best way to package it and make it available? There are some .py files, and a C program which gets compiled and bound into pymqe.cpython-310.so .
I could create a zip package and store it on GitHub, so you would have to download, it, upload it and install it.

What is the right way of doing it?

Also in many customers, Python is on a read only zfs system.

What is the best way of installing it in my local system and running with it, (think of it as evaluation mode) rather than get the sysprog to update production.

Colin

------------------------------
Colin Paice
------------------------------

8. RE: What is the best way to package on z/OS

Like

Colin Paice

IBM Champion

Posted Fri January 07, 2022 10:27 AM

Steven,

One more (hopefully the last). question

With an name like pymqi-1.12.0-cp310-cp310-os390_27_00_1090.whl. The 1090 represents my machine. I tried a build using --plat-name zos, which built, but this did not install.

Is there a way to name it so it can be on any z hardware - I cannot build on all supported hardware, or is there an option I need at install time? I could not find one.

(BTW I've given you a credit for your help in one of my blog posts - I hope that is OK)
Colin

------------------------------
Colin Paice
------------------------------

Original Message

Original Message:
Sent: Thu January 06, 2022 09:14 AM
From: Steven Pitman
Subject: What is the best way to package on z/OS

Hi Colin,

One thing to note - in general for wheels, it's safe to use the package built for a given version on other minor releases. For example, building it for 3.10.0, it's generally safe to use it for 3.10.1 (PTFs) and onwards. But for major releases it is not, so it would not be guaranteed to be compatible with 3.8 or 3.9. The same is true even for building on earlier versions, e.g. build it on 3.8 and trying to use it on 3.9/3.10.

For a wheel package, you'll need to build it for all major versions and cannot just use one. Note that there were some issues in 3.8/3.9 with wheels that have been resolved in 3.10, so it's recommended you to use 3.10.

Regards,

Steven

------------------------------
Steven Pitman

Original Message:
Sent: Thu January 06, 2022 07:39 AM
From: Colin Paice
Subject: What is the best way to package on z/OS

Hi Steven,

Thank you for your very valuable input. I've managed to package my component.

I had to install wheel, and then add import setuptools into my setup.sh and then it worked.

I had to learn how to install wheel on my standalone machine.

I looked into trying to get one .whl package for both python 3.8 and 3.10 - but it looks like the .whl format has the Python version in the file name. If I had pure Python it can have PY2 and PY3.

I'm blogging all the lessons I learned, so I hope people following behind me will have an easier time!

Thanks again for your help

Colin

I'll update

------------------------------
Colin Paice

Original Message:
Sent: Wed January 05, 2022 04:59 PM
From: Steven Pitman
Subject: What is the best way to package on z/OS

Hi Colin,

I think the wheels binary format is what you are looking for. You can install it into a venv (see previous message) using pip3 install wheel, then build your package using python3 setup.py bdist_wheel (assuming it uses setuptools). It'll generate a binary file with both your Python + shared libs already compiled. Then this file can be installed by doing a pip3 install <file.whl>, even for users that do not have wheel installed, so it can still be a single command to install. This should only be used for Python 3.10, not lower versions of IBM Open Enterprise SDK for Python. To answer your question though, the reason it gives you the error is that the bdist distribution format is meant so that you extract/copy it yourself, pip does not handle that. When you use bdist_wheel, it will, which will fix your error.

There's been two main ways Python has typically done distribution of packages - distutils and setuptools. As of Python 3.10, distutils has now been marked as deprecated (to be removed in 3.12), so should no longer be used. That said, there is some documentation still available for it here (and this can be swapped to use setuptools by simply changing the import statements). Setuptools has replaced distutils, and largely provides the same experience / options, and you can find some examples on how to get started with it here - there isn't anything z/OS specific, so any general guide on distutils/setuptools can be used. You may find articles like this useful for simple examples on using setuptools with c extensions.

When it comes to the actual distribution, here's a quick explanation of the commands for creating the archive files (pip will prefer wheels when installing, i.e any dependency):

python3 setup.py sdist - Creates a source distribution, i.e. contains the raw source files, when the installing the user will compile any requirements
python3 setup.py bdist - Creates a binary distribution, i.e contains the compiled shared libs + Python files. The user must extract/copy the files to the destination
python3 setup.py bdist_wheel - Creates a binary distribution, similar to bdist but it will install directly with pip

Regards,

Steven

------------------------------
Steven Pitman

Original Message:
Sent: Wed January 05, 2022 12:00 PM
From: Colin Paice
Subject: What is the best way to package on z/OS

Steven,

I'm struggling with packaging and installing it.
I'm using python3 setup.py bdist to build my file, and in a virtual environment I'm using
python3 -m pip install -v -v -v --user --no-cache-dir /u/pymqi/dist/pymqi-1.12.0.os390-27.00-1090.tar.gz 1>a 2>b
This gives me
pip._internal.exceptions.InstallationError: file:///u/pymqi/dist/pymqi-1.12.0.os390-27.00-1090.tar.gz
does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.

and it is correct - setup.py is not in the package.

I read that pip does not handle bdist files, and I should use bdist_wheel. I could not get this to work, and it looks wheel is not installed by default on z/OS 3.10.

I also saw that setup.py from distutils is deprecated; but I could not find a way of building a C extension... I can use setuptools for pure python - but could not find how to compile things.

Is there any documentation of how to build an extension on z/OS?

regards

Colin

------------------------------
Colin Paice

Original Message:
Sent: Tue January 04, 2022 11:09 AM
From: Steven Pitman
Subject: What is the best way to package on z/OS

Hi Colin,

For storing the package, there's a few options you can take:

1) If the user has access to IBM z/OS XL C/C++ and it's open source, you could submit it to PyPI and have the user install the package with pip, like any other package
2) If they have access to Rocket Git, you could also only store it on GitHub, and install it with pip directly from there
3) If it's not open source, you could store the binary (the pax & tar formats can support file tagging) on Github or any other file management system (ex: Artifactory) and have the user download it

For installing IBM Open Enterprise SDK for Python without a sysprog, you can download the pax edition and follow the Installation documentation. If you already have Python installed with SMP/E or it was installed into a directory you do not have write access to, Python comes with the module venv (sample usage found here). This module can create a local environment of Python, which you can then use to install packages into for any testing required.

Steven

------------------------------
Steven Pitman

Original Message:
Sent: Tue January 04, 2022 07:35 AM
From: Colin Paice
Subject: What is the best way to package on z/OS

I am looking at porting the PYMQI package to z/OS.

What is the best way to package it and make it available? There are some .py files, and a C program which gets compiled and bound into pymqe.cpython-310.so .
I could create a zip package and store it on GitHub, so you would have to download, it, upload it and install it.

What is the right way of doing it?

Also in many customers, Python is on a read only zfs system.

What is the best way of installing it in my local system and running with it, (think of it as evaluation mode) rather than get the sysprog to update production.

Colin

------------------------------
Colin Paice
------------------------------

9. RE: What is the best way to package on z/OS

Like

Steven Pitman

Posted Fri January 07, 2022 01:56 PM

Hi Colin,

Pip only provides the ability to install a package on any arbitrary platform if it's a pure python wheel, i.e that it contains no compiled extensions. The only other way would be to manually rename the file to to pretend it is, e.g. using the suffix <package>-cp310-none-any.whl, but this both not supported nor recommended, and has no guarantee to work.

Thanks,

Steven

------------------------------
Steven Pitman
------------------------------

Original Message

Original Message:
Sent: Fri January 07, 2022 10:26 AM
From: Colin Paice
Subject: What is the best way to package on z/OS

Steven,

One more (hopefully the last). question

With an name like pymqi-1.12.0-cp310-cp310-os390_27_00_1090.whl. The 1090 represents my machine. I tried a build using --plat-name zos, which built, but this did not install.

Is there a way to name it so it can be on any z hardware - I cannot build on all supported hardware, or is there an option I need at install time? I could not find one.

(BTW I've given you a credit for your help in one of my blog posts - I hope that is OK)
Colin

------------------------------
Colin Paice

Original Message:
Sent: Thu January 06, 2022 09:14 AM
From: Steven Pitman
Subject: What is the best way to package on z/OS

Hi Colin,

One thing to note - in general for wheels, it's safe to use the package built for a given version on other minor releases. For example, building it for 3.10.0, it's generally safe to use it for 3.10.1 (PTFs) and onwards. But for major releases it is not, so it would not be guaranteed to be compatible with 3.8 or 3.9. The same is true even for building on earlier versions, e.g. build it on 3.8 and trying to use it on 3.9/3.10.

For a wheel package, you'll need to build it for all major versions and cannot just use one. Note that there were some issues in 3.8/3.9 with wheels that have been resolved in 3.10, so it's recommended you to use 3.10.

Regards,

Steven

------------------------------
Steven Pitman

Original Message:
Sent: Thu January 06, 2022 07:39 AM
From: Colin Paice
Subject: What is the best way to package on z/OS

Hi Steven,

Thank you for your very valuable input. I've managed to package my component.

I had to install wheel, and then add import setuptools into my setup.sh and then it worked.

I had to learn how to install wheel on my standalone machine.

I looked into trying to get one .whl package for both python 3.8 and 3.10 - but it looks like the .whl format has the Python version in the file name. If I had pure Python it can have PY2 and PY3.

I'm blogging all the lessons I learned, so I hope people following behind me will have an easier time!

Thanks again for your help

Colin

I'll update

------------------------------
Colin Paice

Original Message:
Sent: Wed January 05, 2022 04:59 PM
From: Steven Pitman
Subject: What is the best way to package on z/OS

Hi Colin,

I think the wheels binary format is what you are looking for. You can install it into a venv (see previous message) using pip3 install wheel, then build your package using python3 setup.py bdist_wheel (assuming it uses setuptools). It'll generate a binary file with both your Python + shared libs already compiled. Then this file can be installed by doing a pip3 install <file.whl>, even for users that do not have wheel installed, so it can still be a single command to install. This should only be used for Python 3.10, not lower versions of IBM Open Enterprise SDK for Python. To answer your question though, the reason it gives you the error is that the bdist distribution format is meant so that you extract/copy it yourself, pip does not handle that. When you use bdist_wheel, it will, which will fix your error.

There's been two main ways Python has typically done distribution of packages - distutils and setuptools. As of Python 3.10, distutils has now been marked as deprecated (to be removed in 3.12), so should no longer be used. That said, there is some documentation still available for it here (and this can be swapped to use setuptools by simply changing the import statements). Setuptools has replaced distutils, and largely provides the same experience / options, and you can find some examples on how to get started with it here - there isn't anything z/OS specific, so any general guide on distutils/setuptools can be used. You may find articles like this useful for simple examples on using setuptools with c extensions.

When it comes to the actual distribution, here's a quick explanation of the commands for creating the archive files (pip will prefer wheels when installing, i.e any dependency):

python3 setup.py sdist - Creates a source distribution, i.e. contains the raw source files, when the installing the user will compile any requirements
python3 setup.py bdist - Creates a binary distribution, i.e contains the compiled shared libs + Python files. The user must extract/copy the files to the destination
python3 setup.py bdist_wheel - Creates a binary distribution, similar to bdist but it will install directly with pip

Regards,

Steven

------------------------------
Steven Pitman

Original Message:
Sent: Wed January 05, 2022 12:00 PM
From: Colin Paice
Subject: What is the best way to package on z/OS

Steven,

I'm struggling with packaging and installing it.
I'm using python3 setup.py bdist to build my file, and in a virtual environment I'm using
python3 -m pip install -v -v -v --user --no-cache-dir /u/pymqi/dist/pymqi-1.12.0.os390-27.00-1090.tar.gz 1>a 2>b
This gives me
pip._internal.exceptions.InstallationError: file:///u/pymqi/dist/pymqi-1.12.0.os390-27.00-1090.tar.gz
does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.

and it is correct - setup.py is not in the package.

I read that pip does not handle bdist files, and I should use bdist_wheel. I could not get this to work, and it looks wheel is not installed by default on z/OS 3.10.

I also saw that setup.py from distutils is deprecated; but I could not find a way of building a C extension... I can use setuptools for pure python - but could not find how to compile things.

Is there any documentation of how to build an extension on z/OS?

regards

Colin

------------------------------
Colin Paice

Original Message:
Sent: Tue January 04, 2022 11:09 AM
From: Steven Pitman
Subject: What is the best way to package on z/OS

Hi Colin,

For storing the package, there's a few options you can take:

1) If the user has access to IBM z/OS XL C/C++ and it's open source, you could submit it to PyPI and have the user install the package with pip, like any other package
2) If they have access to Rocket Git, you could also only store it on GitHub, and install it with pip directly from there
3) If it's not open source, you could store the binary (the pax & tar formats can support file tagging) on Github or any other file management system (ex: Artifactory) and have the user download it

For installing IBM Open Enterprise SDK for Python without a sysprog, you can download the pax edition and follow the Installation documentation. If you already have Python installed with SMP/E or it was installed into a directory you do not have write access to, Python comes with the module venv (sample usage found here). This module can create a local environment of Python, which you can then use to install packages into for any testing required.

Steven

------------------------------
Steven Pitman

Original Message:
Sent: Tue January 04, 2022 07:35 AM
From: Colin Paice
Subject: What is the best way to package on z/OS

I am looking at porting the PYMQI package to z/OS.

What is the best way to package it and make it available? There are some .py files, and a C program which gets compiled and bound into pymqe.cpython-310.so .
I could create a zip package and store it on GitHub, so you would have to download, it, upload it and install it.

What is the right way of doing it?

Also in many customers, Python is on a read only zfs system.

What is the best way of installing it in my local system and running with it, (think of it as evaluation mode) rather than get the sysprog to update production.

Colin

------------------------------
Colin Paice
------------------------------

10. RE: What is the best way to package on z/OS

Like

Colin Paice

IBM Champion

Posted Sat January 08, 2022 09:29 AM

Steven,
Thank you - it is what I suspected. If Python is to be a success on z/OS, and we want more extensions we need a solution to this. A generic zos would be good, I know this would involve changing core Python functions - and this is the start of a slippery slope.

regards

Colin

------------------------------
Colin Paice
------------------------------

Original Message

Original Message:
Sent: Fri January 07, 2022 01:56 PM
From: Steven Pitman
Subject: What is the best way to package on z/OS

Hi Colin,

Pip only provides the ability to install a package on any arbitrary platform if it's a pure python wheel, i.e that it contains no compiled extensions. The only other way would be to manually rename the file to to pretend it is, e.g. using the suffix <package>-cp310-none-any.whl, but this both not supported nor recommended, and has no guarantee to work.

Thanks,

Steven

------------------------------
Steven Pitman

Original Message:
Sent: Fri January 07, 2022 10:26 AM
From: Colin Paice
Subject: What is the best way to package on z/OS

Steven,

One more (hopefully the last). question

With an name like pymqi-1.12.0-cp310-cp310-os390_27_00_1090.whl. The 1090 represents my machine. I tried a build using --plat-name zos, which built, but this did not install.

Is there a way to name it so it can be on any z hardware - I cannot build on all supported hardware, or is there an option I need at install time? I could not find one.

(BTW I've given you a credit for your help in one of my blog posts - I hope that is OK)
Colin

------------------------------
Colin Paice

Original Message:
Sent: Thu January 06, 2022 09:14 AM
From: Steven Pitman
Subject: What is the best way to package on z/OS

Hi Colin,

One thing to note - in general for wheels, it's safe to use the package built for a given version on other minor releases. For example, building it for 3.10.0, it's generally safe to use it for 3.10.1 (PTFs) and onwards. But for major releases it is not, so it would not be guaranteed to be compatible with 3.8 or 3.9. The same is true even for building on earlier versions, e.g. build it on 3.8 and trying to use it on 3.9/3.10.

For a wheel package, you'll need to build it for all major versions and cannot just use one. Note that there were some issues in 3.8/3.9 with wheels that have been resolved in 3.10, so it's recommended you to use 3.10.

Regards,

Steven

------------------------------
Steven Pitman

Original Message:
Sent: Thu January 06, 2022 07:39 AM
From: Colin Paice
Subject: What is the best way to package on z/OS

Hi Steven,

Thank you for your very valuable input. I've managed to package my component.

I had to install wheel, and then add import setuptools into my setup.sh and then it worked.

I had to learn how to install wheel on my standalone machine.

I looked into trying to get one .whl package for both python 3.8 and 3.10 - but it looks like the .whl format has the Python version in the file name. If I had pure Python it can have PY2 and PY3.

I'm blogging all the lessons I learned, so I hope people following behind me will have an easier time!

Thanks again for your help

Colin

I'll update

------------------------------
Colin Paice

Original Message:
Sent: Wed January 05, 2022 04:59 PM
From: Steven Pitman
Subject: What is the best way to package on z/OS

Hi Colin,

I think the wheels binary format is what you are looking for. You can install it into a venv (see previous message) using pip3 install wheel, then build your package using python3 setup.py bdist_wheel (assuming it uses setuptools). It'll generate a binary file with both your Python + shared libs already compiled. Then this file can be installed by doing a pip3 install <file.whl>, even for users that do not have wheel installed, so it can still be a single command to install. This should only be used for Python 3.10, not lower versions of IBM Open Enterprise SDK for Python. To answer your question though, the reason it gives you the error is that the bdist distribution format is meant so that you extract/copy it yourself, pip does not handle that. When you use bdist_wheel, it will, which will fix your error.

There's been two main ways Python has typically done distribution of packages - distutils and setuptools. As of Python 3.10, distutils has now been marked as deprecated (to be removed in 3.12), so should no longer be used. That said, there is some documentation still available for it here (and this can be swapped to use setuptools by simply changing the import statements). Setuptools has replaced distutils, and largely provides the same experience / options, and you can find some examples on how to get started with it here - there isn't anything z/OS specific, so any general guide on distutils/setuptools can be used. You may find articles like this useful for simple examples on using setuptools with c extensions.

When it comes to the actual distribution, here's a quick explanation of the commands for creating the archive files (pip will prefer wheels when installing, i.e any dependency):

python3 setup.py sdist - Creates a source distribution, i.e. contains the raw source files, when the installing the user will compile any requirements
python3 setup.py bdist - Creates a binary distribution, i.e contains the compiled shared libs + Python files. The user must extract/copy the files to the destination
python3 setup.py bdist_wheel - Creates a binary distribution, similar to bdist but it will install directly with pip

Regards,

Steven

------------------------------
Steven Pitman

Original Message:
Sent: Wed January 05, 2022 12:00 PM
From: Colin Paice
Subject: What is the best way to package on z/OS

Steven,

I'm struggling with packaging and installing it.
I'm using python3 setup.py bdist to build my file, and in a virtual environment I'm using
python3 -m pip install -v -v -v --user --no-cache-dir /u/pymqi/dist/pymqi-1.12.0.os390-27.00-1090.tar.gz 1>a 2>b
This gives me
pip._internal.exceptions.InstallationError: file:///u/pymqi/dist/pymqi-1.12.0.os390-27.00-1090.tar.gz
does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.

and it is correct - setup.py is not in the package.

I read that pip does not handle bdist files, and I should use bdist_wheel. I could not get this to work, and it looks wheel is not installed by default on z/OS 3.10.

I also saw that setup.py from distutils is deprecated; but I could not find a way of building a C extension... I can use setuptools for pure python - but could not find how to compile things.

Is there any documentation of how to build an extension on z/OS?

regards

Colin

------------------------------
Colin Paice

Original Message:
Sent: Tue January 04, 2022 11:09 AM
From: Steven Pitman
Subject: What is the best way to package on z/OS

Hi Colin,

For storing the package, there's a few options you can take:

1) If the user has access to IBM z/OS XL C/C++ and it's open source, you could submit it to PyPI and have the user install the package with pip, like any other package
2) If they have access to Rocket Git, you could also only store it on GitHub, and install it with pip directly from there
3) If it's not open source, you could store the binary (the pax & tar formats can support file tagging) on Github or any other file management system (ex: Artifactory) and have the user download it

For installing IBM Open Enterprise SDK for Python without a sysprog, you can download the pax edition and follow the Installation documentation. If you already have Python installed with SMP/E or it was installed into a directory you do not have write access to, Python comes with the module venv (sample usage found here). This module can create a local environment of Python, which you can then use to install packages into for any testing required.

Steven

------------------------------
Steven Pitman

Original Message:
Sent: Tue January 04, 2022 07:35 AM
From: Colin Paice
Subject: What is the best way to package on z/OS

I am looking at porting the PYMQI package to z/OS.

What is the best way to package it and make it available? There are some .py files, and a C program which gets compiled and bound into pymqe.cpython-310.so .
I could create a zip package and store it on GitHub, so you would have to download, it, upload it and install it.

What is the right way of doing it?

Also in many customers, Python is on a read only zfs system.

What is the best way of installing it in my local system and running with it, (think of it as evaluation mode) rather than get the sysprog to update production.

Colin

------------------------------
Colin Paice
------------------------------

Python

Python