1) When there are no dependencies, directly using the root installation path of the SDK for Python. Installing into the root installation of Python is not recommended as it is deleted when updating.
2) Using Virtual Environments to contain the applications dependencies. A virtual environment is an isolated / self-contained directory which contains all packages required for a given project. For recommendations on how these can be setup, you can see the blog Best Practices – Managing Python Installs.
3) Using a container. A container is a way deploying z/OS UNIX applications and provides application & OS isolation. You can find more information about creating & using them in our blog Creating your first Python Container Image.
For this blog, we’re going to be going over what are the differences between these different ways of deploying Python applications, the pro’s & cons of each, and when would one be preferred over the other.
Virtual Environments provide a middle ground between direct installation and full containerization. You can create a virtual environment using Python’s built-in venv module, and it creates a directory owned by that user for storing packages. Virtual Environments provide the following benefits:
1) It’s the standard way to isolate your Python packages. For each project, a virtual environment can be created which contains all dependencies required to run it.
2) Reproducible via a requirements.txt file - which allows you to specify specific versions of packages that should be used to help guarantee consistency across different systems.
3) Simple workflow - create a virtual environment, install your dependencies, then you can start using it immediately for both your development work & deployment.
4) No elevated privileges required - users can create and manage their own virtual environments without system administrator access.
It does have some drawbacks however:
1) As virtual environments are Python specific, they only isolate Python packages. If there are requirements to build these packages such as C/C++ libraries, they are not isolated to specific versions. Likewise with any additional binaries on your systems.
2) Inconsistently set up environments may cause different behavior when transferring across users & machines.
What are some situations when you’d want to use a virtual environment directly?
1) When wanting to share a set list of Python packages with multiple users on a system while maintaining isolation from other projects.
2) When using Python to perform automation on z/OS or interacting with subsystems outside of z/OS UNIX.
3) When having Python interoperate with other languages already running on z/OS, for example JCL or REXX.
4) Deployment of a Python application that doesn’t need to be run across multiple systems or does not have complex build dependencies.
5) For development and testing environments where you need quick iteration without the overhead of container builds.
A container is a completely self-contained package to run a given application. It contains everything, including system libraries and standard tooling. On z/OS, this is provided by the product z/OS Container Platform, which then has various premade images for it -– providing container images for a base image of z/OS UNIX, and ones that contain Python, C/C++, Java, or more for building applications with each respective language. Container images provide the following benefits:
1) They’re completely self-contained, containing all libraries, binaries, and any application code that is required to run both the program, the filesystem and network interfaces. This guarantees that if you download a container image, it has a known good starting point.
2) They are portable across systems - create the container image on one machine and it can be transferred to another and run. Since it contains everything required to run it, the program will still run as expected, eliminating "it works on my machine" problems.
3) They can easily scale up and down based on demand. Containers can be run using Podman but can also be controlled with Kubernetes for orchestration and automated scaling.
4) Version control and rollback - container images can be tagged with versions, making it easy to roll back to a previous version if issues arise.
5) Consistent environments - development, testing, staging, and production can all use the same container image, ensuring consistency across the entire deployment pipeline.
Some downsides of using a container instead of a virtual environment:
1) It requires elevated privilege to run a container image, though the process inside it do not need to be elevated.
2) Consuming more system resources than a virtual environment such as disk space requirements, memory, and CPU usage due to the inclusion of the entire operating system layer.
3) Longer build times - creating and updating container images takes more time than installing packages in a virtual environment.
4) Increased complexity - building, managing, and deploying containers requires additional knowledge and tooling compared to virtual environments.
What are some use cases when you would want to deploy your application using containers?
1) When you want to deploy the same application on multiple systems and want the same environment. It can be used for build, test, staging & deployment.
2) When you would like to be able to scale up & down your application based on demand.
3) When wanting to be able to roll back versions quickly in case of regressions or breakage.
4) When your application has complex system-level dependencies that need to be isolated and versioned alongside your Python code.
5) For microservices architectures where each service needs to be independently deployable and scalable.
Best Practices for Virtual Environments & Containers
1) Install only what is required to run the application and keep it all in the same spot. For example, using a requirements.txt to specify Python dependencies, and use that to install your dependencies in both a virtual environment and container image. Additionally, only install what is required for the given task. If deploying a given application, the packages required for testing it do not need to be installed in production.
2) Pin dependency to their exact version that is wanted to avoid accidental breakage - do not use “latest” for either the Python version or contain image version. Keep development, CI, staging, and production all in sync to ensure consistent behavior across environments.
3) Run your deployed application with as minimal permissions as possible, using the lowest required user account as possible, for example having a dedicated user for running an application, not a developers account. This follows the principle of least privilege and improves security.
4) Do not bake environment variables (such as database hostnames) or secrets into a container image. Keep these separate and inject them at runtime using environment variables, configuration files, or secret management systems. This prevents sensitive information from being stored in version control or container registries.
5) Regularly update dependencies and base images to include security patches and bug fixes. Establish a process for testing and deploying updates to minimize security vulnerabilities.
6) Document your deployment process thoroughly, including how to create environments, install dependencies, and configure the application. This ensures consistency and makes it easier for team members to understand and maintain the deployment.
Conclusion
Choosing the right deployment method depends on your specific requirements:
- Use direct installation for simple scripts with no dependencies that need to be available system-wide.
- Use virtual environments for most Python applications that need dependency isolation but don't require full system-level isolation. This is the sweet spot for many z/OS Python applications.
- Use containers when you need complete environment reproducibility, plan to deploy across multiple systems, require system-level dependency isolation, or need to scale your application dynamically.
In many cases, you may use a combination of these approaches: virtual environments for development and testing, and containers for production deployment. The key is to choose the method that best balances your needs for isolation, reproducibility, resource usage, and operational complexity.
Want to learn more about using Python on z/OS? Visit our Python Portal for a convenient way to access more resources.