Hello Hermann, I'm not an expert on Kubernetes or Liberty Operators I'm afraid, but here is what we have,
- If enabling the peer recovery with a mounted PVC when running a Deployment configuration works at all if we run more than one pod?
- Peer recovery works with multiple Liberty servers. These can be in separate pods. Each server must use a distinct recovery ID.
- if we run a Deployment with more than one replica all replicas use the same PVC for the tranlogs. Doesn't that cause locking issues?
- No. File-based transaction logs using peer recovery rely on file-locking. The mounted filesystem must support NFS 4 file locking. Each peer group will use a single parent directory. Each directory under there is specific to one server and can only be accessed by one server at a time. Only if a server stops updating its own directory will another server step in to take over and recover its in-doubt transactions.
- If we use the StatefulSet type of deployment - will peer recovery work for the tranlogs?
- Yes. Each server must use a distinct recovery ID.
- What are the recommendations to handle the translogs?
- Option 1: use a database for transaction logs (this is the preferred option with fewer topological constraints)
- Option 2: mount a shared directory at /tranlogs, configure the transactionLogDirectory to /tranlogs, set the recovery group the same and the recovery identity different on each server.
- When should the OpenLibertyApplication be deployed as a StatefulSet and when as a Deployment?
- A StatefulSet should always be a safe option.
- An application that only relies on remote resource managers (e.g. DB2, MQ) to store state will work in a deployment to multiple pods.
- If it is, then peer recovery will ensure resources are unlocked if a server goes away without completing its transactions.
------------------------------
Dave Schlick
WebSphere Support Specialist
IBM UK Limited
Portsmouth
------------------------------
Original Message:
Sent: Thu April 24, 2025 04:44 AM
From: Hermann Huebler
Subject: Liberty Operator: Recommendations for configuration of transaction logs on Kubernetes / OpenShift environments
We are deploying out Liberty Applications on RedHat Openshfit using the OpenLibertyApplication CRD provided by the Liberty Operator. In this environment we are seeking recommendations for how to handle the tranlogs to allow scaling the application.
The following link in the Liberty documentation https://openliberty.io/docs/latest/transaction-service.html#cloud recommends to setup a recoveryGroup`and recoveryIdentity to allow peer recovery for scaled pods. As we are using an RWX PersistentVolume which we mount over /opt/ol/wlp/output/defaultServer/tranlog this would mean that the tranlogs are shared across all replicas of the Deployent. Will/can this work at all?
On the other hand the OpenLibertyApplication CRD provides the capability to deploy the application as a StatefulSet (instead of a Deployment) with pod specific volumes which could be mounted over /opt/ol/wlp/output/defaultServer/tranlog for each pod to provide pod specific tranlogs.
Therefore I'm trying to understand
- if enabling the peer recovery with a mounted PVC when running a Deployment configuration works at all if we run more than one pod?
- if we run a Deployment with more than one replica all replicas use the same PVC for the tranlogs. Doesn't that cause locking issues?
- if peer recovery works if we deploy the OpenLibertyApplication as a StatefulSet?
- what are the recommendations to handle the translogs?
- when should the OpenLibertyApplication be deployed as a StatefulSet and when as a Deployment?
Thanks a lot in advance for any feedback and clarification, Hermann
------------------------------
Hermann Huebler
Cloud Architect
Alpium IT Solutions GmbH
Vienna
Austria
#IBMChampion
------------------------------