Introduction
IBM Certified Containers for Sterling Order Management on Red Hat OpenShift Container Platform offer a more simple and more efficient way to install, deploy, manage, and scale across multiple environments. IBM Sterling Order Management Software Operator is used to manage and automate the lifecycle of the Sterling Order Management Software. The Operator provides custom resources to automatically create or update the Kubernetes resources to achieve the expected state. Please refer to IBM documentation on operators for more details on this.
The Operator provides the Custom Resources (CRs) for OMEnvironment, OMServer, Order Hub, and Order Service.
Kustomize is a tool that allows customizing custom resources in a declarative and maintainable way. Here we have documented our experience with using Kustomize to deploy IBM Certified Containers for Sterling OMS using operators on the Red Hat OpenShift Container Platform (RHOCP).
CR deployment Directory structure
Kustomize uses the base and overlays directory structure. The base directory has the resource definitions that are needed for all overlays. The overlay has customization and additional resources based on the environment. The base and overlay directories contain a kustomization.yaml file that specifies the resources to be customized and any additional configuration for Kustomize.
For example, let's say you want to customize a Deployment resource. You could create an overlay directory with the following structure:
|
base
|__ deployment.yaml
overlays
|--- dev
|__ kustomization.yaml
|__ prod
|__ kustomization.yaml
|
In the base/deployment.yaml file, you would specify the resource definition.
In the overlays/dev/kustomization.yaml file, you would specify the resources to be customized and any additional configuration for Kustomize.
For example, you can re-use the deployment resource definition from the base with different namespaces and add suffixes to the name using the following kustomization.yaml
|
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: dev
commonLabels:
env: dev
nameSuffix: -dev
resources:
- ../../base/deployment.yaml
|
Once you have created your overlays directory and populated it with the necessary files, you can apply your customizations to the Deployment using the following command:
|
$ oc apply -k overlays/dev
|
Using Kustomize this way allows you to easily customize your resources and apply the customizations consistently across your RHOCP cluster. It also makes it easier to maintain your customizations, as you can track changes to your customized resources using version control systems like Git.
OMS CD Resources
The OMS deployment typically involves the following resources
- OMEnvironment
- DB Verify to apply database entity changes
- CDT Export and Import
OMEnvironment
The OMEnvironment CR is used to bring up a fully functional Order Management Software integrated environment including Order Service, Order Hub, and multiple agent and integration servers. Please refer to IBM documentation for more details on the OMEnvironment Custom Resource.
The database connection properties for each environment can be overridden or added using the kustomize overlays for different environments.
|
base
|-- om-environment.yaml
overlays
|--- mc
|--- kustomization.yaml
|--- database/db-config.yaml
|__ customerproperties-patch.yaml
|__ dev
|--- kustomization.yaml
|--- database/db-config.yaml
|__ customerproperties-patch.yaml
|
The sample kustomization.yaml to use the customer overrides properties and database configuration looks like below.
|
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: dev
commonLabels:
env: dev
nameSuffix: -dev
resources:
- ../../base/om-environment.yaml
patches:
- path: ./database/db-config.yaml
- path: ./customerproperties-patch.yaml
|
The db-config.yaml has the required database configuration for the OMS application.
|
apiVersion: apps.oms.ibm.com/v1beta1
kind: OMEnvironment
metadata:
name: oms-env
spec:
database:
db2:
dataSourceName: jdbc/OMDS
host: dbhost.ibm.com
name: IBMOMS
port: 50000
schema: OMSDEV
secure: false
user: omsdbuser
|
The customerproperties-patch.yaml has the properties defined for the OMS application.
|
apiVersion: apps.oms.ibm.com/v1beta1
kind: OMEnvironment
metadata:
annotations:
name: oms-env
spec:
serverProperties:
customerOverrides:
- groupName: BaseProperties
propertyList:
yfs.customProperty: "true"
|
The kustomization.yaml can be used to patch various objects in the resources such as server profiles, agent servers, integration servers, application servers, image names, variants, tags, JVM Arguments for the servers, queue bindings config maps to name a few.
Configuration Deployment Tool
Configuration Deployment Tool(CDT) is used to export the configuration data to XMLs. The exported CDT XMLs can be imported to the required target databases. The tool allows the seamless application of the business rules to target environments such as development, test, and production environments.
CDT Export
CDT Export is modeled as a Kubernetes Job and runs only for the Master Configuration environment.
The job uses ydkprefs.xml and exportCDT.sh script stored as config maps. The config maps are created as part of the deployment. The job uses ombase image to create a container.
The ydkprefs.xml stores the CDT table preferences to be ignored for CDT export. The ydkprefs.xml is mounted to the /opt/ssfs/runtime/resources/ydkresources/ydkprefs.xml. The environment variables are introduced to allow/disallow the CDT Export and to restrict the namespace in which the job can be executed. The exportCDT.sh updates the system_overrides.properties with the required database connection details using the config maps created by the OMEnvironment (container scripts packaged with the ombase image such as evaluateEnvs.sh and collateEnvs.sh are used for this). The CDT tool (packaged with the OMS runtime), cdtshell.sh is invoked to export the CDT to XMLs. The CDT image is created with the generated CDT export XMLs and pushed to the private repository.
|
resources:
- ./cdt/cdt-export-job.yaml
configMapGenerator:
- name: ydk-prefs-cm
files:
- ydkprefs.xml=./cdt/ydkprefs.xml
options:
disableNameSuffixHash: true
- name: cdt-export-script-cm
files:
- ./cdt/exportCDT.sh
options:
disableNameSuffixHash: true
patches:
- target:
group: batch
version: v1
kind: Job
name: oms-cdt-export
patch: |-
- op: add
path: /spec/template/spec/containers/0/env/-
value:
name: RUN_CDT
value: "Y"
|
CDT Import
The CDT Import process is used to apply the business rules and other configuration changes to the target environment databases.
The CDT Import is modeled as a Kubernetes job similar to the CDT Export Job. The job creates a container with ombase image. The cdt image is pulled and the contents are copied to a directory in the pod.
The ydkprefs.xml stores the CDT table preferences to be ignored and appended for CDT import. The ydkprefs.xml is mounted to the /opt/ssfs/runtime/resources/ydkresources/ydkprefs.xml. The importCDT.sh is invoked in the ombase container. The importCDT.sh updates the system_overrides.properties with the required database connection details using the config maps created by the OMEnvironment (container scripts packaged with the ombase image such as evaluateEnvs.sh and collateEnvs.sh are used for this). The CDT tool (packaged with the OMS runtime), cdtshell.sh script is executed to import/deploy the CDT export XMLs. The job has an environment variable to determine whether to deploy the changes or only compare.
The CDT import job can be included and customized by adding the below in the kustomization.yaml with the required values in the environment overlay, overlays/dev.
|
resources:
- ./cdt/cdt-export-job.yaml
configMapGenerator:
- name: ydk-prefs-cm
files:
- ydkprefs.xml=./cdt/ydkprefs.xml
options:
disableNameSuffixHash: true
- name: cdt-import-script-cm
files:
- ./cdt/importCDT.sh
options:
disableNameSuffixHash: true
patches:
- path: ./cdt-import-patch.yaml
- target:
group: batch
version: v1
kind: Job
name: oms-cdt-import
patch: |-
- op: add
path: /spec/template/spec/containers/0/env/-
value:
name: IMP_CDT
value: "Y"
|
The cdt-import-patch.yaml can include the overrides to the CDT import job such as image tags, service account, environment specific config maps.
|
apiVersion: batch/v1
kind: Job
metadata:
name: oms-cdt-import
spec:
template:
spec:
serviceAccountName: oms-sa-dev
containers:
- name: om-base
image: "cp.icr.io/cp/ibm-oms-enterprise/om-base:10.0.2209.0-amd64"
initContainers:
- name: cdt-image
image: "repository.ibm.net:5000/repository/container-image-registry/oms/om-cdt/om-cdt:10.0.2209.0-20221114-142406"
volumes:
- name: container-utils
configMap:
defaultMode: 493
name: oms-env-dev-scripts
- name: config
configMap:
defaultMode: 420
name: oms-env-dev-system-props
|
DB Verify
OMS database can be extended by creating new custom tables, extending some existing tables, adding indexes, sequences, views, etc. The OMS fix packs also modify the database entities. The database entity repository is stored in the OMS runtime along with the extensions. The dbverify.sh is an OOTB tool that can be used to compare the runtime entity repository against the database. The database changes can be applied or shared with the DBA for review.
The DB Verify process is modeled as a Kubernetes job that uses config maps created by the OMEnvironment CR. The shell script, rundbverify.sh executes db verify, and database extensions are created as config maps. The job creates a container using om-base/om-agent. The database extensions and shell script are mounted on the container to compare the runtime against a database.
The rundbverify.sh updates the system_overrides.properties with the target database details using the config maps created by the OMEnvironment CR (container scripts packaged with the ombase image such as evaluateEnvs.sh and collateEnvs.sh are used for this). The DB verify tool (packaged with the OMS runtime), dbverify.sh is invoked to compare and apply the database DDL changes to the target database.
The job can be defined in the base directory with the customizations in the overlays. The custom database extensions need to be included in the overlays/dev/database/extn/ directory.
|
resources:
- ../../base # includes all the resources defined in the base/kustomization.yaml
configMapGenerator:
- files:
- ./database/extn/YFS_ITEM_NODE_DEFN.xml
- ./database/extn/YFS_ITEM.xml
name: db-extn-cm
options:
disableNameSuffixHash: true
patches:
- path: ./dbverify-patch.yaml
- target:
group: batch
version: v1
kind: Job
name: oms-dbverify
patch: |-
- op: add
path: /spec/template/spec/containers/0/env/-
value:
name: RUN_DBVERIFY
value: "N"
|
The dbverify-patch customizes the DB verify job with the environment-specific config maps and the container image tag.
|
apiVersion: batch/v1
kind: Job
metadata:
name: oms-dbverify
spec:
template:
spec:
serviceAccountName: oms-sa-dev
containers:
- name: oms-dbverify
image: "cp.icr.io/cp/ibm-oms-enterprise/om-base:10.0.2209.0-amd64"
volumes:
- name: container-utils
configMap:
defaultMode: 493
name: oms-env-dev-scripts
- name: config
configMap:
defaultMode: 420
name: oms-env-dev-system-props
|
Conclusion
In summary, Kustomize is a powerful tool for customizing your resources on RHOCP in a declarative and maintainable way. The Kustomize can also be used on the non-RHOCP Kubernetes. By using Kustomize overlays, you can easily apply customizations to your resources and track changes to your customized resources using version control systems.
Many thanks to Murthy (rtumkur@us.ibm.com) for the guidance and mentorship in the use of kustomize for IBM Sterling Order Management deployment.
IBM Expert Labs team has successfully helped Sterling order management customers in deploying OMS to Kubernetes platforms. Reach out to us for IBM experts who can help you plan and deploy the IBM Sterling Order Management Certified Containers.
