Originally posted by: LiorAronovich
New for IBM Spectrum Conductor 2.4.0: deploy an IBM Spectrum Conductor cluster within an IBM Spectrum LSF (LSF) cluster. This enhances the utilization of cluster resources by running IBM Spectrum Conductor workloads in conjunction with other types of workloads using fine-grain resource sharing within an LSF cluster.
Controlling and using an IBM Spectrum Conductor cluster
To start an IBM Spectrum Conductor cluster within an LSF cluster, use LSF’s bsub command to submit an IBM Spectrum Conductor controller job. This starts an IBM Spectrum Conductor management cluster within the LSF cluster. The IBM Spectrum Conductor management cluster runs system and instance group management services, and supports automatic cluster resizing for running more or less instance groups. Hosts are acquired exclusively from LSF for use in the management cluster.
After deploying an IBM Spectrum Conductor management cluster within an LSF cluster, you can access the management cluster and the instance groups running in the cluster using the IBM Spectrum Conductor cluster management console (GUI), CLI commands, and RESTful APIs. Workloads submitted via the IBM Spectrum Conductor instance groups run as jobs on compute hosts in the LSF cluster. The compute hosts used to run IBM Spectrum Conductor workloads are shared using fine-grain slot-level resource sharing with other types of workloads processed by LSF. The IBM Spectrum Conductor GUI has been enhanced in version 2.4.0 for the LSF environment; specifically, the GUI receives inputs related to LSF for acquiring resources and running workloads in the LSF cluster.
To terminate an IBM Spectrum Conductor management cluster, use LSF’s bkill command that specifies the IBM Spectrum Conductor controller job number.
Installing IBM Spectrum Conductor
The integration supports a simple installation and automatic setup of IBM Spectrum Conductor in an LSF environment. You perform a standard shared file system installation of IBM Spectrum Conductor in an LSF environment. When you start your IBM Spectrum Conductor management cluster using LSF’s bsub command, the IBM Spectrum Conductor cluster is automatically configured to run in a mode that works with the specified LSF cluster.
Running and monitoring IBM Spectrum Conductor workloads
With this integration, IBM Spectrum Conductor workloads run as jobs in the LSF cluster, under the execution user specified for the workload. When submitting IBM Spectrum Conductor workloads, this integration enables you to acquire resources from the LSF cluster, by submitting the workload jobs to LSF using LSF queues, application profiles, host groups, resource requirements, project names, and resource allocation limits, that are specified for the IBM Spectrum Conductor workloads. You can specify these LSF parameters for each type of resource allocation within each instance group of IBM Spectrum Conductor. By using these LSF parameters for IBM Spectrum Conductor workloads, you can plan, manage and monitor resource allocation and consumption for IBM Spectrum Conductor workloads using LSF capabilities. In addition, you can use LSF accounting tools for monitoring IBM Spectrum Conductor workloads in LSF.
The integration supports user impersonation for IBM Spectrum Conductor workloads running in an LSF cluster. Namely, the user submitting the IBM Spectrum Conductor application is the execution user under which corresponding jobs run in the LSF cluster. In addition, job groups are created in the LSF cluster for IBM Spectrum Conductor applications, under the execution users. Each job group corresponds to an application, and groups the jobs associated with the application.
Check it out!
For further information on integrating IBM Spectrum Conductor with LSF, see IBM Knowledge Center.
Try it out by downloading the evaluation version of IBM Spectrum Conductor 2.4.0, or, contact your IBM representative to purchase the full production version.
Let us know what you think
Comment and share your experience on our Slack channel!
#SpectrumComputingGroup