IBM Spectrum Conductor 2.5.0 includes cloud bursting capabilities that enable to dynamically add and remove cloud hosts based on workloads resource demand.
Take advantage of the cloud bursting capabilities to:
- Reduce the total cost of ownership, as hosts can be provisioned only when needed.
- Increase flexibility in using multiple cloud providers and different types of resources according to workload requirements.
- Improve scalability and resiliency by leveraging the scalability and resiliency capacities of cloud providers.
- Improve security and compliance by leveraging state-of-the-art security and compliance features that are implemented by cloud providers.
IBM Spectrum Conductor 2.5.0 uses a system service called
HostFactory to connect with cloud providers. A requester plug-in monitors workload requirements and cluster utilization in the IBM Spectrum Conductor cluster. The requester plug-in generates scale-out cloud requests when workload resource demand exceeds resource capacity, and generates scale-in cloud requests when the cluster has excess resource capacity. Cloud resource requests can be also submitted manually or based on configured schedule by using a graphical user interface (GUI) or RESTful API. Provider plug-ins work with cloud provider interfaces to provision and return cloud hosts. The supported cloud providers are IBM Cloud®, Amazon Web Services, and Microsoft Azure.
IBM Spectrum Conductor 2.5.0 includes the following new cloud bursting capabilities:
Multi-tenancy enables to define multiple requester instances and multiple provider instances, and associate each requester instance to work simultaneously with multiple provider instances. A requester instance specifies the configuration parameters of generating cloud resource requests. A provider instance specifies the configuration parameters of working with a cloud provider account. Therefore, each requester instance can work simultaneously with multiple cloud providers from the supported providers and with multiple cloud accounts. In addition, the configuration of requester and provider plug-ins is now separate from the configuration of requester and provider instances. Each dynamic instance is associated with a plug-in.
The cluster utilization scaling policy of the IBM Spectrum Conductor plug-in for cloud bursting, generates scale-out and scale-in requests automatically based on comparison of utilization of hosts in the cluster to thresholds of utilization and time duration. Monitoring can be done on the entire cluster or on a list of host groups. This policy enables to apply cloud bursting for any type of workload.
The workloads service level scaling policy of the IBM Spectrum Conductor plug-in for cloud bursting, generates scale-out and scale-in requests automatically based on comparison of required completion times for workloads with projected completion times for the workloads, which are calculated by using applications profiling. Monitoring can be done on the entire cluster, on a list of instance groups, or on a list of host groups. This policy is designed for Spark batch applications.
The cost-based host selection mechanism enables to automatically choose the most cost-effective provisioning solution across cloud providers and host types, for any resource request. Typically, cloud providers offer multiple types of hosts, and each host type can be assigned with a different price and charging model. In addition, each requester instance can work with multiple cloud provider instances. The cost-based host selection mechanism enables to minimize costs by selecting the most cost-effective provisioning solutions.
The scheduled host requests mechanism enables to configure and automatically generate scale-out and scale-in cloud resource requests based on schedule. You can configure single instance requests or recurring requests, start times, return times, cloud provider instances to use, and the requested resources.
For more information about the cloud bursting capabilities in IBM Spectrum Conductor 2.5.0, see Cloud bursting with host factory.
Give it a try and tell us what you think by downloading IBM Spectrum Conductor 2.5.0 on Passport Advantage or the evaluation version! We hope you are as excited as we are about this new release!
Log in to this community page to comment on this blog post. We look forward to hearing from you on the new features, and what you would like to see in future releases.
#SpectrumComputingGroup