High Performance Computing

High Performance Computing Group

Connect with HPC subject matter experts and discuss how hybrid cloud HPC Solutions from IBM meet today's business needs.

 View Only

Installing IBM Spectrum Conductor 2.4.0 to a local file system with shared file system deployments

By Archive User posted Wed November 13, 2019 01:23 PM

  

Originally posted by: Jenna Lau-Caruso


image

Starting with IBM Spectrum Conductor 2.4.0, you can configure a cluster installed on a local file system, to deploy application instances and instance groups to a shared file system.
 

This configuration allows your to install your cluster's management services to local disk, increasing the performance of these critical services, while still taking advantage of time-savings and space-efficiencies of deploying application instances, Anaconda distributions and instance groups to a shared file system.
 

To set up shared deployments on a cluster which has been installed to a local file system, perform these simple configuration steps:

 

  1. Configure the ELASTIC_HARVEST_LOCATION environment variable
    • To configure at installation time, specify the ELASTIC_HARVEST_LOCATION environment variable to a location on a shared file system that is accessible by all hosts in the cluster.
    • To configure if you have already installed IBM Spectrum Conductor, modify the ELASTIC_HARVEST_LOCATION environment variable in the elk-shipper and SparkCleanup services:
      <ego:EnvironmentVariable name="ELASTIC_HARVEST_LOCATION">

/path/on/shared/filesystem

</ego:EnvironmentVariable>

Restart the services for the changes to take effect.

  1. Configure the ascd service for shared deployments
    1. Edit the $EGO_CONFDIR/../../ascd/conf/ascd.conf file to set ASCD_SHARED_FS_DEPLOY=ON.
    2. Restart the ascd service for the change to take effect.
       
  2. Optionally configure the SparkCleanup service to run on a single host
    1. Make the following changes to the SparkCleanup service to configure the service to run on a single host:
      <sc:MaxInstances>1</sc:MaxInstances>

<ego:ConsumerID>/ManagementServices/EGOManagementServices</ego:ConsumerID>

<ego:ResourceGroup>ManagementHosts</ego:ResourceGroup>

  1. Restart the service for the changes to take effect.

 Note: Skip this step if you plan to enable the shuffle service on your instance groups, specifying a local file system location for the shuffle data.

 

Once configured, ensure that you specify a location on a shared file system for all application instances, Anaconda distributions, and instance groups. A single host will automatically be selected for each package deployment.

 

Read more about it!

For more information about installing IBM Spectrum Conductor on a local file system with shared file system deployments, check out the detailed steps in IBM Knowledge Center

 

Let us know what you think!

Give it a try and share your feedback on our Slack channel.


#SpectrumComputingGroup
0 comments
0 views

Permalink