High Performance Computing

Originally posted by: Jonathan.Leung

Starting with IBM Spectrum Conductor 2.4.0, when you update your Spark version, you no longer need to create new instance groups. Instead, simply update existing instance groups to use that newer Spark version. This flexibility allows Spark configuration and user notebooks to be maintained, without modifying resource plans.

Once you have the updated Spark version on your IBM Spectrum Conductor 2.4.0 system, you're ready to update any existing instance group to use that Spark version. Follow these simple steps within the cluster management console:

1. Stop your instance group, then click Configure to access the Instance Group Configuration page.
   
   Note that for backwards compatibility, you can update your instance groups to higher Spark versions, which will be listed:
    
2. After making your selection, the system prompts you to review and back up your current Spark configuration to save your changes:
   
   Once you mark this acknowledgement, click Modify Instance Group. Your instance group will be ready to use.

For more information updating instance groups to use a new Spark version, see IBM Knowledge Center.

As always, if you have questions or comments, reach out to us on our Slack channel!