High Performance Computing

High Performance Computing Group

Connect with HPC subject matter experts and discuss how hybrid cloud HPC Solutions from IBM meet today's business needs.

 View Only

Interim fix for IBM Platform Conductor for Spark is now available

By Archive User posted Tue April 19, 2016 12:03 PM

  

Originally posted by: DonBis


imageAn interim fix for IBM Platform Conductor for Spark v1.1 is now available for download from IBM Fix Central. This interim fix adds support for Spark 1.6.1 and the Zeppelin 0.5.6 notebook, along with fixes and enhancements for Spark 1.5.2. It also contains other significant fixes and enhancements to the core product and the Platform Management Console (PMC). This interim fix applies for Linux x86-64 only. It is recommended that all customers apply this fix.

Before you download any update from Fix Central:

  • Know your IBM user ID and password. You are required to log in to the Fix Central web site before you can download an update.
  • To search for a Spark version, click the Find Product tab and enter Platform Conductor.
  • To get information about the download process or help during the process, see Fix Central help.

You can download the interim fix from the IBM Fix Central pcs-1.1-build398394 page. To download, simply click the pcs-1.1-build398394 link or select the check box next to interim fix: pcs-1.1-build398394 and click Continue at the bottom of the page.

For detailed instructions on how to install this fix, as well as a list of all of the enhancements and fixes, download the file called "Read Me" from the pcs-1.1-build398394 page. Readme files for Spark 1.5.2, Spark 1.6.1, and the Zeppelin 0.5.6 notebook are also available for download on the same page.

We also have an update to the IPython-Jupyter notebook sample integration available. This update adds support for Spark 1.6.1 and IBM Platform Conductor for Spark authentication security integration.

If you have not downloaded IBM Platform Conductor for Spark yet, you can find an evaluation version on our Service Management Connect page. If you have any questions about IBM Platform Conductor for Spark or any of our new fixes and enhancements, post them in our forum!


#SpectrumComputingGroup
2 comments
0 views

Permalink

https://community.ibm.com/community/user/blogs/archive-user/2020/05/28/interim-fix-for-ibm-platform-conductor-for-spark-is-now-available

Comments

Archive User
Wed July 13, 2016 03:21 PM

Originally posted by: Jenna Lau-Caruso


Hi Ben, Sorry to hear that you have been having difficulties applying the interim fix. Unfortunately, due to the nature of this interim fix and the way it is applied, there is no easy way to verify that the patch has been fully applied without double checking each file to verify it has been copied successfully to the correct locations. The --egoversion value obtained from the spark-submit command refers to the build of the Spark package, not the Conductor with Spark version overall. As you observed, this value will be different for Spark 1.4.1 and Spark 1.6.1 once the new Spark 1.6.1 package has been added. Thank you for your feedback, I will certainly bring it back to the development team for consideration for any future interim fixes.

Archive User
Wed July 13, 2016 10:58 AM

Originally posted by: Ben_Liao


After I applied interim fix: pcs-1.1-build398394, how can I know that the patching has failed or succeeded? Any approach rather than checking all replaced files? I know that spark-submit --egoversion can check the ego build ID. But I found that even in the same master host, if I ran this under spark1.4.1 deployment directory, it responded "Build 374706" which should be the original one.However, if I did that under spark1.6.1 deployment directory, I got "Build 398394". However, even in the latter situation, I found the patching was not successful, because some files were not replaced yet(checked the modified time of files). This caused master batch service failure during startup. It took us a long time to troubleshoot the issue. Thanks.