High Performance Computing Group

High Performance Computing Group

Connect with HPC subject matter experts and discuss how hybrid cloud HPC Solutions from IBM meet today's business needs.


#Cloud
#IBMCloud
 View Only
Expand all | Collapse all

Spark Instance Group Logs?

  • 1.  Spark Instance Group Logs?

    Posted 04/05/16 06:10 AM

    Originally posted by: ErezHadad


    Hi,

     

    I recently installed Conductor in a RHEL 7.2 VM.

    After the installation, I also added the Spark 1.5.2 extension and the Zeppelin 0.5.5 extension.

    However, when I try to start a new Spark Instance Group (SIG), it always gets into an error state before I can do anything with it.

    Before I raise this as a Conductor issue, I would like to investigate the SIG logs to see if I'm doing something wrong.

     

    Can anyone please direct me to the logs of the SIG?

     

    Regards,

    Erez Hadad


    #SpectrumComputingGroup


  • 2.  Re: Spark Instance Group Logs?

    Posted 04/06/16 06:48 AM

    Originally posted by: ErezHadad


    I believe I have found the logs.

    To view the logs of a particular service in a SIG that has crashed, you work with the PMC as following:

    1. In the SIG list (Workloads->Spark->Service Instance Groups), click on the SIG to open it
    2. In the SIG view, go to the "Services" tab to look at the list of processes and find the one who failed.
    3. Click on the failed service name to open the "Service Details" view
    4. In the "Service Details" view, click on the "Instances" tab
    5. Select the failing instance of the service by clicking on the radio button near its name
    6. From the menu of the "Instances" tab, click on "View Logs"
    7. This will open another dialog that shows a list of log files. The top one is the most recent.
    8. Click the top log file and the browser will ask you to download the text file - open it in editor etc

    Warning - if you pause during the process, the console might reset back to the SIG list view.

     


    #SpectrumComputingGroup