Originally posted by: thloeber

When creating a Spark Instance Group using Spark 1.5.2 with Zeppelin 0.5.5 the create is successful but the deploy fails with permissions errors pm $EGO_TOP.  As who do this commands run? I would assume egoadmin (which is the id I specified during the install).  The $EGO_TOP directory and sub-directories are owned by eqoadmin:egoadmin with 755 permissions. 

Logging on to RS server with user Admin and consumer /.
Successfully logged on to RS server.
Starting to install package SCMonFS-Spark-1-5-2_Spark1.5.2
Successfully started package install
HOST                            STATUS     
scmonspark01.raleigh.ibm.com    ERROR       Domain <Application>: The install command, /opt/ibm/platform/soam/7.1.1/linux2.6-glibc2.5-ppc64/bin/soamdeploy download SCMonFS-Spark-1-5-2_Spark1.5.2 -v 3 -c / -r LINUXPPC64, has returned an error exit code, -2. Domain <Application>: The install command, ./scripts/deploy.sh, has returned an error exit code, 1. touch: cannot touch '/opt/ibm/platform/temp.txt': Permission denied Location "/opt/ibm/platform" does not have write permission.  Domain <Application>: The package SCMonFS-Spark-1-5-2_Spark1.5.2 failed to install. The install command specified in the deployment.xml file returned an error.
scmonspark02.raleigh.ibm.com    ERROR       Domain <Application>: The install command, /opt/ibm/platform/soam/7.1.1/linux2.6-glibc2.5-ppc64/bin/soamdeploy download SCMonFS-Spark-1-5-2_Spark1.5.2 -v 3 -c / -r LINUXPPC64, has returned an error exit code, -2. Domain <Application>: The install command, ./scripts/deploy.sh, has returned an error exit code, 1. touch: cannot touch '/opt/ibm/platform/temp.txt': Permission denied Location "/opt/ibm/platform" does not have write permission.  Domain <Application>: The package SCMonFS-Spark-1-5-2_Spark1.5.2 failed to install. The install command specified in the deployment.xml file returned an error.
Command failed on 2 hosts.

Originally posted by: Jenna Lau-Caruso

The deployment of the Spark and Zeppelin packages are run by the execution user that you specified when you created your Spark instance group.

It also looks like you trying to deploy to an existing directory. Generally, we would recommend deploying to a new directory that is unique for your Spark instance group (for example, /opt/ibm/spark152 or /opt/ibm/platform/spark152)

Originally posted by: thloeber

I created a /opt/ibm/platform/spark152 directory on both hosts in the resource group.  And tried the create again, but this time specified egoadmin as the execution user. Previously I had used a user I created via the master console web ui which I granted Cluster Admin privileges. This user value is not explained very well in the documentation. The creation was successful but now the deploy has been running for over 30 minutes. I've attached a screenshot below.

On both hosts I can see new files and directories in the /opt/ibm/platform/spark152 directory.

[egoadmin@scmonspark01 spark152]$ ls -al
total 286332
drwxr-xr-x  5 egoadmin egoadmin       103 Apr 20 10:18 .
drwxr-xr-x 20 egoadmin egoadmin      4096 Apr 20 10:13 ..
drwxr-xr-x  2 egoadmin egoadmin       140 Apr 20 10:18 scripts
drwxr-xr-x 12 egoadmin egoadmin      4096 Feb  8 15:33 spark-1.5.2-hadoop-2.6
-rw-rw-r--  1 egoadmin egoadmin 293195189 Apr 20 10:18 spark-1.5.2-hadoop-2.6.tgz
drwxrwxrwx  4 egoadmin egoadmin       101 Apr 20 10:18 Zeppelin-0.5.5

Originally posted by: thloeber

I had to refresh the browser manually. Once I did that I could see the Park Instance was created and deployed and I was able to start it. It did not start automatically although I left that box checked when I created it.

Originally posted by: Jenna Lau-Caruso

I'm glad to hear that your deployment was successful.

There may be some confusion around starting of the Spark Instance Group. This is a manual step, as you observed. The option provided during Spark Instance Group creation is to automatically start the deployment, not the services.