IBM Spectrum Computing Group

Enabling LDAP user authentication with integrated infrastructure deployment of IBM Spectrum Conductor with Spark v2.1

By Archive User posted Wed August 17, 2016 03:31 PM

  

Originally posted by: Benjamin_Avdicevic


Introduction

IBM Spectrum Conductor with Spark v2.1 allows you to setup a multitenant Spark environment  starting from bare-metal hardware.  This is done by choosing the Integrated Infrastructure installation method during deployment. Integrated installation supports two user authentication methods: local and LDAP. LDAP authentication requires additional manual steps after installation to configure.  The purpose of this article explains how to setup LDAP user authentication in your IBM Spectrum Conductor with Spark v2.1 integrated infrastructure installation.

 

Technical Overview

Infrastructure integrated installation of IBM Spectrum Conductor with Spark v2.1 uses optimized IBM Spectrum Cluster Foundation v4.2.2 for deploying, managing and monitoring your infrastructure.  IBM Spectrum Cluster Foundation provides its own Infrastructure Management Console for provisioning and managing bare-metal servers.  Once you deploy an IBM Spectrum Conductor with Spark cluster using the Infrastructure Management console, you will have an additional console for managing applications and workload, called Application Management Console.  Figure 1 illustrates the difference in scope between the two management consoles.

 

Figure 1 Infrastructure management console

image

 

The process to enable LDAP user authentication in your integrated infrastructure Spark environment will configure both infrastructure and application consoles for LDAP user authentication.  This will ensure that LDAP user authentication works seamlessly across your entire Spark environment.

 

Furthermore, Integrated Infrastructure installation method can be applied in two scenarios:

  • Without high availability (i.e. for testing and evaluation)
  • With high availability (i.e. for production)

 The article will explain how to enable LDAP user authentication for both scenarios.

 

            NOTE:  Inside the Infrastructure Management Console, hosts are referred to as nodes. 

 

Assumptions

  • You must have an existing corporate LDAP server that meets the IBM Spectrum Conductor with Spark v2.1 requirements.
  • All management and compute hosts must have direct network access to your corporate LDAP server, as in Figure 2 below.
  • You have completed the installation IBM Spectrum Conductor with Spark v2.1 on your management host(s) following the Integrated Installation method documentation.

 

Limitations

  • There is no fully automated way to setup LDAP authentication during Integrated Infrastructure installation. Following process must be followed after installation to setup LDAP user authentication.
  • Integrated Infrastructure installation method does not support Kerberos, or other network user authentication methods.
  • LDAP user authentication must be enabled prior to deploying your IBM Spectrum Conductor with Spark cluster via the cluster template using the Infrastructure console. 

 

Enabling LDAP in deployment without high availability

This section describes the process of enabling LDAP user authentication in environment installed without high availability (HA).  Once enabled, both local and LDAP users will be able to authenticate in your infrastructure and application management consoles.

Overview

Figure 2 illustrates a typical network topology for installation without high availability.  For example, you have a single management host, an LDAP server, and one or more compute hosts, where the compute and management hosts have a direct network access to the LDAP server.

Figure 2 Deployment overview of Integrated Installation without high availability

image

 

Process

The following steps should be done as root user on your IBM Spectrum Conductor with Spark management host. 

 

To ensure you’re ready to start, run the pcmadmin command on your management host to check status of required services, like this:

 

[root@cs21demo ~]# pcmadmin service list

SERVICE             STATE          ALLOC     SERVER             

=======             =====          =====     ======             

ACTIVEMQ            STARTED        5         cs21demo           

PCM-PLC             STARTED        7         cs21demo           

PCM-PURGER          STARTED        8         cs21demo           

PCM-WEB             STARTED        15        cs21demo           

PCMD                STARTED        14        cs21demo            

PTC                 STARTED        9         cs21demo           

RULE-ENGINE         STARTED        10        cs21demo           

elk-elasticsearch   STARTED        2         cs21demo           

elk-indexer         STARTED        17        cs21demo           

elk-kibana          STARTED        16        cs21demo

 

If all services are in STARTED state, you’re ready to proceed to step 1.

 

STEP 1) Run the following command on your management host to enable LDAP user authentication IBM Spectrum Cluster Foundation hosts:
 

# pcmadmin system ldap --enable

 

You must have ready following information to complete the setup:

  • LDAP server hostname or IP address (i.e. ldap://LDAP_server:389)
  • base domain (i.e. dc=example,dc=com)
  • Bind user distinguished name (i.e. uid=pcmuser,ou=user,dc=example,dc=com)
  • Bind user password

Follow the wizard to complete the setup.  See documentation for more information

 

STEP 2)  Login to your infrastructure management console as root user, as shown in Figure 3.

Figure 3 Deployment overview of Integrated Installation without high availability
image

STEP 3)  From Resources > Logistics > Cluster Templates menu, see Figure 4,  un-publish and then re-publish the CS_cluster template.

Figure 4 Deployment overview of Integrated Installation without high availability
image


NOTE:  You don’t have to make any changes to the template. This is necessary in order to add post installation script to setup LDAP on compute hosts. For more information, see IBM Spectrum Cluster Foundation documentation.

 

STEP 4)  Run the following command on your management host in order to  shutdown EGO service:

 

# egosh ego shutdown

 

5.     STEP 5)  Modify /opt/ibm/spectrumcomputing/kernel/conf/ego.conf file, using an editor such as vi, to set parameter EGO_SEC_PLUGIN like this:

 

EGO_SEC_PLUGIN=sec_ego_pam_default


Save your changes and quit the editor.
 

6.     STEP 6)  Run the following command on your management host to start EGO service again:

 

# egosh ego start

 

NOTE: At this point, LDAP user authentication is configured from infrastructure perspective, for both management and compute hosts.  From application perspective, LDAP is configured only on your management host (in steps 4-6).  Following steps will ensure that it is configured for all compute hosts in your Conductor with Spark cluster.

 

STEP 7)  Create an ego.conf.append file on your primary management host in directory /install/osimages/default_image_profile/cfmdir/opt/ibm/spectrumcomputing/kernel/conf/, like this:

 

# mkdir –p /install/osimages/default_image_profile/cfmdir/opt/ibm/spectrumcomputing/kernel/conf/

 

# cat << EOF >

/install/osimages/default_image_profile/cfmdir/opt/ibm/spectrumcomputing/kernel/conf/ego.conf.append

# APPENDED BY XCAT SYNCFILE TO SETUP LDAP

EGO_SEC_PLUGIN=sec_ego_pam_default
EOF

 

NOTE: This will append EGO_SEC_PLUGIN line to bottom of default ego.conf file on all compute hosts during cluster deployment. It will configure EGO service to use Linux PAM for authentication. PAM was already configured for LDAP in step 3. For more information on .append files, refer to xcat synclist file feature.

 

8.     STEP 8)  Deploy your IBM Spectrum Conductor with Spark cluster from your Infrastructure Management Console.

 

9.     STEP 9)  Update the PAM security file to allow all users to be able to login.

a.     On management host, update /etc/security/access.conf and remove line:
 

ALL EXCEPT  root root: ALL

 

b.     On all compute hosts, update /etc/security/access.conf and remove the line:
 

ALL EXCEPT  root root: ALL

 

HINT: Use xcat xdsh utility to remove from all hosts simultaneously.  Even better, write a post-install script to automatically do this after host provisioning.

NOTE:  If you add additional compute hosts to your IBM Spectrum Conductor with Spark cluster, or if you re-provision existing compute hosts, you will have to do this again on those hosts.

 

10.  STEP 10)  From your Application Management Console, create a Spark Instance Group and enable PAM authentication, as per step 7 in documentation.

 

Once your Spark instance group is created and deployed, you are done.  LDAP user authentication is now enabled across both your infrastructure and application layers.

Verification of LDAP without high availability

Here are the recommended steps to verify that LDAP user authentication is configured correctly in your IBM Spectrum Conductor with Spark environment.

            NOTE:  Following authentication examples should also work for your local system users.

1. LDAP users can log in via SSH

From your workstation, run the following command:
 

# ssh -l dev1 managerhost1


where, dev1 is a user define in your LDAP server, and managerhost1 is your IBM Spectrum Conductor with Spark management host.

 

dev1 user should be able to log in without problems via SSH.  If not, check that the PAM security file is configured correctly, as per step 9 above.

2. LDAP users can log in via EGO

Once logged via SSH into managerhost1 as user dev1, run the following commands to verify EGO user authentication.

 

$ source /opt/ibm/spectrumcomputing/profile.platform
$ egosh user logon


Login with dev1 user login details (i.e. username and password)
 

dev1 should be able to login without problems, as in Figure 5 below:

 

Figure 5 LDAP user "dev1" successfully logged in to EGO

image

 

3. LDAP users log in via the Infrastructure Management console

LDAP users should now be able to log into your Infrastructure Management console, just like local users.  Figures 6 shows the login screen and Figure 7 shows that the user dev1 is indeed logged in.

 

Figure 6 LDAP user dev1 logging in to the Infrastructure Management console

image

 

Figure 7 LDAP user dev1 logged into the Infrastructure Management console

image

 

4. LDAP users log in via the Application Management console

Lastly, LDAP users should now be able to log into your Application Management console.  Figures 8 shows the login screen and Figure 9 shows that the user dev1 is indeed logged in.

Figure 8 LDAP user dev1 logging in to the Application Management console

image

 

Figure 9 LDAP user dev1 logged into the Application Management console

image

Enabling LDAP in deployment with high availability

This section describes the process of enabling LDAP user authentication in environment installed with high availability.  Once enabled, both local and LDAP users will be able to authenticate in your infrastructure and application management consoles.

Overview

Figure 10 illustrates a typical network topology for installation with high availability.  For example, you have three management hosts, an LDAP server, and one or more compute hosts, where the compute and management hosts have a direct network access to the LDAP server.

 

Figure 10 Deployment overview of Integrated Installation with high availability

image

Assumptions

Process

The following steps should be done as root user on your IBM Spectrum Conductor with Spark primary management host.  Primary management host is the first host you installed the baremetal package for IBM Spectrum Conductor with Spark.

 

1.     STEP 1)  Run the following command on your primary management host to enable LDAP for IBM Spectrum Cluster Foundation cluster hosts:

 

pcmadmin system ldap --enable

 

STEP 2)  Login to your infrastructure console as root user (i.e. same as step 2 above).

 

STEP 3)  Un-publish and then re-publish all cluster templates (i.e. same as step 3 above).

 

4.     STEP 4)  Run the following command to stop EGO service on all three management hosts:

 

# egosh ego shutdown.

 

STEP 5)  Modify the /opt/ibm/spectrumcomputing/kernel/conf/ego.conf file on primary management host, using an editor such as vi, to set parameter EGO_SEC_PLUGIN (i.e. same as in step 5 above).

 

STEP 6)  Run the following command to start EGO service on all three management hosts:

 

# egosh ego start

 

STEP 7)  Create an ego.conf.append file on your primary management host (i.e. same as in step 7 above).

 

STEP 8)  Deploy your IBM Spectrum Conductor with Spark cluster from your Infrastructure Management Console (i.e. same as in step 8 above).

 

STEP 9)  Update the PAM security file on all your management and compute hosts to allow all users to be able to login (i.e. same as in step 9 above).

 

STEP 10)  From your Application Management Console, create a Spark Instance Group and enable PAM authentication, as per step 7 in documentation (i.e. same as in step 10 above)

 

Verification of LDAP with high availability

To verify LDAP user authentication in environment with high availability, you should follow the steps in the above verification section for environment without high availability.

 

The additional verification you should do is to test LDAP authentication with SSH and EGO services on all three management hosts.  This will ensure that in case of failover, LDAP user authentication will continue to work.

 

Working with LDAP users

Once you’ve LDAP user authentication, you can assign user roles to them, just as you can with local users.  Following describes how to do it for both infrastructure and application consoles.

 

Defining LDAP users as system administrators of the Infrastructure Management Console

In your Infrastructure Management console you can define LDAP users as additional administrators of your infrastructure.

 

User roles are managed via the Systems and Settings >> User Assignment menu, as seen in Figure 11.

 

Figure 11 User management menu item in Infrastructure Management Console

image

 

Figure 12 shows an example of giving user dev1 a role of system administrator.

 

Figure 12 Example of assigning LDAP user dev1 as System Administrator

image

 

Figure 13 shows the confirmation message you should get once you complete assigning administrator roles to your LDAP users.

 

Figure 13 Confirmation message that user dev1 was given system administrator role

image

Assigning user roles in the Application Management Console

Application management console offers much more fine grained user role assignment. 

 

In your application console, user roles are managed via the Systems & Services > Users >> Roles menu, as seen from Figure 14.

 

Figure 14 User management menu item in application management console

image

 

Lastly, Figure 15 shows an example of giving the LDAP user dev1 roles of Cluster Admin and Consumer Admin.

Figure 15 Example of assigning roles to dev1 LDAP user the application management console

image

 

Conclusion

In this article you have learned how to enable LDAP user authentication for integrated infrastructure installation of IBM Spectrum Conductor with Spark v2.1.  You have seen how to enable LDAP for both installations with and without high availability.  You have learned how to verify and troubleshoot LDAP user authentication.  Lastly, you have seen how to use the management consoles to manage user roles.

 

Take the next step

Download an evaluation version of IBM Spectrum Conductor with Spark v2.1.0 (baremetal deployment option) from our Service Management Connect page and try it out!

0 comments
0 views

Permalink