AIOps

Author(s): Divya Kamath, Vijaya Bhaskar R Siddareddi

 

Overview

IBM Infrastructure Automation is an offering available as a part of Cloud Pak for Watson AIOps (Ref-https://www.ibm.com/docs/en/cloud-paks/cloud-pak-watson-aiops/3.5.0?topic=infrastructure-automation). It brings together Terraform based Managed Services and Infrastructure Management capabilities for Architects and IT Operations teams.

With Managed Services, IT team can build self-services which can be used by various personas – like Developers and Testers – in the organization to provision infrastructure on their own, without the intervention of IT operators. With Infrastructure Management, day-2 operations on infrastructure spread across various environments – cloud / on-prem / hybrid – can be managed through a single pane of glass.

Enterprises typically have the need of positioning solution stacks in hybrid platforms including multiple public clouds and different on-prem data centres for various reasons like cost, proximity to customers, resource availability and security. When the infrastructure and application stack to be provisioned is complex or spread across multiple platforms, provisioning and managing those manually becomes challenging. Different skillsets would be required as the platforms to provision and manage infrastructure are different. Adhering to enterprise security policies and managing compliance across all platforms would become a tedious task.

IBM’s Infrastructure Automation provides a centralized hub using which resources pertaining to all hybrid platforms (like VPC, subnets, virtual machines, and load balancers) can be provisioned, followed by application stack deployment (like application server, database, application packages and containers) and day 2 operations (like start/stop/reset/reconfigure/delete virtual machines, and install security patches ) can be managed for the same.

 

Infrastructure as Code (IaC) using IBM Infrastructure Automation

IaC is all about provisioning and managing IT using code rather than through manual processes. Terraform is an open-source software tool which provides IaC capability.

IBM Infrastructure Automation has open-source Terraform embedded in it to provide powerful IaC capability along with service composer. Service composer is a simple-to-use graphical interface to build self-services, in which terraform templates can be orchestrated to create workflows along with Ansible playbooks for configuration management. Like terraform templates and Ansible playbooks, service blueprints provide IaC capability and can be stored in source control systems like GitHub and GitLab.

In this article we will explain about how to build a complete solution stack in a target environment using IBM Infrastructure Automation through following steps:

1. Pick-up an existing terraform script and convert that into a terraform template in IBM Infrastructure Automation
2. Create an Ansible (AWX: open source Ansible Tower) connection
3. Create an AWS Cloud Connection
4. Compose a Service using the (above) terraform template along with available Ansible playbooks from AWX.
5. Publish service to catalog
6. Deploy solution stack through self-service
7. Verify the solution stack deployed

The overall Architecture of IBM Infrastructure Automation setup on IBM Cloud and the Solution Stack provisioned on AWS using that setup are represented in the diagram below:

Step 1: Pick up an existing terraform script and convert that into a terraform template

In this step, terraform script is imported into terraform template as needed by Managed Services. This terraform template would be used in building self-services.

1.1 Here is a terraform script which provisions an AWS EC2 VM:
https://github.com/diimallya/demo-terraform-templates/tree/master/create-ec2-micro. This can be imported in Infrastructure Automation by navigating to menu (from IBM Automation Cloud Pak home page) Automate Infrastructure -> Library -> Terraform templates (tab) -> Import template.   
          

1.2 Fill in the required fields in dialog presented - GitHub repository URL, GitHub repository download directory and then click on Import.       

     
1.3 Provide a name, short description and long description. On the right side pane, under Cloud providers section select Amazon EC2 and deselect Other.

 

1.4 In Manage template tab, click on Update parameters and select Import type as From template source. Click on Update and then Save.

 

Once saved, the imported template is shown under My templates section.

Step 2: Create an Ansible connection

In this step we configure Ansible Tower integration in Managed Services. This is leveraged for executing Ansible playbooks as part of service workflow.

Note: This step assumes AWX is installed on the same OpenShift cluster. Ref: https://github.com/ansible/awx/blob/devel/INSTALL.md

2.1 From the IBM Automation Cloud Pak menu, click on Automate Infrastructure -> Manage -> Ansible Automation (tab) -> Click on Create connection

2.2 Enter AWX credentials – Ansible connection name, User, Password and URL and click on Create.

 

Step 3: Create an AWS Cloud connection
In this step we configure an AWS account in Managed Services. This is the target environment to provision solution stack. 

3.1 From the IBM Automation Cloud Pak menu, click on Automate Infrastructure -> Manage -> Cloud connections (tab). Click on Create connection.

3.2 Enter a name for connection and AWS account details like Access Key ID, Secret Access Key and then click on Create.

 

Step 4: Compose a Service including terraform and Ansible blocks

This is the major step of building a service using terraform templates and Ansible components. Typically operations team persona in the organization builds this service in service composer to further make it available for the end users to deploy.

Note: Before proceeding with this step, following pre-requisites need to be performed.

Pre-requisites:

1. Ansible integration needs to enabled for CP4WAIOps Infrastructure Automation - in the OpenShift console as mentioned in the documentation, if not done already. Ref: https://www.ibm.com/docs/en/cloud-paks/cloud-pak-watson-aiops/3.4.2?topic=integrations-ansible-tower-integration
2. In Ansible Tower, create install-tomcat job template, with a setting of Inventory and Credentials fields marked as Prompt on launch. Refer the screenshot below. The following GitHub repository may be used to setup install-tomcat job template: https://github.com/divyakamathorg/tomcat-ansible 

4.1 From IBM Automation Cloud Pak menu, navigate to Automation Infrastructure -> Library -> Services (tab) -> Create Service

 

4.2 Give a name to the service, version, category and click on Create.

4.3 In the first tab, short description, long description, feature list details of the service can be documented. This is an optional step.

4.4 In the Composition tab, on the left had side one can find all the terraform templates available along with Rest Hook, Decision etc elements and also Ansible components. Drag and drop create-ec2-micro from Templates->Amazon EC2 section and on right side pane, under Basic information tab, provide version of the template to be used in the service and target AWS connection name where VM needs to be provisioned.

 

4.5 On right side pane, second tab i.e. Parameters, various variables with default values initialized in terraform templates are shown. If any of these values need to be taken from users, that can be done by clicking on Link parameter option below the variable value.

 

4.6 Click on Create tab on top right to create a new input parameter which would be linked to that terraform variable during template execution. Datatype for this input parameter, a name and default value which would be populated in user dialog can be chosen. Once all values are provided, click on Save.

 
4.7 Below is the screenshot after 2 parameters (aws_instance_name and aws_key_pair_name) are created and assigned to respective template variables.

  
Note: We can see these parameters and modify, if required later in third tab of the service creation UI i.e. Parameters.

4.8 In the Composition tab, we need to further drag and drop Ansible components – Credential, Inventory and install-tomcat job template. This is to create a Credential in AWX for accessing provisioned VM, and to create an Inventory in AWX which contains the IP address / hostname of the newly provisioned VM, followed by execution of install-tomcat Ansible job template by giving these credential and inventory elements created as input.

 

4.9 Now click on Credential element in the middle pane, and click on Parameters tab of Credential element, on right side pane. Provide names for credential_name_suffix and user_name.

 

4.10 Next click on ssh_private_key The terraform template (create-ec2-micro) has one of the output variables as vm_ssh_private_key. We need to link this output variable value of the template as input to create credentials in AWX. Click on Link parameter below ssh_private_key.

   
4.11 In the dialog that appears, click on Templates section in left side pane, select output variable vm_ssh_private_key (output) of the template, and click on Save.

4.12 After creating the link, ssh_private_key variable value would appear as shown in screenshot below.

4.13 Similarly, Inventory element needs to be provided all the required values. Click on Inventory in middle pane and then in right side pane, provide an inventory name under Inventory tab.

 

4.14 Under Hosts tab, click on Link parameter.



4.15 From Templates section, select output variable of template – vm_public_ip (output) and click on Save.

4.16 After creating the link, Host value would appear as shown in screenshot below.

 

4.17 Next, we need to provide these Credential and Inventory elements as input to install-tomcat Ansible job template. Click on install-tomcat and select Parameters tab on right side pane.

 

4.18 The field inventory_name needs to be linked with Ansible inventory element output inventory_name (output)

 

4.19 The credential_name has to be linked with credential_name (output).

4.20 This completes the service building phase. Click on Save button on top right.

 

Step 5: Publish service to catalog

Once the service is saved, it can be published to the catalog and assigned access to required namespaces. Only published services are available in service catalog for the end users to consume.

5.1 Click on Publish.

 

5.2 Click on Service Library on the top left. Against the service Provision EC2 with Tomcat, click on 3 vertical dots, and select Assign access. Select required namespaces (Ex: default, services) and click on Assign.

Step 6: Deploy Solution Stack through Self-Service

This step explains how an end user can provision the solution stack by leveraging the service published by operations team persona earlier.

6.1 To deploy a service available in catalog, an end-user persona can click on Automate Infrastructure -> Service Catalog menu option from IBM Cloud Pak home page.

6.2 Click on the tile corresponding to published service – Provision EC2 with Tomcat setup

 

6.3 Select plan and Click Next.

6.4 Provide inputs for deploying service and click Deploy.

 

6.5 Service instance deployment starts and the progress of the same can be viewed by clicking on Go to Instances which navigates to Instances page.

 

Step 7: Verify the solution stack deployed

This step details on how to verify provisioned solution stack. When a service is deployed, service instance is created and all provisioned resources are associated with it.

7.1 When the service instance deployment is complete, the state of the instance changes from In progress to Active. Click on instance name demo1 to verify the solution stack deployed.


7.2 Each activity in service orchestration workflow has an instance created for the same during service deployment. We can click on those to verify the resources provisioned as part of that activity.

 

7.3 On clicking the first activity instance i.e. terraform template instance – demo1-00-inst, we can see the input and output parameters of the template execution. Public IP and state of the VM can also be viewed.

 

7.4 Click on second tab of service instance demo1, to view the logs of install-tomcat job template execution from AWX.

 

7.5 On browsing the URL http://<vm_public_ip>:8080, tomcat installation on the VM can be verified.

 

Note: You may have to allow traffic to port 8080 in the security group associated with your AWS EC2 VM if it is not in place already.

Summary

In this article, we have explained how Terraform templates and Ansible playbooks can be orchestrated in a workflow to build an entire solution stack by leveraging IBM Infrastructure Automation. 

IT Operations team can easily build such services and publish in a self-service catalog for the service consumer personas like developers and testers in the organization. Whenever a solution stack is required on-prem or on any public clouds, it can be provisioned through these self-services. When they are no longer required, service instances can be terminated to destroy all the resources which are provisioned as a part of that service instance.

In this way IBM Infrastructure Automation can be leveraged to build, deploy and manage self-services in Enterprises.