Modelling automation using siteplanner in Cloud Pak for Network Automation

Introduction

For any automation, input data needs to be fed into it which will drive the automation. As automation gets complex, it becomes necessary to represent this data in a proper, user understandable format so that it could be easily modeled, fed into automation and maintained for future updates and configuration changes.

In this blog, we will try to learn how to model all the components related to the automation through siteplanner and pass that data to the backend automation using the example of an Openshift cluster deployment automation. But before we can get to that, we need to be familiar with various types of elements which could be modeled in the siteplanner. So lets learn that first.

Types of model elements

There are various types of elements available within the siteplanner, for example, regions, sites, racks, devices, device types and roles, cables, interfaces, power feeds, clusters, virtual infrastructures, managed entities, services etc.

We can broadly classify these into two types of elements:

1. Default elements
   These are the elements which are available in ready to use form within the site planner. For example a device. It already has predefined fields which we can see if we go to the respective element in UI.
2. Custom elements
   These are elements which could be custom created to have any number and type of desired properties as per requirement through code. Eg: managed entities etc.

Modelling default elements and integrating them with automation

As explained earlier, default elements come with predefined set of parameters fields. We can create an instance of them by just filling in the values for those fields. For example, in the siteplanner user interface (UI) we can go to Devices > Device. Click on Add button and in the page which opens up afterwards, fill in the values for parameters to define the device.

Once we have created any default element, the next step is to pass the information available in the default model element to the backend automation code. For this, we would have to make use of “automation context”.

An automation context defines the plumbing that has to exist between the model and automation. It has three main parts to it:

1. Automation Context: Defines the CP4NA Automation assembly to associate the query and element with. This is the backend automation code which will get executed when you try to build the element.
2. Assignment: Defines the type of default element with which the automation should be associated with. For example, this could be a device or a cluster etc. Additionally, types and roles could be specified to confine within that element to a specific type or role.
   For our example, lets consider device automation. Suppose we want to have server automation to set BIOS, firmware etc. In this case, we may want automation to run for a specific role of devices like “OCP worker node”. So while creating automation context, we can provide ObjectType as Device and Role as “OCP worker node” (Note, for this, the “OCP worker node” role should have been created prior to this as part of modelling) in assigment. So the automation will only be associated with devices which have been added with the role of “OCP worker node”.
3. Data: These are graphql queries which could be used to query the data that needs to be passed to the automation assembly. A graphql editor is available at <siteplanner_url>/ graphql-engine/graphql which could be used to test the query and see what data it returns. Once query looks fine, same could be copied to the “Data Expressions” field in the automation context page.
   
   
   
   

Modelling custom elements and integrating them with automation

A network automation will invariably have various types of elements which would require customization. Due to various reasons like properties changing across vendors, continuously evolving standards, usecases etc customization would be required in terms of components that constitute elements or their properties. CP4NA provides a way to create custom elements programmatically through managed entities. This provides a powerful way to extend the modelling capability to pretty much any requirement.

Managed Entities

We don’t need managed entities to model the OCP cluster. But since its an important element type, lets try to understand how to create them using the example of creating the model for an eNodeB and go through the steps one by one.

To create managed entities, one has to follow the below process:

1. Define all the properties of the entity in the CP4NA assembly descriptor.
   For eNodeB example we have considered, we may want to add couple of generic properties, say, sitename, latitude, longitude, enodeb id etc. In the assembly descriptor, all the generic properties could be specified under “properties” section of the descriptor.
   
   

   name: assembly::eNB:1.0

   description: eNodeB descriptor

   properties:

       sitename:

           type: string

       latitude:

           type: string

       longitude:

           type: string

   …

Along with these, an eNodeB is also going to consist of CU, DU and cells. So we may want to put these in different sections so that users can add one or more instances of these and fill in their respective properties. In order to do this, we can define them as of type “map” and provide “entry-schema” to define the properties under each of them.

2. Push the assembly to CP4NA through lmctl tool
   More information about this is available at https://github.com/IBM/lmctl and https://github.com/IBM/lmctl/blob/master/docs/command-reference/pkg/push.md
3. Create Managed Entity Type by linking it to the assembly
   This can be done by going to Virtualization > Managed Entity Types in siteplanner user interface. Clicking on Add and selecting the assembly descriptor we had created earlier for this entity. We can also provide any tags if required.
4. Create a Managed Entity of type created in previous step
   Once a managed entity type is created, we can go ahead and create as many instances of it as we want. This can be done by going to Virtualization > Managed Entities. Clicking on Add button, selecting the managed entity type we had created earlier and clicking on Create Filling in the properties for the managed entity instance. Once Create button is pressed, we will get a form with all the properties we had defined in the assembly earlier.
   So there is no need of creating any UI code. It is rendered based on the properties defined in the descriptor.
   Note that they wont have any maps defined in the descriptor. They can be added after managed entity is created.
    
   Once you create the managed entity, we should be able to see the build button getting enabled as we had linked the managed entity type to the assembly descriptor in step 3. So after filling in the values, if we click on this build button, it would invoke the assembly descriptor we had specified in the earlier step.

Understanding modelling with an example

So far, we got introduced to different types of elements available within siteplanner and how to create them. Lets try to explore the process of modelling by taking the example of Openshift cluster automation.

An OCP cluster automation would require information about all the following components:

• Servers which would constitute the master and worker nodes of the cluster
  • Their IPMI address, interface details, node labels, any performance or network profiles which need to be configured on them etc
• OCP cluster properties
  • OCP version
  • Image repository properties
  • API and ingress VIP, cluster port, domain etc
• DHCP-DNS properties
  • Details of the DHCP server
  • Details of the DNS server
  • IP addresses of worker nodes to be added to DHCP-DNS servers (this could be properties within respective worker nodes but which need to be passed to this during automation)
• Provisioning server (server from where OCP assisted installer runs and executes OCP automation related code)
  • Endpoint details of provisioning server
  • Details of Assisted Installer

It may appear simple enough to be maintained in few configuration files. But think about the complexity in case of a telecom site rollout for example, where we will have hundreds or thousands of sites having different worker nodes on which various elements like CUs, DUs are going to be deployed, each with their own properties and each of those then need to be mapped to RUs, cells, OCP cluster etc. As we can clearly see, as complexity increases, there comes the need for a modeller in which we can easily define these entities, establish relationship between them, pass them on to automation and maintain it for future changes or updates. Siteplanner provides this functionality. 

Modelling steps

To model an automation, we can go through below steps :

1. Think about the automation for which model is needed and identify the components or elements which constitute the solution.
2. Create a simple component diagram which would make it easy to understand different elements which constitute the automation and the relationship between them (This is an optional step. Though modelling could be done in site planner even without this, creating component diagram would help in visualizing the model and later in identifying the elements within siteplanner to which different components could be mapped to.)
3. Identify to what element in site planner each element in the component diagram could be mapped to
4. Model each of those elements in site planner by using GUI or APIs.

Lets go through this process for OCP cluster automation modelling.

1. Identify the components or elements which constitute the solution
   We have already done this exercise in the earlier section and we have identified the components, namely:
   1. Master nodes
   2. Worker nodes
   3. OCP cluster (logical entity which is going to represent the cluster & its properties)
   4. DHCP, DNS servers
   5. Provisioning server
2. Create component diagram
   A simple component diagram could be created using any diagram editor tools like Mermaid. The diagram created for OCP cluster automation could be seen below:

3. Identify what element within site planner we can use to represent the components in the component diagram:

4. Model each of those elements in site planner.
   There are three ways through which data could be populated in the site planner

• • Using graphical user interface
    For the ease of use, in this article, all the steps will be explained through actions in UI.
  • Using APIs
    

    All that can be added in the UI can also be added through APIs. You can click on the API link available at the bottom right hand corner of site planner UI to go to the swagger documentation for all the APIs available.
    For the device example we have taken here, we can populate the device details by making the post call to /dcim/devices.

  • By importing the data using import button
    Most of the element category pages have “import” button which can be used to import data in bulk.

Modelling regions and sites

Each device will be physically located in some telecom cell site and that needs to be represented in the model. For this, regions (Organization > Regions) and sites (Orgarnization > Sites) could be used. Note that regions could be hierarchical where a region created earlier can be represented as the parent region of another.

Modelling Master & worker nodes

Master and worker nodes could be modeled as Devices (Devices > Devices option in UI). But before we can do that, its necessary to create:

• Device Type for the type of nodes we are using - Dell R740,
• Device Role – ocp_controlplane_master_node, ocp_worker_node.
• Manufacturer – Dell

All of the above could be created through the UI.

Once they have been added, we can create the device by going to Devices > Devices page, and clicking on Add button to add a new device. We can define the server properties in the page that opens next.

Notice the Site parameter which exists within Device. Using this, device can be mapped to the site created in the earlier step.

Once we have created a device, further components could be added to it by clicking on “Add Components” button, for example, interfaces. Please also note the “config context” tab available within the device page.



This can be used to extend and encompass everything for which inbuilt fields are not available. For example, we can use it to define the BIOS properties, SRIOV related network configuration profiles etc.

In similar fashion, other devices like DHCP-DNS server and provisioning server could also be modeled.

Modelling cluster 

Cluster

We need a representation for cluster to group together master and worker node devices created in previous step. This can be done using “Clusters” element (Virtualization > Clusters). We can go to Virtualization > Clusters. Click on Add button to add a new cluster. We can specify the name, type (to be able to specify here, we should have created a cluster type using Virtualization > Cluster Type element), group (to be able to specify here, we should have created a cluster group using Virtualization > Cluster group element), Site etc. Once properties are filled in, we can press “Create” button to create it.

Once the cluster element has been created, nodes which should be part of it could be added using “Add devices” button.

OCP cluster

Virtual Infrastructure can be used to model the OCP cluster.

Virtual infrastructure can be used for representing any logical layer or entity which should exist above a cluster. For example, when we define a cluster through Virtualization > Clusters option, it has only the option to add related devices to it. Apart from this, we may want to add some additional properties related to the cluster. For example, in case of an OCP cluster, there may be properties like ingress vip, cluster port etc. To include all these properties, virtual infrastructure could be used.

We can create Virtual Infrastructure element by following below steps:

1. Define Virtual Infrastructure Type
   This can be done by going to Virtualization > Virtual Infrastructure Types. Clicking on Add button, providing a name, say OCP cluster (and comment and tag if required), clicking on Create.
2. Create Virtual Infrastructure element of the created type
   This can be done by going to Virtualization > Virtual Infrastructures. Clicking on Add button, providing
   1. Name - say, controlplane_cluster
   2. Type – Choose the Virtual Infrastructure Type created earlier
   3. Clusters – Point to the cluster created using the default Virtualization > Clusters
   4. Configuration – we can define a map of properties here to pass on the properties related to OCP. For example,
      

      {

          "api_vip": "172.29.231.160",

          "ingress_vip": "172.29.231.161",

          "bridge_name": "ocp-br",

          "cluster_port": 6443,

          "cluster_version": 4.6,

          "cluster_domain": "ibm.com",

          "airgap_environment": true
      }

Modelling DHCP, DNS servers

Like how we had created the master and worker nodes earlier, we can use Devices to model DHCP, DNS servers as well. We can use “config context” within device to represent any information which could not be accommodated within the fields available by default.

Creating automation context to link the model with automation

We have already seen the steps to define the automation context. We can see below the automation context defined by mapping the virtual infrastructure of type “ocp_controlplane_cluster” with the assembly “assembly::cluster-automation::1.0”,

The “data expressions” query used is given below for reference,

  controlplane: virtual_infrastructure(filters: {id: $id}) {

    configuration

    tags

    cluster: cluster {

      name

      tags

      master_nodes: devices(filters: {device_role__name: "ocp_controlplane_master_node"}) {

        __key: id

        name

        tags

        config_context

        device_type {

          manufacturer {

            name

          }

          model

        }

        interfaces(filters: {name: "nic1"}) {

          name

          mac_address

          ip_addresses {

            address

          }

        }

        services(filters: {name: "ipmi_service"}) {

          ipaddresses {

            address

          }

          port

        }

      }

      worker_nodes: devices(filters: {device_role__name: "ocp_worker_node"}) {

        __key: id

        name

        tags

        config_context

        device_type {

          manufacturer {

            name

          }

          model

        }

        interfaces(filters: {name: "nic1"}) {

          name

          mac_address

          ip_addresses {

            address

          }

        }

        site {

          name

          region {

            name

          }

        }

        platform {

          name

        }

        services(filters: {name: "ipmi_service"}) {

          ipaddresses {

            address

          }

          port

        }

      }

    }

  }

  ddi_server: device(filters: {device_role__name: "ddi_server_role"}) {

    config_context

  }

provisioner: device(filters: {device_role__name: "provisioning_server_role"}) {

    config_context

}

}

As mentioned earlier, we can make use of graphql editor available at <siteplanner_url>/ graphql-engine/graphql to test the query and see what data it returns before adding it as part of the automation context.

Now, we have completed all the steps of modelling the OCP cluster. CP4NA automation assembly code (cluster-automation assembly referred in the automation context) could be written for the backend automation to make use of the data we are passing through automation context. 

Summary

We saw various element types that siteplanner component within CP4NA provides to model the components of network automation. We took the example of OCP cluster and went through the process of creating a model for its automation and linking it with the backend assembly descriptor with the help automation context. We also saw how to create custom elements with the help of managed entities. We hope this would have provided a basic overview of the siteplanner capability and how one can go about modelling automation using it.