Turbonomic

Turbonomic

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only

Webhooks: Improvements, now with a UI

By Anshuman Biswas posted Wed March 06, 2024 10:50 PM

  

Introduction

Webhooks are automated messages sent by applications when something happens. They have a message — or payload — and are sent to a unique URL— essentially the applications's phone number or address. Webhooks are almost always faster than polling, and require less work at the customer's end. They are much akin to an SMS notifications. 
 
The Webhook integration capabilities of Turbonomic, focuses on day one automation scenarios and integrating with more complex workflows that customers require during action execution. 
Turbonomic introduced Webhooks some time ago to empower customers beyond the standard action execution experience. These were designed to replace the more legacy-oriented ActionScripts, which relied on SSH for launching scripts to execute custom actions. The initial version of the Webhooks feature equips customers with the ability to:
 
- Dispatch notifications to collaboration platforms like Slack
- Seamlessly integrate Turbonomic with workflow management systems
- Apply custom logic to override Turbonomic actions

Implementation

This iteration of Webhooks built on the already existing API implementation by providing customers with a UI interface to create, edit and view the already created webhooks. The Webhook falls under the Workflows umbrella, that contains other integrations such as ServiceNow, ActionsScripts and ActionStream Kafka! The feature also made several tweaks to the API interface which show up in the UI. Starting with the Workflows setting, that may be accessible by navigating to the *Settings* options on the left panel and then selecting the *Workflows* icon:
Workflow Settings
The user interface presents a dedicated dashboard tailored exclusively for Workflows, with Webhooks serving as the sole type of Workflow featured. Presently, other Workflow variants are located within the Targets page but will progressively migrate to this dashboard interface. Each Webhook's intricate specifications are deliberately obscured from user view, displaying solely the identifier (referred to as the Name to keep it generic), Workflow classification (currently superfluous), and its endpoint address. The expectation is that the combination of the identifier and endpoint address will sufficiently aid users in discerning their desired Webhook. However, prior to engaging with these details, users must first initiate the creation of a Webhook via the "Create Workflow +" button: 
Workflow Dashboard

The user is guided through a sequential progression of pages, each dedicated to a specific aspect of the Webhook's configuration necessary for establishing the eventual connection. Initially, the user is directed to a page where the default option is the creation of a Webhook. On the left-hand side of the screen, a navigation panel delineates the current step in the process and outlines the subsequent steps required to complete the Webhook setup.

Create Workflow
Proceeding to the subsequent page prompts the user to input basic details for the Webhook being defined. These details encompass the Webhook's name, a descriptive note to elucidate its intended function, the URL address to dispatch the Webhook to, and the designated HTTP Method, with "POST" as the default selection (although alternative methods like GET, PUT, PATCH, and DELETE are also accommodated). Additionally, a toggle feature is provided to enable validation of the Webhook server's SSL certificates, along with an option to specify proxy information. It is important to note that each Webhook can be configured with its distinct proxy settings.
Create Workflow Page 1
Moving forward, the next stage revolves around configuring user authentication for the Webhook. This encompasses the integration of two authentication mechanisms. Firstly, a basic authentication method is available, safeguarding the Webhook through the utilization of a username-password pair. It's worth noting that all passwords stored in Turbonomic employ a one-way hash function, ensuring their confidentiality. Once stored, the password contents remain inaccessible to both the user and Turbonomic.
Create Workflow Page 2
Another supported authentication mechanism in this implementation is OAuth. Specifically, we've implemented the Client Credentials grant type. This requires inputting a Client ID, a Client Secret, and an Authorization Server URL, as depicted.
Create Workflow Page 2, oAuthThe penultimate segment addresses headers. Typically, headers consist of a straightforward key-value pair. However, following feedback from certain customers, we've opted to enable users to define headers wherein the value remains hidden. In essence, the value of the specified header is encrypted upon storage in Turbonomic. Consequently, users are unable to directly access the header value when editing a saved header. This feature aims to safeguard users who wish to include sensitive tokens in headers for authentication purposes. It's important to note that although the header values are encrypted within Turbonomic, standard HTTP protocol behavior dictates that these values are not encrypted when the action Webhook HTTP request is sent from Turbonomic to the customer's webhook server.
Create Workflow Page 3
The concluding phase is rather straightforward, yet it offers useful guidance for users to ensure the proper setup of the Webhook. This page exclusively presents the Webhook Payload and its associated Media Type options, which include "JSON", "XML", and "Text". The significance of these options becomes clearer in the context of Webhook validation. While the payload can comprise any text, the strength of this implementation lies in its capability to utilize variables from the action API DTO corresponding to the action that initiates the Webhook. The specific variables that can be employed are outlined in the Turbonomic documentation. Here's an illustrative example:
{
    "blocks": [
        {
            "type": "header",
            "text": {
                "type": "plain_text",
                "text": ":scales: $action.target.className $action.actionType Action :scales:",
                "emoji": true
            }
        },
        {
            "type": "section",
            "text": {
                "type": "mrkdwn",
                "text": ">- *Status* `$action.actionState` via `$!action.actionMode EXECUTION` by `$!action.userName` at `$!action.updateTime`\n>- *Instance*: `$action.target.displayName`\n>- *Details*: `$action.details`\n>- *Reason*: `$action.risk.subCategory`\n>- *Risk*: #foreach($resizeRisk in $action.risk.reasonCommodities) `$resizeRisk` #end"
            }
        },
        {
            "type": "actions",
            "elements": [
                {
                    "type": "button",
                    "text": {
                        "type": "plain_text",
                        "text": ":turbonomic: See Action in Turbo :turbonomic:",
                        "emoji": true
                    },
                    "value": "Click Here",
                    "url": "https://anshuman-dev-3-leader.fyre.ibm.com/app/#/view/main/action/$action.uuid"
                }
            ]
        }
    ]
}
At last, the page offers an option to validate the template payload and test the Webhook. The template itself necessitates the substitution of variables such as $action.actionState with the corresponding state of the action. Our templating engine of choice is Velocity, an open-source project under Apache. This robust templating language facilitates the inclusion of if statements and loops within the payload template, ensuring proper parsing and rendering before dispatching the Webhook. Nevertheless, users may wish to verify the template's accuracy prior to initiating the Webhook request. To accomplish this, upon identifying an action UUID, users can validate the template using the provided payload and receive the finalized payload intended for the Webhook.
 
Additionally, users may opt to test the Webhook with all associated headers and details even before its creation. This option facilitates the immediate testing of the Webhook functionality. Upon clicking the designated button, a Webhook Request incorporating the defined details will be dispatched. Any occurring errors will be promptly displayed on the screen. Once the user is content with the Webhook definition, they are encouraged to proceed by clicking the "Create" button to finalize the Webhook creation process.
Create Workflow Page 4

Validation

Once a Webhook has been created, users have the option to either modify its configuration or access details for quick reference. The editing process mirrors the creation process, with the exception of the initial selection page for a Webhook.
 
The "View" option presents several tabs. Typically, when no errors are present, the default tab showcases essential details such as the Webhook description. Additional tabs, including "Headers," "Body," and "Policies," display the headers defined for the Webhook, the payload intended for the Webhook, and the associated policies respectively. It's noteworthy that the Webhook is only triggered when linked with a policy that activates either upon action generation or action execution.
Dashboard View Details
View Details 1
View Details 2
However, validation is initiated either upon saving a Webhook or when a Policy is saved with the Webhook associated. For straightforward errors like JSON validation issues and invalid Action API DTO fields, validation can be conducted during the Webhook saving process since no additional dependencies are necessary. However, this system is designed to identify more nuanced errors, such as those arising from changes in the action type. For instance, if a Webhook has a valid action property tailored for specific action types, but the policy is defined for alternative action types, errors may occur when dispatching the Webhook. To proactively prevent such errors, validation is triggered upon saving a policy.
Validation Error 1
Validation Error 2

Policy Improvements

The efforts undertaken in preceding sections facilitated seamless Webhook creation and validation. Leveraging UX tools developed in React, users could not only manage the lifecycle of Webhooks but also efficiently test and edit substantial payloads—a task previously challenging via CLI or tools like POSTMAN. However, the functionality of Webhooks as a Workflow remained static until advancements were made with policies to offer users more options.
 
Previously, users were limited to selecting only one workflow per action stage, which included ON GENERATION, BEFORE EXECUTION, EXECUTION, and AFTER EXECUTION. Furthermore, certain stages mandated action failure upon Webhook failure, while others permitted action execution to proceed despite failures. These inconsistencies resulted in confusion, prompting the team to enhance the functionality of policy automation.
Policy Page Details
The initiative began with rebranding the section as "Automation Workflow" to better align with its provided functionalities. The action stages were subsequently categorized into the aforementioned main sections. Within each action stage, users gained the flexibility to select multiple workflows, designate workflows as either critical or non-critical, with critical workflows leading to action failure upon failure, while non-critical workflows have no impact on action failure. However, a constraint exists where the selection of a Critical On-generation workflow mandates the use of ServiceNow, allowing only one critical workflow (ServiceNow) to be chosen.
 
Furthermore, workflows associated with ServiceNow audit are only selectable as non-critical On Generation or non-critical After Execution. Another enhancement involved refining the summary page for Automation Workflow, which now includes these new sub-stages and provides a tally of the number of workflows—a feature not previously necessary due to the one workflow per action stage constraint.
Policy Page Summary
Furthermore, workflows associated with ServiceNow audit are only selectable as non-critical On Generation or non-critical After Execution. Another enhancement involved refining the summary page for Automation Workflow, which now includes these new sub-stages and provides a tally of the number of workflows—a feature not previously necessary due to the one workflow per action stage constraint.

Use Case

For many Turbonomic customers, the reality entails the necessity of implementing intricate workflows when executing actions. For instance, various teams often have specific requirements to execute an action:
 
  1. The Ops team requires advance notification of the impending change.
  2. The Change Management System necessitates updating.
  3. Approval from the App Owner is essential for changes to the application.
  4. Disruptive changes must occur within a designated maintenance window.
  5. The Ops team needs to be informed prior to the change taking place.
  6. Handling application restarts is crucial when implementing disruptive changes to the underlying layer hosting the application.
  7. Implementation of changes to modify the underlying layer is necessary.
  8. The Ops team needs confirmation once the change is completed.
Workflow Management
The significant advantages of this feature include the capability to define and validate webhooks within the UI, conduct pre-creation testing, and validate for errors when attaching to a policy. Furthermore, users now have the flexibility to associate multiple policies per action stage, with the ability to designate certain workflows as non-critical (e.g., failure to post to Slack), thereby enabling the execution of complex use cases.
1 comment
61 views

Permalink

https://community.ibm.com/community/user/blogs/anshuman-biswas/2024/01/09/webhooks-improvements-now-with-a-ui

Comments

Patrycja Hubl- Lis
Mon March 11, 2024 06:31 AM

Great! Thank you for sharing this blog!