Who need to read this blog

This article is NOT ONLY for who is new to IBM DevOps Test (including IBM DevOps Loop and DevOps Test Hub), and want to understand how to start to use these tool, BUT ALSO for who has the entry level of skills with our test tool, but from time to time confused on:

• how to use this feature and that feature
• want to know why there is a feature like this and why I need this
• want to do some "real" work but got some problem
• want to know whether there is some feature I missed.

This article is NOT a marketing article because it's focus on technology side. 

Now let's start the journey here.

1. Overview

1.1 Product Overview

IBM DevOps Test is a comprehensive solution that helps teams automate and manage software testing across the entire DevOps lifecycle. It supports continuous testing, integrates with CI/CD pipelines, and ensures high-quality releases by identifying issues early. With powerful test automation, performance testing, and service virtualization, it speeds up delivery while reducing costs and risks.

There are 2 kinds of ways you can access the DevOps Test features:

1. Test component inside of IBM DevOps Loop 

IBM DevOps Loop is a platform that connects development, testing, and operations in a continuous feedback cycle. It helps teams visualize and optimize the entire DevOps process, enabling faster delivery and better collaboration. By providing real-time insights and automation, it ensures continuous improvement across the software lifecycle. There is a component named "Test" which provides the DevOps Test features.

2. IBM DevOps Test Hub

IBM DevOps Test Hub is a standalone testing product. You can think it's almost equal to the DevOps Test component of IBM DevOps Loop. 

Note:

In the article, what we are dicussing is apt to both these 2 kinds of accesses. The screenshots and demo may generated from one product, but they will work for both products.

1.2 Web client and thick client

Traditionally, IBM DevOps Test provide Eclipse-based IDE(s) as the workbench of test. So you can develop, record, improve test assets, including test data, using the  IDE client.

With the release of IBM DevOps Test Hub, especially IBM DevOps Loop, now we have a browser-based web client, which can enable us to develop, record, improve test assets using our browser. It not only work as a alternative of thick client, but also bring new features not included in the thick client, since it has a central server to provide more features, especially the AI based features.

1.3 New features of web client

Here is a list of some new features provided by web client:

• Web UI test recording using web client
• test data generation/fabrication
• test logic generation with AI
• test database relationship analysis
• etc.

This series of blogs will explore how to use the new features, with each post focusing on a particular area.

1.4 Version Reference

All the features discussed by this series of blogs is based on:

• IBM DevOps Loop v1.0.3
• IBM DevOps Test Hub v11.0.6

Note: The product features mentioned in this article are subject to change due to the rapid evolution of the product

2. Web UI test recording using web client

You can create a test from scratch, or by recording. In this article, we will focus how we can do it by recording to make the test script creation simple.

2.1 Agent for recording

To record and create a test script, we are using different approach from thick client recording:

When you started the recording, DevOps Test will check whether you have installed the agent already. After that, you can click the "Record" to start the recording. 

You can see the details in the demo in the end of this article.

2.2 Branch

You may be confused by the branch in DevOps Test. Here we discuss how we can use the Git branches.

When you open the UI of DevOps Test and open your project, you may see a screen like this:

This is the default branch. You also can create other branches in Git repo, which will be listed in here as well. But these branches, is not the branch you used to work and save your changes.

When you begin to create or modify something, you will be prompt for a branch. You can create a branch by select "Create a new edit branch", or select an existing one. But you need to know the branches here is not the branches defined in Git repo. It's a "edit branch", which is a kind of local branch only existing in DevOps Test side, not in remote repository side.

In this example, you can see "master" and "dev" are remote repository branch. The "work_on_data" is the local edit branch forked from "dev". Switch to an edit branch then you can work on your test asset creation/modification.

When you finished your work on the DevOps Test, you can click the "Publish" button, to submit the change of edit branch into the upstream branch. In my example, I submit the changes from "work_on_data" branch to the "dev" branch. So your work will be visible to others in our team. Then the owner of Git repository can merge the dev branch into master if needed, just as a normal repository procedure.

2.3 Test Data setting step

In beginning of each Web UI test script, there is a step called "Configure settings and data". Variables are generated automatically by DevOps Test. It will make easy for you to covert this test script into a test data driven test script easily. We have anther demo video to cover this content from this series. If you have interest you check the end of this blog for link.

The simple way to use this step, is modify the "value" of each variable with a different value, before run the test. Like the User_ID variable was replaced every time in the demo video, to avoid the same username being registered again.

Another way is to use a Test dataset to use different variables in each run, this is a kind of Test Data Driven testing.

The third way is to create a test suite outside. Then use this test suite to call this test script, and passing the test data from suite level. The data could be from a dataset as well. The benefit of this way is to collaborate multiple testing together. You can use same data to pass to multiple test scripts. 

2.4 Layout of step

In every test step, you can see there are 4 parts in the UI area. The top one is a Smartshot, which is a screenshot with HTML elements together. The left bottom is Details view, which is the setting on what this step do, and which element it's for. The middle bottom view is the Elements view, which list all HTML elements in hierarchy. You can drag and drop the elements from Smartshot view or the Elements view into Details view to define the target element. The right bottom is the properties of the element selected.

2.5 Verification

To add a verification, you just need to select the designed property in Properties view, then click "Create user action with this property"> Verify, a verification step will be added BEFORE the current step.

The trick is actually to decide which property to use as the property to check. There is some tips:

• always use the properties with BOLD font. Only they can be used for verification.
• replace the value with the variable, if they may change from test to test
• always "Create user action with this property", not use "Replace property in step target"
• "Add property to step target" is to used to add multiple check against the same HTML element, in one verification step. For example, you want to make sure the content, type, tagname in the same step.

2.6 Verification dynamically returned value

In some circumstance, we need to verify dynamically generated value, like order ID in this demo. We can't define the verification value in advance. We need to find the first occurrence of this value returned. After we located it, we will fetch the ID and assign the value stored, then verify it in order history later.

We can get the Order ID from the web UI. But The value is a mixture of string, symbol, and number. We can easily extract the number using regex expression.

We can see the expected value "118488" in the "Modified Value", it is what the Regular Expression will return. The "Order #" has been stripped and we got a clean Order ID.

2.7 Test Reports

There are 2 different types of test result, Functional Report and Test Log

Access Functional Report

* Test Editor - test script UI - functional Report

* Analyze -> results

Access Test log

* Test Editor - test script UI - Test log

* Analyze -> Test Log

Comparison of Functional Report and Test log

3. Demo

4. More to come

This blog is focused on the overview of Test in Loop, and how to create web UI testing script by recording. As part of a series blogs, we have more topics to cover.In the following blogs, we will introduce:

• Enhance your test script with Data Driven testing.
• Test management
• Test fabrication
• Test data analysis and visualization 

We will update the new blogs link as below:

• Technical guide: Testing using DevOps Loop (2) -- Test Data management overview and basic data driven testing (under construction)
• Technical guide: Testing using DevOps Loop (3) -- Test Data management (continued)