What Does “Random Number” Mean to You?

The U.S. National Institute of Standards and Technology defines a random number as “A value in a set of numbers that has an equal probability of being selected from the total population of possibilities and, in that sense, is unpredictable.”

In creating tests, there are times you find that you need a random number. DevOps Test Performance (DTP) and DevOps Test UI (DTUI) allows substitution values from its built-in random number data source. You can read more in the 11.0.3 product documentation here.

There is a nuance to this. Let us look at this DTUI test:

Each of the Log Message Details:

The ‘X’ being highlighted in lilac indicates that the string will be substituted from a built-in data source. If you are curious, these are the built-in data source options

Getting back to the example, the test will enter a series of random numbers (between 0 and 100) into the DTUI Unified Report.  I ran the test at 6:05PM.

I ran the test again at 6:06PM.

The sequence of random numbers will always be 40, 87, 2, 24, 90, 66,... Once you know that, it seems less random, doesn’t it?

Here’s the thing, this implementation was developed for a load testing tool. When scaled over hundreds, if not thousands, of virtual users, the static nature of the list of random numbers is less noticeable. Essentially, the distribution of numbers is random within a single execution.

This can be alleviated, to a degree, when using load test agents. This document discusses how to use RPT_VMARGS to change the seed for each agent. The sequence of numbers will still be the same for each agent, but at least it will not be the same across all agents.

That is OK for most purposes but let us suppose you want a different definition of random, one where you will never know the value before running the test.

We’re going to switch up the sample test. First, we’ll modify the log message to

Random number 6 -> X and Y

X remains substituted from the random number data source.

Y will be substituted from a different data source.

1. Highlight the Y in the string.
2. Right-click to select Substitute > Select Data Source.
   
   
   
   
3. Click on the Create new Built-in Datasource icon.
   
   
   
   
4. Select Timestamp. Click Next.
   
   
   
   
5. Truncate on left of the Epoch timestamp for two digits. Click Finish.
   
   
   
   
6. In the Select Data Source dialot, make sure the box next to the new variable is checked.
   Click Select.
   
   
   
   
   
7. The next dialog prompts you to continue through the test looking for other substitution opportunities. For real world tests, choose Yes. It can make life easier. For our purposes here, I am choosing No and updating the rest manually.

Now the numbers that are generated will be based on time. Time is constantly changing.

It would be extremely unlikely to see the same sequence of random numbers repeated using the timestamp data source.

If you want the same sequence of random numbers over and over, use the random number data source. This is a perfectly acceptable solution for load testing, where the important thing is the general load rather than specific content.

If you want the same sequence of random numbers over and over, slightly adjusted, on a larger scale, use performance test agents and RPT_VMARGS in conjunction with the random number data source.

If you want a different set of numbers with each test run for a small-scale test, perhaps a UI Test, use the timestamp data source. The numbers will most likely be different, but they will not be randomly distributed.