Rational Test

 View Only

Daily build, Daily test, Daily bread

By Suhas Kashyap posted Wed December 08, 2021 06:17 PM



Daily builds need daily testing. In continuous engineering, you develop many releases. You need to test all these releases. Continuous engineering requires continuous testing.

Performance is an important aspect of your application. So, you want to test the performance. Low-intensity performance testing could be useful for you. In low-intensity Performance Tester, you sample only the response times.

If you test daily, you want to see which build shows a different performance. Performance Tester provides a report that provides visual insight.

Performance Tester, what do you mean?
If you say Performance Tester, you can mean different types.

  • Load testing
    In load testing, you want to know what your application does under a specific load. For example, what are the response times when 100 users send requests to your application at the same time?
  • Stress testing
    Stress testing gives you information about what load causes your application to collapse. Except the load, also the consequences of the application failure are interesting. Do you see data corruption? Is your error management in place?
  • Soak or endurance testing
    In soak testing, you check what happens when your application suffers a certain load for a long time. A great way to discover memory leaks.
  • Spike testing
    What happens when many users access your application in a short period. In other words, your application gets a peak load. You want to see whether your application manages to handle dramatic changes in load. That means a sudden increase and then, a sudden decrease of load. For example, your online shop offers an item against a price that nobody can resist. Customers have 3 days to put the item in their shopping baskets. The next 3 days, the number of requests to the application goes up sky high but drops abruptly when the customers receive a We’re-sorry message on day 4.

For an encyclopedic description of Performance Tester, jump to the Wikipedia article of Software performance testing.
Kind of Smoke Testing
However, you just might not have the people and time to do all these tests. So, what problems do you expect and how much time do you want to invest?

A fair tradeoff could be sampling only the response times before every release. You start a single-user test before every new release and watch the response times.

The response times say something about the first impression how an application performs under load. A different response time is likely to give different results in a real Performance Tester. In fact, all performance tests consist of a set of response times. 

Maybe you can compare sampling the response time with smoke testing.

Take an informed decisionThrough daily testing of your build, you can immediately pinpoint the build where performance issues occur or start to occur. With the HTTP trending report of Performance Tester, you can discover deviations from the trend in a glance. See the picture.

Now, you can take the opportunity to fix and to defend your reputation.

​Do you want to go live with your release or do you need to wait for a while? For you to make an informed decision.
Run the Test Daily
For a low number of users, you can use the Rational Performance Tester Starter Edition. For testing the response times of a daily build, you can put just a single user in a test. Also, a schedule of five or less users could be an option.
In short, you can think of the following work.

  1. Run the test or schedule every day or after every new release.
  2. Right-click the test or schedule.
  3. Click View Trending Report … 

The rest is self-documenting.

In the graphs, you see that on one day the response is much slower. Also, the page health shows a drop. In that sense, you are dealing with an outlier. Now, it’s time to figure out what’s wrong with the build. In this example, the development team fixed the issue. The next build shows normal values again. 

​Share Your Experiences
​In continuous engineering, you want to see the performance for every build. If a build shows strange response times, you know that something is wrong in the build. Be fruitful and fix.

​If you’d like to share your experiences with low-intensity performance testing, leave a comment here. Follow me on Twitter @gunangwaney.

Gunang Waney
Product Specialist
Rational Testing
​Follow me on Twitter @gunangwaney

Gunang Waney works as a product specialist of the Rational Testing tools. He is passionate about communicating technical knowledge and making things work for customers. Gunang graduated from Utrecht University, The Netherlands with an MSc degree and works in Amsterdam, The Netherlands. When he can escape from his computer, he spends time with his family.