In part 1 of the “Are you ready for production?” series of blogs, I mentioned that non-functional testing is a pre-requisite for production readiness. In this blog, I will talk about when/where the performance testing of a software solution happens in the software life cycle, in the real world, and the corresponding advantages or disadvantages.

Before discussing the when/where the performance testing happens, I must mention that the most important requirements to understand clearly before you start performance testing are primarily response times and throughputs. These performance requirements may also be communicated differently for different software solutions. For example, for a web solution, a response time requirement may be communicated as web page rendering time requirement, a throughput requirement may be communicated as a number of online users a web site needs to support, etc. As a matter of fact, if these performance requirements are not considered in the design phase, the design can end up in an utter failure.

From my experience, enterprises conduct performance testing using either one of the following two distinct approaches:

High-Risk Performance Testing Approach

Some enterprises wait for the requirements, design, development, functional testing, integration testing, and quality assurance testing phases to complete before starting the performance testing effort. With this approach, there is often a high risk of missing production rollout dates. The following issues are often the reason for missing production rollout dates:

1. Performance related design issues
2. Performance related code defects
3. Performance related integration testing challenges

This approach is typically associated with the Waterfall methodology.

Low-Risk Performance Testing Approach

Some enterprises shift performance testing left as much as possible so that it is interleaved with the requirements, design, development and other testing phases. Because the performance testing phase happens early with this approach, any requirement, design, development issues that are performance-related are addressed long before the full solution is rolled out in production.

Tackling performance related issues early minimizes the risks of missing production dates because:

1. The design in the early phase tends to be simpler than in later phases. Testing the design for performance early helps uncover design issues and provide early feedback into the design. What I am actually talking about here is testing a proof of concept (POC) code that implements the design. If the POC code fails to meet the performance goals during the performance testing, the design may have to change.
2. The code that implements the design is often smaller. So, making code changes to improve performance tends to be less difficult in this case than for larger and more complex code. So, if performance testing uncovers implementation issues, making performance related changes is relatively quick.
3. The design and code usually have less dependencies. For example, the code to be tested may involve just one REST service that makes a database query. In this case, the code has a dependency only on the database. Testing such a code is simpler than testing a code that requires a database, a messaging system, and another third-party service.
4. You learn the performance behavior of discrete pieces of your overall solution much better. For example, you test a REST service that makes a database query. Then, suppose the service has now another dependency, such as a messaging system, in addition to the database. The knowledge you have about the service performance behavior without the messaging system is better than no knowledge. This knowledge helps with performance troubleshooting when the overall solution gets ever more complicated.

In the next blog, I will continue this topic further.