Gaurav Dangaich
In the fast-paced world of technology, ensuring that systems perform optimally under varying loads is a critical challenge. As a Performance Engineer, our ability to predict and manage system performance can make the difference between seamless user experiences and frustrating downtimes.
This blog will delve into two foundational concepts that are instrumental in tackling this challenge: Workload Modelling and Little's Law. By understanding and applying these principles, we can gain valuable insights into system behaviour, identify potential bottlenecks, and optimize resources effectively. Let's explore how these techniques can transform the approach to performance engineering.
What is Workload Modelling?
Definition and Importance
Workload Modelling is the process of representing the load or demand placed on a system in a way that allows for analysis and prediction of system performance. This involves identifying and simulating the different types of activities or transactions a system will handle, the frequency of these activities, and their resource consumption.
Effective workload Modelling is essential because it helps engineers:
- Predict system behaviour under different load conditions.
- Identify potential bottlenecks and performance issues before they occur.
- Optimize resource allocation and ensure efficient system operation.
- Plan for scalability and capacity requirements.
Steps in Workload Modelling
- Workload Characterisation: Identify the different types of workloads the system will handle. This could include user transactions, batch jobs, background processes, etc.
- Measurement and Data Collection: Gather data on the frequency, volume, and resource consumption of these workloads. Tools like performance monitors, log analyzers, and profiling tools can be used.
- Workload Classification: Group similar workloads together based on their characteristics and resource usage.
- Model Creation: Develop mathematical or simulation models that represent these workloads and their interactions with the system.
- Validation: Compare the model's predictions with actual system performance to ensure accuracy.
Types of Workload Models
- Synthetic Workloads: Created artificially to simulate expected usage patterns.
- Trace-driven Workloads: Based on actual usage data collected from the system.
- Analytical Workloads: Use mathematical formulas to represent workloads.
Little's Law: A Key Principle in Queueing Theory
Definition
Little's Law is a fundamental theorem in queueing theory that relates the average number of items in a queueing system (L), the average arrival rate of items (λ), and the average time an item spends in the system (W). The law is expressed as:
L = λ * W
Where:
L is the average number of items in the system
λ is the average arrival rate of items
W is the average time an item spends in the system
Application in Performance Engineering
Little's Law is incredibly useful in performance engineering for several reasons:
- Simplification: It provides a straightforward way to relate key performance metrics without needing to delve into complex calculations.
- Versatility: The law applies to a wide range of systems, from simple queues to complex networked environments.
- Predictive Power: It can help predict how changes in one metric (e.g., arrival rate) will affect others (e.g., system load).
The formula above in performance testing terms connects three fundamental performance metrics: the number of users, transaction rate, and response time. Specifically:
The number of users (U) active on an application equals the rate of transactions (T) initiated by users multiplied by the average response time (R). Therefore, the formula is:
U = T * R
Where:
U represents the user load in the system
T stands for transactions per second
R is the average response time
Thus, User Load = Transactions per second * Response Time
Despite its apparent simplicity, this formula often presents challenges for performance testers when creating test scenarios. Understanding and simplifying this formula accurately is crucial.
Example to Highlight Missing Factors
Consider a client requirement: 100 users accessing the server at 5 transactions per second, with each transaction having a 2-second response time. Substituting these values into the formula gives us:
100 = 5 * 2
100 = 10
The above equation does not hold true. What's missing?
To make the scenario realistic, a performance tester includes think time. Think time is the period a real user takes to interact with a page, such as reading content or filling out a form. Suppose the think time per page is 5 seconds. For 5 pages (Home, Login, Search Product, Product Order, Logout), the total think time is 20 seconds (i.e., ((5-1) * 5)). The logout page requires no think time, so only 4 pages are considered. Since think time also affects response time, Little’s Law updates the equation to:
100 = 5 * (2 + (4 * 5))
100 = 5 * (2 + 20)
100 = 5 * 22
100 = 110
Again, this does not balance.
To achieve the desired TPS, performance testers add pacing (the delay between two user journeys). Let's assume a pacing time of 70 seconds. Including this in the response time, the equation becomes:
100 = 5 * (2 + 20 + 70)
100 = 5 * 92
100 = 460
Clearly, the formula does not satisfy the client's requirement. Thus, after considering think time and pacing, the formula based on Little’s Law is:
U = T * (R + TT + P)
User Load = Transactions per second * (Response Time + Total Think Time + Pacing)
This is the basic formula used for workload modelling. However, if the results do not align with client expectations or the server experiences more load than anticipated due to incorrect calculations, manual adjustments are necessary.
Potential reason for incorrect calculations:
In Little’s Law, the average arrival rate and waiting time refer to a single transaction. However, in performance testing, waiting time should represent the total time a user spends on the website from login to logout.
Resolution:
In our example, a user navigates 5 pages (or transactions) during an end-to-end journey, with each page having a 2-second response time. The total response time for the journey is 10 seconds (2 seconds per page). The revised formula is:
User Load = Transactions per second * [Overall Response Time + Total Think Time + Pacing]
100 = 5 * [(2 + 2 + 2 + 2 + 2) + (4 * 5) + 70]
100 = 5 * [10 + 20 + 70]
100 = 5 * 100
100 = 500
This indicates another issue: the given transaction rate is per page, but the response time metric is for the entire journey. Therefore, transactions per second must be divided by the number of pages to convert to iterations per second:
100 = (5 / 5) * [(2 + 2 + 2 + 2 + 2) + (4 * 5) + 70]
100 = 1 * [10 + 20 + 70]
100 = 1 * 100
100 = 100
Thus, the modified formula for workload modelling in performance testing, based on Little's Law, is:
User Load = (Transactions per second/Number of pages or requests) * (Overall Response Time + Total Think Time + Pacing)
By integrating Workload Modelling with Little's Law, performance engineers can gain a comprehensive understanding of system behaviour. Workload models provide detailed insights into how different types of transactions impact system performance, while Little's Law offers a high-level view of the relationship between key performance metrics.
Conclusion
Workload Modelling and Little's Law are indispensable tools in the performance engineer's toolkit. By understanding and applying these concepts, we can predict system behaviour, optimize performance, and ensure the systems are robust and responsive under various load conditions. Whether we're dealing with a high-traffic website, a complex enterprise application, or any other performance-critical system, mastering workload Modelling and Little's Law will empower us to deliver optimal performance and a superior user experience.