“When you can measure what you are speaking about, and express it in numbers, you know something about it.”
~ William Thomson Kelvin, physicist, mathematician and engineer
In Agile and Lean software development environments, the numbers and data are metrics used to understand, control, and improve what we do and how we do it. Insights are patterns and inferences originating from those metrics revealing strengths and weaknesses of the product/process. Agile provides the foundational basis for improving and managing the software development product and process through: active customer collaboration, dynamic change response, short development cycles, frequent deliveries, continuous face-to-face team communications (Scrum & Sprint), and ongoing learning. Improvements in quality, cost, and delivery time are achieved through the use of optimized processes, tools, methods, models, and techniques to deliver quality software quickly, repeatedly, and reliably. Lean’s main thrust is the identification and elimination of waste, which is any activity consuming resources, time, or space without maximizing customer value. Finally, anything considered not valuable by the customer is waste.
Value stream mapping is a Lean tool used to visualize and analyse end-to-end “holistic” workflow capabilities of product, value, and time from current state (idea or concept) to delivered state through the production stream as detailed in the following table.
| Workflow item |
Description |
| Stages (process) |
Steps of development lifecycle from idea to delivery visualized as both performed task & person/team resource performing task |
| Value-unit |
Communication of stage-related value-unit and conveyed data |
| Time-in-stage (stage time) |
Average time of working and wait in a stage per feature/product when value-unit is added at each respective stage/gate |
A value stream map consists of all actions (all stage value-units) both value added and non-value added to advance the product from current state to delivered state via end-to-end enhanced stage flow. Importantly, various perspectives for evaluating value must be taken into consideration to devise a working representative value stream map.[1] For example, an alternative to grouping in value-added and non-value-added categories is grouping according to aspects of task value detailed in the aforementioned reference. Connecting and tracking the flow of end-value units (realized in delivery) through the value stream with projected business results, a solution combining VSM and FLOW[2], may provide one such opportunity for future improvements. Furthermore, the tool is used to: identify process bottlenecks preventing efficient and continuous process flow, initiate root-cause analysis, direct scheduling, aid forecasting activities, and provides the basis for lean production.
Value stream management improves flow and focuses on continuously optimizing business value, producing idea (code)-to-cash, and stakeholding in product/development management meetings while ensuring efficient operating environments committed to the highest quality. Two important points regarding quality attributes and metrics[3] within the value stream include that they are:
- project-specific according to varying functions, specifications, and priorities of different software applications
- monitored differently according to position within the stages of the value stream and not only prioritized by application domain
VSM is a strategic asset in value stream managements toolbox for effectively monitoring these key performance indicators (KPIs) predicated by an accurate and reliable value stream map representing the product/process.
Metrics measurement is used by software development teams to understand, evaluate, control, make informed decisions, and predict flow of products/processes during software life-cycle stages. Alternatively stated,
“The goal of applied software measurement is to give software managers and professionals a set of useful, tangible data points for sizing, estimating, managing, and controlling software projects with rigor and precision”.
~ C. Jones, Applied Software Measurement: Global Analysis of Productivity and Quality
The five categories for the reasons/effects[4] of using and implementing software metrics are:
- Fixing software process issues (identifying problems, improvement opportunities)
- Understanding/improving quality (understanding quality level, increasing quality, ensuring thorough testing)
- Problem avoidance/rectification (faster response, fixing defects, changing behaviour, preventing issue actualization)
- Agile progress tracking (sprint progress, increasing visibility, achieving goals, balancing workflow)
- Agile planning (prioritizing, scoping, resourcing)
Based on targets/trends and potential value-units being added to the software product/process, metrics provide deeper insights to upper management ensuring effective Agile planning/progress plans are being executed. Ensuring alignment of KPIs with your goals, changes in work structure, and adjustments in your development cycle are characteristics of high performance teams in DevOps environments. The DevOps Research and Assessment LLC (DORA) has developed and validated five metrics concentrated on a “high-level systems view of software delivery and performance and predict an organization’s ability to achieve its goals.” The DORA key software metrics are presented in a complementary blog article located here.
The key metrics used in UrbanCode Velocity (now DevOps Velocity) value streams are:
- Lead time
- Wait time
- Throughput
- Work item distribution
- Change defect rate
- Load
Lead time
Lead time is the time required for a work item to go from idea acceptance as unit of work to value realization from that work that includes both value added processing time and non-value added waiting time between sub-processes. The processing time for the work item is the time duration expended by a person or team on the activity. Ideally, the lead time value should trend to one day to shorten time-to-market and improve customer satisfaction.
Wait time
The wait (queueing or waste) time is an estimate of the time that the work item spends idle in a non-productive state during its processing by the value stream. Software product customizations (SPCs) are customer tailored solutions (typically high priority) requiring implementation in the current software release. Various sources of waiting time in SPCs that value stream managers are required to minimize and/or eliminate for maximizing both customer and product value include: code reviews, QA testing, security testing, and release cycles.
Throughput
The throughput of a value stream is the average rate of work unit items completed during a time period and quantifies the level of delivery value in the flow. The rate of work units processed by a given stage in a period of time may be expressed as its stage throughput. In order to improve responsiveness to customer requirements, goals identifying throughput improvements and then implemented will yield lead time reductions for value streams with fixed load capacities.
Work item distribution
The work item distribution is the percentage of work items completed relative to the total number of work items in a given set (same item types). Sets are user-specified and may include process-based (release or sprint) or time-based (completed items in given time period) items. This metric is used to prioritize specific work types aligned with projected business results.
Change defect rate
In the value stream, change defect rate is a quality measure expressed as a ratio of the number of defects integrated within the value stream over the completed items during a given time interval. Change defect rate aids with understanding the predictability of our value stream operation and provides insights for these questions:
- Are defects being detected early in our development processes?
- Is there consistent and reliable delivery in production environments without impacting our customers or business?
Teams operating with high change defect rates have rapid advance potential at the expense of having inadequately established quality standards allowing them to effectively move quickly and efficiently. Fixing team effort on defect identification in the value stream prior to entering production provides the basis for lowering change defect rate percentage rates.
Load
The load is the number of work items (active or waiting) active in a value stream at a given time. Load measures utilization capabilities of value streams related to productivity in the process flow.
Summary
In closing, we have detailed the quantitative metrics integrated within the UrbanCode Velocity value stream while demonstrating important aspects of each KPI and its resulting contribution in the end-to-end workflow of the software product/process. Equally important are the qualitative metrics elements that incorporate (and not limited to): observations, discussions, interviews, questionnaires, and schemas which represent the subjective side of value in VSM. Realistically, the qualitative metrics do present additional processing difficulty and more detailed analysis and in exchange do provide a more representative reflection of your actual product/process coupled with UrbanCode intelligence to maximize your entire measurement spectrum for an exceptionally tuned software performance experience.
[1] McManus HL. Product Development Value Stream Mapping (PDVSM) manual. ed, 2005; 32
[2] Ali, N.B., Petersen, K., Schneider, K.: Flow-assisted value stream mapping in the early phases of large-scale software development. J. Syst. Softw. 111, 213–227 (2016)
[3] Arvanitou, E.M., Ampatzoglou, A., Chatzigeorgiou, A., Galster, M., Avgeriou, P: A mapping study on
design-time quality attributes and metrics. J. Syst. Softw. 127, 52–77 (2017)
[4] Kupiainen, E. , Mäntylä, M.V. , Itkonen, J. , 2015. Using metrics in Agile and Lean Software Development –
A systematic literature review of industrial studies. Inf . Softw . Technol . Elsevier 62, pp. 143-163.