I was recently interviewed by Heiner Himmelreich at Boston Consulting Group about the critical role software measurement plays in outsourced application development. Below is an excerpt of our discussion:
What is the state of software measurement in outsourced application development?
The quality of software measures has gotten better, but the quality of their use has gotten better only in disciplined organizations. Disciplined organizations know that you can’t manage outsourced application development effectively without numbers. Successful improvement techniques like lean and Six Sigma rest on process and product measurement. Strong measurement is a hallmark of strong management.
What about agile development?
Many so-called agile projects are not performing agile methods rigorously. They use the agile label as an excuse to shortcut practices, especially measurement. However, I have visited disciplined agile organizations whose walls were covered with burn down charts, test progress displays, and other measures of effectiveness.
Are there areas where disciplined organizations could improve?
Product measurement has been weak for decades. Testing assesses mostly the functional aspects of applications, leaving the nonfunctional, structural quality poorly assessed. Without analyzing the source code, especially at the architectural, system level, organizations cannot predict their operational risk or the cost of corrective maintenance. Too often, cost exposures are not discovered until application maintenance is outsourced.
Case studies at Allianz, Credit Suisse, AT&T, and other organizations have reported reductions in defects, operational incidents, and corrective maintenance costs of 50% or more when structural quality measures were used to enforce improvement targets. Companies should insist that their suppliers use these measures.
Are there standards for structural quality measures?
The Consortium for IT Software Quality developed standards that were adopted by the Object Management Group [an international technology standards consortium] for measuring reliability, security, performance efficiency, and maintainability in source code. These measures identify violations of good architectural and coding practice that are so severe they must be eliminated.
How should these measures be used in managing outsourcers?
CISQ measures should be included in contracts for establishing acceptance thresholds and for determining award fees. Acceptance targets, such as no more than one reliability weakness per 100 function points, can be set for each measure. These measures should also be used for specifying weaknesses, such as SQL injection, that must not appear in the code.
In an era of nine-digit defects [failures whose related damages exceed €100,000,000], executives must establish policies that include structural quality measures, and they must hold their organizations accountable for using those measures to improve applications.