Why Measurement Matters in Outsourced Application Development

by

I was recently interviewed by Heiner Himmelreich at Boston Consulting Group about the critical role software measurement plays in outsourced application development. Below is an excerpt of our discussion:

What is the state of software measurement in outsourced application development?

The quality of software measures has gotten better, but the quality of their use has gotten better only in disciplined organizations. Disciplined organizations know that you can’t manage outsourced application development effectively without numbers. Successful improvement techniques like lean and Six Sigma rest on process and product measurement. Strong measurement is a hallmark of strong management.

What about agile development?

Many so-called agile projects are not performing agile methods rigorously. They use the agile label as an excuse to shortcut practices, especially measurement. However, I have visited disciplined agile organizations whose walls were covered with burn down charts, test progress displays, and other measures of effectiveness.

Are there areas where disciplined organizations could improve?

Product measurement has been weak for decades. Testing assesses mostly the functional aspects of applications, leaving the nonfunctional, structural quality poorly assessed. Without analyzing the source code, especially at the architectural, system level, organizations cannot predict their operational risk or the cost of corrective maintenance. Too often, cost exposures are not discovered until application maintenance is outsourced.

Case studies at Allianz, Credit Suisse, AT&T, and other organizations have reported reductions in defects, operational incidents, and corrective maintenance costs of 50% or more when structural quality measures were used to enforce improvement targets. Companies should insist that their suppliers use these measures.

Are there standards for structural quality measures?

The Consortium for IT Software Quality developed standards that were adopted by the Object Management Group [an international technology standards consortium] for measuring reliability, security, performance efficiency, and maintainability in source code. These measures identify violations of good architectural and coding practice that are so severe they must be eliminated.

How should these measures be used in managing outsourcers?

CISQ measures should be included in contracts for establishing acceptance thresholds and for determining award fees. Acceptance targets, such as no more than one reliability weakness per 100 function points, can be set for each measure. These measures should also be used for specifying weaknesses, such as SQL injection, that must not appear in the code.

In an era of nine-digit defects [failures whose related damages exceed €100,000,000], executives must establish policies that include structural quality measures, and they must hold their organizations accountable for using those measures to improve applications.

Get the Pulse Newsletter  Sign up for the latest Software Intelligence news Subscribe Now <>
Open source is part of almost every software capability we use today. At the  very least libraries, frameworks or databases that get used in mission critical  IT systems. In some cases entire systems being build on top of open source  foundations. Since we have been benchmarking IT software for years, we thought  we would set our sights on some of the most commonly used open source software  (OSS) projects. Software Intelligence Report <> Papers
In our 29-criteria evaluation of the static application security testing (SAST)  market, we identified the 10 most significant vendors — CAST, CA Veracode,  Checkmarx, IBM, Micro Focus, Parasoft, Rogue Wave Software, SiteLock,  SonarSource, and Synopsys — and researched, analyzed, and scored them. This  report shows how each measures up and helps security professionals make the  right choice. Forrester Wave: Static Application Security Testing, Q4 2017  Analyst Paper
This study by CAST reveals potential reasons for poor software quality that  puts businesses at risk, including clashes with management and little  understanding of system architecture. What Motivates Today’s Top Performing  Developers Survey
Bill Curtis SVP and Chief Scientist at CAST
Dr. Bill Curtis is SVP and Chief Scientist of CAST and heads CAST Research Labs which reports statistical profiles of global trends in structural quality, as well as applying visualization and machine learning to structural quality analysis. He is also Executive Director of the Consortium for IT Software Quality (CISQ), chartered to produce international standards for automating the measurement of structural quality from source code. He co-edits several ISO 25000 series software quality standards. He is best known for leading development of the Capability Maturity Model (CMM) and People CMM at the Software Engineering Institute. He has 40 years’ experience in software, has authored 4 books and 150+ papers, and is a Fellow of the IEEE for his contributions to software process improvement and measurement.
Load more reviews
Thank you for the review! Your review must be approved first
Rating
New code

You've already submitted a review for this item

|