Earlier this week, our own Jitendra Subramanyam joined industry luminary Capers Jones, Chief Scientist Emeritus of Software Productivity Research (SPR) to co-host a webinar on curbing application software outages like the ones seen in the financial sector over the past couple months. The webinar, titled “Stop High-Profile Outages by Quantifying Application Risks,” focused on the importance of static analysis of application software during the build and/or customization phases to identify potential issues than can them be fixed, preventing a future outage.
Effectiveness of Static Analysis
Jones has long been a proponent of static analysis over merely testing software. In his 2009 book, Applied Software Measurement, Jones wrote, “In terms of defect removal, testing alone has never been sufficient to ensure high quality levels. All of the best-in-class software producers such as AT&T, HP, Microsoft, IBM, Raytheon or Motorola utilize both pretest design reviews and formal code inspections. Design reviews and code inspections can both be used with client-server applications and should improve defect removal efficiency notably.”
It was this point that Jones and Subramanyam stressed throughout the webinar. They noted that defects in software design are the hardest to catch and eliminate and urged developers not to wait until testing to try to find these defects. They said that rather than waiting until testing, to identify and catch defects early with automated code reviews and static analysis. To illustrate his point further, Jones revealed the following:
- Testing by itself is time consuming and not very efficient. Most forms of testing only find about 35% of the bugs that are present.
- Static analysis prior to testing is very quick and about 85% efficient. As a result, when testing starts there are so few bugs present that testing schedules drop by perhaps 50%. Static analysis will also find some structural defects not usually found by testing.
- Formal inspections of requirement and design are beneficial too. Formal inspections create better documents for test case creation, and as a result improve testing efficiency by at least 5% per test stage.
- A synergistic combination of inspections, static analysis and formal testing can top 96% in defect removal efficiency on average, and 99% in a few cases. Better, the overall schedules will be shorter than testing alone.
- The average for a combination of six kinds of testing - unit test, function test, regression test, performance test, system test and acceptance test - without preliminary static analysis is only about 85%.
A Quality Foundation
The main idea the co-hosts intended to resonate with those who attended the webinar was the importance of building in structural quality from the very start. One way to do this is to incorporate CAST’s Automated Analysis and Measurement into the application software development process.
CAST automates the analysis and measurement of applications. Covering a wide range of platforms, languages and frameworks, CAST incorporates software engineering and application domain expertise into its algorithms. Subject matter experts use CAST’s objective quality metrics to quickly drill down to root causes and remediate quality hot spots. Improvements in quality are quantified using the same quality measures, making it possible to quantify the effectiveness of quality improvement activities and satisfy the six essential ingredients of effective code reviews.
By incorporating CAST into the development processes, businesses can go a long way toward preventing high-profile outages and take the risk out of that part of their businesses.