The 2020 CRASH Report: Bigger Applications, Poorer Security, and Busting the Off-the Shelf Myth


Being in the Software Intelligence space, it’s only natural that we collect some of that intelligence to measure and assess enterprise IT software. CAST technology is used to evaluate the structural condition of about 10,000 core IT applications every year. A subset of these analyses is permitted to flow into our anonymized benchmarking database, CAST Appmarq, by far the most expansive repository of Software Intelligence on real IT systems anywhere in the world. These are not just “apps”, rather they are mission-critical systems that form the basis for operations at large multinational organizations. Every two years, CAST Research Labs uses this data to publish an industry report – the CAST Research on Application Software Health, or CRASH Report, for short.

The 2020 CRASH Report is here. And, as always, it is chock full of data and analysis about the software that runs businesses and governments. We pulled a few key takeaways from the report and have outlined them in this article.

No Such Thing as Off-the-Shelf at Enterprise Scale

Since we’re looking at a pretty representative sample of all the world’s large enterprise applications, we can start to draw some conclusions based on size distributions alone. It’s interesting to note that the ABAP applications are generally larger than Java and .NET custom applications. Between .5 and 5 MLOC, the percentage of large ABAP applications is even higher than COBOL. This doesn’t bode well for COTS systems, or even low and no-code solutions. In the end, large enterprises need to customize even the most standard off-the-shelf software - they always have, and they always will. Furthermore, these customizations tend to be large.

Application size distribution


Applications Are Getting Bigger

Even though we’re all talking about refactoring our applications and changing big monoliths into bands of microservices, the overall trend is not moving in our favor. The mean application size in the previous CRASH report, released two years ago, was 554,782 lines of code and the median was 146,359. In the current sample, the mean grew to 618,338 lines and the median to 166,149. That’s an increase of 13.5% in two years. Of course, some of those applications might be modularized, with some decoupled components. But they’re still getting bigger.

Microsoft Tech is Large in the Enterprise

The other interesting observation here is that .NET applications tend to be larger than Java-based applications. The last CRASH report also showed a prevalence of large .NET applications. That continues to be the case in the current dataset, attesting to Microsoft’s success in the enterprise market.

Enterprise Security Programs Are Failing

Looking at the number of software flaws, or rule violations, per thousand lines of code (KLOC) gives us a metric for the density of security flaws. Exploring that metric based on technology yields some interesting findings.

densities of software flows


The first observation is that the number of security findings is higher than any other category of software flaws. This is surprising. One would think that most organizations have gotten these issues under control, with all the investment and focus on security. Every large organization has a SAST solution to remove coding weaknesses. CAST Application Intelligence Platform finds many of the same issues as typical SAST products, but more importantly it flags system-level security flaws that are more structural and therefore insidious. The findings in this security benchmark are a combination of both. The IT industry seems to still have room for progress in raising its application security posture.

Software Flaw Densities Correlate to Maintenance Cost

Perhaps less urgent, but still important especially in today’s economic climate is the cost implications of flaw densities. ABAP has the highest overall density of flaws and COBOL has the lowest. This correlates strongly with maintenance cost. Up until recently, as the COBOL developer market is getting distorted by severe staff shortages, COBOL has been the least expensive software to maintain and ABAP the most expensive (when adjusted per amount of functionality). Having to maintain systems that contain many flaws certainly takes more effort than systems with low technical debt.

Insurance Has More Agile Software than Banks

Vertical analysis was done on multiple health factors of Java EE applications, since that is the largest tech sample. Here we feature the analysis on the Changeability of applications. Government systems look especially strong with regards to Changeability, which makes sense as we know they tend to follow standards such as CISQ for rigorous quality control. The Financial Services industry has one of the greatest numbers of bad applications that are hard to change or enhance. Telecom is also quite bad with a few outliers that have extreme numbers of Changeability flaws. Insurance applications, on the other hand, have much lower violation density, among the best across all verticals. Maybe that is why we often hear about expensive maintenance in banking systems, and major failures whenever they try to change anything or add services. Insurance companies have been slowly transforming their technology without making national news headlines with tech glitches.

Whatever You Do, Know Where You Stand

weaknesses per klocThere’s a lot more to this report than we can cover in a short article. You can have a look for yourself here. We also recommend that you examine your most critical applications and see how you compare to your industry peers. To do that, you need to analyze your software using the CAST Application Intelligence Platform. If your business depends on software, it is advisable to know whether you’re standing on a solid foundation, or on digital quicksand.


Filed in: CAST News
  This report describes the effects of different industrial factors on  structural quality. Structural quality differed across technologies with COBOL  applications generally having the lowest densities of critical weaknesses,  while JAVA-EE had the highest densities. While structural quality differed  slightly across industry segments, there was almost no effect from whether the  application was in- or outsourced, or whether it was produced on- or off-shore.  Large variations in the densities in critical weaknesses across applications  suggested the major factors in structural quality are more related to  conditions specific to each application. CRASH Report 2020: CAST Research on  the Structural Condition of Critical Applications Report
Get the Pulse Newsletter  Sign up for the latest Software Intelligence news Subscribe Now <>
Open source is part of almost every software capability we use today. At the  very least libraries, frameworks or databases that get used in mission critical  IT systems. In some cases entire systems being build on top of open source  foundations. Since we have been benchmarking IT software for years, we thought  we would set our sights on some of the most commonly used open source software  (OSS) projects. Software Intelligence Report <> Papers
Making sense of cloud transitions for financial and telecoms firms Cloud  migration 2.0: shifting priorities for application modernization in 2019  Research Report
Rado Nikolov
Rado Nikolov EVP & CMO
Rado is a passionate marketer with an entrepreneurial streak and a track record of putting products on the map. Throughout his years of success in developing, selling and marketing enterprise software to senior IT & LOB leaders, he experienced firsthand the pervasive lack of clear visibility into the condition of complex software systems, until he discovered the software intelligence produced by the ‘MRI for software’, from CAST.
Load more reviews
Thank you for the review! Your review must be approved first
New code

You've already submitted a review for this item