Do DevOps and Agile Improve Software Quality?

by

DevOps and Agile are arguably two of the most significant movements in software development in decades. Recent studies provide a look at how these methods and practices affect how software is made. The 2018 State of DevOps report, delivered by DORA and industry partners, surveys the effect of DevOps practices on the software industry. While VersionOne’s Annual State of Agile surveys Agile adoption and impact.

Both studies provide a look into how development and delivery teams are maturing in DevOps and Agile as well as insight into how the teams perceive the value these practices have on their business. However, there are some blind spots in these surveys that prevent us from fully understanding the value to the business and, most importantly, how the software developed and delivered improves.

Agile and DevOps Change the Way Software Is Made

The 2018 State of DevOps report provides insight into the return on investments in technology, process and culture to drive profit, quality and customer outcomes.

DORA-State-of-DevOps

As you can see in the graphic above, DORA finds that elite performers significantly outperform others in the areas of code deployments, lead times from commit to deploy, change failure rates and time to recover from incidents. The report adds that software delivery and availability unlock competitive advantages such as increased profitability, productivity, market share and customer satisfaction. They identify vital technical practices such as monitoring and observability, continuous testing, database change management, and integrating security earlier in the software development process drive high performance in software delivery.

Similarly, VersionOne’s State of Agile study reports positive findings of the use of and the effect of Agile practices. Over the past few years, the survey has consistently shown that organizational culture stands out as a critical factor in the success of adopting and scaling agile. VersionOne reports that 25% of the respondents say that ‘all or almost all’ of their teams are agile, up 8% from 2016. They also found that cultural factors contribute to scaling agile including internal agile coaches (53%), consistent practices and processes across teams (43%), and the implementation of a common tool across teams (41%). The most notable changes from last year’s survey are the growth in the importance of Customer/User Satisfaction over business value.

So overall, we, as an industry, are moving in the right direction concerning developing and delivering software. Which aligns to DevOps and Agile thinking regarding the motivation to adopt Agile and DevOps – a focus on software delivery. A complete list of the top reasons cited for adopting Agile and DevOps includes:

  • accelerate software delivery
  • enhance the ability to manage changing priorities
  • increase productivity
  • improve business/IT alignment
  • enhance software quality
  • enhance delivery predictability
  • improve project visibility
  • reduce project risk
  • improve team morale
  • improve engineering discipline
  • reduce project cost
  • increase software maintainability
  • better manage a distributed team  
  • improve software availability

However, there a few gaps in both reports that don’t adequately answer the question if we are improving in all these areas. Specifically, are we developing and delivering better software; listed above as enhance software quality, reduce project risk, improve engineering discipline, increase software maintainability and improving software availability.

One gap in these reports is that they are survey based. Martin Fowler makes this point in his foreword of DORA’s book, Accelerate…they are still surveys that capture subjective impressions, and I wonder how their population sample reflects the general IT world. I’ll have more confidence in their results when other teams, using different approaches, are able to confirm their reasoning.” 

Without getting into a religious discussion, I point this out as an area of improvement not as an attack on either study. In my next post, I share research efforts that are doing as Mr. Fowler suggests.

The second concern is the measures; better measures exist to more clearly indicate the goodness of the software itself. Here, Fowler reminds us that DORA’s findings are on IT delivery, not the entire software development process, “…their book focuses on IT delivery, that is, the journey from commit to production, not the entire software development process.” Almost all the measures used by DevOps and Agile communities are about the process, not about the product.

There are software product measures that can provide a more robust view of the ‘output’ of the software development process. These can be automatically gathered throughout the development process that, when combined with DORA findings, can provide a more holistic view of the teams’ performance based on objective analysis and measurement of the software itself.

While it is clear and a positive finding that Agile and DevOps continue to accelerate, the question many organizations, executives and researchers are asking is: Are DevOps and Agile practices improving the software itself? 

In my next few posts, I will present findings from recent studies that have examined the software developed in DevOps and Agile environments. I’ll illustrate how the inclusion of product measures can improve our understanding of the impact of Agile and DevOps and enhance feedback to development and delivery teams helping them to improve the engineering discipline, customer satisfaction, and value to their business.

Until then, what's your experience? Are Agile and DevOps improving the software developed and delivered? How are you demonstrating that to your organization?

Filed in: DevOps
Get the Pulse Newsletter  Sign up for the latest Software Intelligence news Subscribe Now <>
Open source is part of almost every software capability we use today. At the  very least libraries, frameworks or databases that get used in mission critical  IT systems. In some cases entire systems being build on top of open source  foundations. Since we have been benchmarking IT software for years, we thought  we would set our sights on some of the most commonly used open source software  (OSS) projects. Software Intelligence Report <> Papers
In our 29-criteria evaluation of the static application security testing (SAST)  market, we identified the 10 most significant vendors — CAST, CA Veracode,  Checkmarx, IBM, Micro Focus, Parasoft, Rogue Wave Software, SiteLock,  SonarSource, and Synopsys — and researched, analyzed, and scored them. This  report shows how each measures up and helps security professionals make the  right choice. Forrester Wave: Static Application Security Testing, Q4 2017  Analyst Paper
This study by CAST reveals potential reasons for poor software quality that  puts businesses at risk, including clashes with management and little  understanding of system architecture. What Motivates Today’s Top Performing  Developers Survey
Pete Pizzutillo
Pete Pizzutillo Vice President
Pete Pizzutillo is Vice President at CAST and has spent the last 15 years working in the software industry. He passionately believes Software Intelligence is the cornerstone to successful digital transformation, and he actively helps customers realize the benefits of CAST's software analytics to ensure their IT systems are secure, resilient and efficient to support the next wave of modern business.
Load more reviews
Thank you for the review! Your review must be approved first
Rating
New code

You've already submitted a review for this item

|