DevOps and Agile are arguably two of the most significant movements in software development in decades. Recent studies provide a look at how these methods and practices affect how software is made. The 2018 State of DevOps report, delivered by DORA and industry partners, surveys the effect of DevOps practices on the software industry. While VersionOne’s Annual State of Agile surveys Agile adoption and impact.
Both studies provide a look into how development and delivery teams are maturing in DevOps and Agile as well as insight into how the teams perceive the value these practices have on their business. However, there are some blind spots in these surveys that prevent us from fully understanding the value to the business and, most importantly, how the software developed and delivered improves.
Agile and DevOps Change the Way Software Is Made
The 2018 State of DevOps report provides insight into the return on investments in technology, process and culture to drive profit, quality and customer outcomes.
As you can see in the graphic above, DORA finds that elite performers significantly outperform others in the areas of code deployments, lead times from commit to deploy, change failure rates and time to recover from incidents. The report adds that software delivery and availability unlock competitive advantages such as increased profitability, productivity, market share and customer satisfaction. They identify vital technical practices such as monitoring and observability, continuous testing, database change management, and integrating security earlier in the software development process drive high performance in software delivery.
Similarly, VersionOne’s State of Agile study reports positive findings of the use of and the effect of Agile practices. Over the past few years, the survey has consistently shown that organizational culture stands out as a critical factor in the success of adopting and scaling agile. VersionOne reports that 25% of the respondents say that ‘all or almost all’ of their teams are agile, up 8% from 2016. They also found that cultural factors contribute to scaling agile including internal agile coaches (53%), consistent practices and processes across teams (43%), and the implementation of a common tool across teams (41%). The most notable changes from last year’s survey are the growth in the importance of Customer/User Satisfaction over business value.
So overall, we, as an industry, are moving in the right direction concerning developing and delivering software. Which aligns to DevOps and Agile thinking regarding the motivation to adopt Agile and DevOps – a focus on software delivery. A complete list of the top reasons cited for adopting Agile and DevOps includes:
However, there a few gaps in both reports that don’t adequately answer the question if we are improving in all these areas. Specifically, are we developing and delivering better software; listed above as enhance software quality, reduce project risk, improve engineering discipline, increase software maintainability and improving software availability.
One gap in these reports is that they are survey based. Martin Fowler makes this point in his foreword of DORA’s book, Accelerate, “…they are still surveys that capture subjective impressions, and I wonder how their population sample reflects the general IT world. I’ll have more confidence in their results when other teams, using different approaches, are able to confirm their reasoning.”
Without getting into a religious discussion, I point this out as an area of improvement not as an attack on either study. In my next post, I share research efforts that are doing as Mr. Fowler suggests.
The second concern is the measures; better measures exist to more clearly indicate the goodness of the software itself. Here, Fowler reminds us that DORA’s findings are on IT delivery, not the entire software development process, “…their book focuses on IT delivery, that is, the journey from commit to production, not the entire software development process.” Almost all the measures used by DevOps and Agile communities are about the process, not about the product.
There are software product measures that can provide a more robust view of the ‘output’ of the software development process. These can be automatically gathered throughout the development process that, when combined with DORA findings, can provide a more holistic view of the teams’ performance based on objective analysis and measurement of the software itself.
While it is clear and a positive finding that Agile and DevOps continue to accelerate, the question many organizations, executives and researchers are asking is: Are DevOps and Agile practices improving the software itself?
In my next few posts, I will present findings from recent studies that have examined the software developed in DevOps and Agile environments. I’ll illustrate how the inclusion of product measures can improve our understanding of the impact of Agile and DevOps and enhance feedback to development and delivery teams helping them to improve the engineering discipline, customer satisfaction, and value to their business.
Until then, what's your experience? Are Agile and DevOps improving the software developed and delivered? How are you demonstrating that to your organization?