Agile and DevOps are arguably two of the most significant movements in software development in the latest decade. Recent studies provide a look at how these methods and practices affect how software is made. The 2018 State of DevOps Report, delivered by DORA and industry partners, surveys the effect of DevOps practices on the software industry. While VersionOne’s Annual State of Agile surveys Agile adoption and impact.
Both studies provide a look into how development and delivery teams are maturing in Agile and DevOps as well as insight into how the teams perceive the value these practices have on their business. However, there are some blind spots in these surveys that prevent us from fully understanding the value to the business and most importantly how the software developed and delivered improves outcomes.
Agile and DevOps Have and Are Changing the Way Software is Made
The State of DevOps report provides a view of the DevOps industry with studies spanning five years and over 30,000 survey responses. This research provides insight into the return on investments in technology, process, and culture drive profit, quality, and customer outcomes. The report covers a lot of aspects ranging from cloud development, use of open source components and the effects of outsourcing. For the sake of scope, I’ll mention the most relevant key findings to this discussion below.
As you can see, DORA finds that elite performers significantly outperform others in the areas of code deployment, lead times from commit to deploy, change failure rates and time to recover from incidents. The report adds that software delivery and availability unlock competitive advantages such as increased profitability, productivity, market share, customer satisfaction. They identify key technical practices such as monitoring and observability, continuous integration, database change management and integrating security earlier in the software development process drive high performance in software delivery.
The VersionOne’s annual State of Agile study reports positive findings of the use of and effect of Agile practices. Over the past few years, the survey has consistently shown that organizational culture stands out as a critical factor in the success of adopting and scaling agile. VersionOne reports that 25% of the respondents say that ‘all or almost all’ of their teams are agile, up 8% from 2016. They report that top cultural factors contribute to scaling agile including internal agile coaches (53%), consistent practices and processes across teams (43%), and the implementation of a common tool across teams (41%). The most notable changes from last year’s survey are the growth in the importance of Customer/User Satisfaction over business value.
Overall as an industry, are moving in the right direction in terms of developing and delivering software. Which aligns to DevOps and Agile thinking regarding the motivation to adopt Agile and DevOps – a focus on software delivery. A more complete list of the top reasons cited for adopting Agile and DevOps includes:
However, there a few gaps in both reports that don’t adequately answer the question if we are improving in all these areas. Specifically, are we developing and delivering better software; listed above as enhance software quality, reduce project risk, improve engineering discipline, increase software maintainability and improving software availability?
One gap in these reports is that they are survey based. Martin Fowler makes this point in his foreword of DORA’s book, Accelerate:
“…they are still surveys that capture subjective impressions, and I wonder how their population sample reflects the general IT world. I’ll have more confidence in their results when other teams, using different approaches, are able to confirm their reasoning.”
Without getting into a religious discussion, I point this out as an area of improvement not as an attack on either study. In my next post, I share research efforts that are doing as Mr. Fowler suggests, “using different approaches, are able to confirm their reasoning.”
The second concern are the measures: better measures exist to more clearly indicate the goodness of the software itself. Here Martin Fowler reminds us that DORA’s findings are on IT Delivery, not the entire software development process. Almost all the measures used by DevOps and Agile communities are about the process, not about the product. There software product measures that can provide a more robust view of the ‘output’ of the software development process. These can be automatically gathered throughout the development process that, when combined with DORA findings, can provide a more holistic view of the teams’ performance based on objective analysis and measurement of the software itself.
While it is a clear and positive finding that Agile and DevOps continue to accelerate, the question many organizations, executives and researchers are asking is: Are Agile and DevOps practices improving the software itself?
In order to know the answer to that question, we must first seek to recognize how the inclusion of product measures can improve our understanding of the impact of Agile and DevOps and enhance feedback to development and delivery teams helping them to improve the engineering discipline, customer satisfaction and value to their business.