The purpose of this white paper is to portray the worldwide state of agile adoption for our readers. While much has been written about the strengths and weaknesses of the technology, little data has been published to show how widely agile methods are used. This paper corrects that by providing data from our databases for public consumption. As shown in Figure 1, agile methods have become the dominant software development paradigm used throughout the world based on data from 330 organizations. Some of these organizations are offshoots of the 120 firms and government organizations from which we have received data. Figure 2 summarizes which agile methodologies are in use by these organizations. As many said that they were using a hybrid approach, i.e., one that combined agile with traditional concepts, we have included their response and categorized them as either hybrid or hybrid/lean (agile combined with lean).
We used a technology introduction life cycle model developed by Moore (1) to identify the stage of adoption firms were in. As part of this use, we assumed a technology like agile would take 18 years +/- 3 to get fully adopted based on studies done by Riddle and Redwine (2). The four stages of technology adoption are illustrated in Figure 3 and are as follows:
- Early adopters – organizations in this stage took agile in its early stages and put it to work operationally to derive maximum benefits. They took some risk with the methodologies and worked with the vendors to refine them. The primary challenge was stabilizing the methods.
- Early majority – organizations in this stage jumped on the bandwagon as agile started to receive good press and adopted the methodology. It should be noted that the early adopters during this stage fanned the methodology out and put it to use organization-wide. Again, the primary motivation was the benefits. The primary challenge was fan-out.
- Late majority – more conservative organizations now got on the bandwagon with agile. They waited until they thought all was stable and then rushed to take advantage of the benefits. Most used a normal change management process where they first got executive support, then trained, piloted, adapted and rolled out a methodology with vendor help, and finally tuned their products for widespread organizational use.
- Laggards – those organizations who for whatever reason stick with their existing way of doing business. Most are considering moving to agile. However, they might have some good business reason for sticking with their existing ways of doing business. Independent of the reason, they are not currently tapping the benefits of agile adoption.
The chasm represents the time it takes to get the technology adopted by the early majority. This is the difficult time because it is here where effort is needed to get the vast majority of users to stop what they are currently doing and move to a new way of doing business.
Government versus Industry Agile Use
Table 1 looks at government versus industry agile use. As expected, government lags. However, the surprise here is that government in Europe, primarily the United Kingdom, embraced agile methods at least five to eight years before the agencies in the United States made a like commitment. The Table also shows based on the data supplied, industry is past adoption. Government lags behind by between 5 to 7 years. It also shows that over half the organizations in industry are well beyond the early adopter stage of the technology introduction life cycle.
Please note that we did not have enough data to complete the table for all geographical areas. For instance, we included only the United States instead of the Americas because our data from Canada and Brazil was lean. We also stripped out the United Kingdom from our European data to illustrate its agile leadership position in this theatre of operations.
Lessons Learned by Early Majority Adopters
The experiences of those in the early majority paves the way for others who are interested in inserting agile methods in their organizations. Their experiences are valuable because they have gone through the transformation process, fanned out the technology, and put the methods that they finally adopted to work in their development operations. When first adopting, the following success tactics (3) are deemed important by those embracing agile methods:
- Define your transformation goals and expectations, i.e., is agile for use in one organization or enterprise-wide, and develop related measures of success.
- Tie your agile transformation effort to your firm’s business goals and tailor your measures of success accordingly.
- Educate your management, i.e., from executives on down; get their support; and then engage them and keep them informed throughout the development.
- Educate your entire organization, including your customers, as to why you are changing.
- Select the agile method that you will use, train your people in its use, and tailor it as you use it on one or more pilot projects; i.e., at least one pilot for each organization that will use it.
- Have the team that pilots the methodology define a set of rules, metrics and processes for its use. Make sure that this extra effort does not weight them down and cause them to fail.
- Develop a train the trainer program for use during fan-out as part of your pilot project. Make sure that you use real project artifacts and experiences as examples in the courseware.
- Refine the methodology as you tailor it for use on the current and next projects.
- As the pilot nears its end, develop a plan of action and milestones aimed at fanning the methodology out organizationally or enterprise-wide. If need be, create a separate team to ready for the transition by creating the infrastructure and prerequisites for success.
- Publish a retrospective when you complete the pilot to ensure you capture lessons learned.
When fanning the technology out organizationally or enterprise-wide, here are a few things you can do to speed up the process
- Start small, build big. The sum of a series of a few small wins often turns into a big one.
- Fan experienced people from your pilot projects out to lead the next projects and train the staff.
- If using Scrum (4), build up a cadre of Scrum Masters so you can use them effectively.
- Work with the product owners and help them understand their role; i.e., they are there to set priorities and act as the voice of the customer, not act as a project manager.
- Embrace test-first concepts and automate your regression testing; i.e., those tests that you baseline and use to revalidate your builds each iteration, whenever possible.
- Look for risks at your daily standups and then prioritize and manage them based on their consequences.
- Enlist the aid of your quality assurance personnel as you look for opportunities to cut waste. Understand when push gives to shove, quality stands out as the discriminator.
- Use the numbers; i.e., per the metrics and measures that you defined, to manage.
- Build momentum for change through people by spreading the word via networking, newsletters and publicity events that it works, is good for the company, is good for the employee and is fun.
When in development operations with mature agile processes, the following ideas may help:
- Continue to refine and reinvent yourself. Else, the momentum will die and your initiative will end. Use retrospectives as a learning tool. Reinvention can be simple. Embrace the next best thing. For example, if agile, take on lean. They are compatible and complementary.
- Modify your infrastructure and institutionalize your processes. This doesn’t mean setting up a process group and conducting appraisals like under CMMI® (5). Instead, create a handbook and training. Establish roles for support groups and career paths for new jobs like product owner and Scrum Master. In addition, put quality assurance personnel to work helping to engineer quality into the products rather than trying to inspect it in.
- Finally, as agile is scaled, look at redefining project management roles and responsibilities. Make them the synchronizers and coordinators rather than the overlords. Look at methods like Disciplined Agile Delivery (DAD) (6) for hints at realizing balance between agile and more traditional methods if this is what you think will work for you.
How Laggards Can Catch Up
Being behind isn’t always a bad thing. By being smart, I believe that you shave the amount of time and effort required to make the transformation to agile. Instead of going through the entire process, you can take the following shortcuts to move to agile in two to three rather than four to five years:
- Acquire rather than build your primary agile capabilities by buying a firm putting its people to work as your primary agile workforce. This gets rid of a year of piloting and around 10 staff-years of effort assuming that you can quickly adapt the methods that they have embraced to your line of business. To move out smartly, key personnel clauses and bonuses may be necessary. These contractual clauses are aimed at retaining the key resource you are after as part of the acquisition, agile skilled and experienced people.
- If you cannot buy a business, hire a qualified firm to help you. This is a good tactic when you are developing products on contract and can pass off the costs to the third party paying the bills. Make sure that the candidates will take your best interests to heart and not treat the engagement as a money stream. Use win-win terms to make the partnership rewarding for all parties. Make them put up something to gain. For example, ask them to provide free training and/or tools as a consideration to show that they are truly committed to partnering. Provide them with referrals and testimonials as part of your commitment to them.
- Avoid consultants. While they can help in certain situations, they often bleed the effort for their own personal gains. Go with an agile consulting firm as an alternative. If they work out, buy or partner with them.
- Focus on the bottom line. Gather the data as part of your transformation to keep the critics at bay. Use the numbers to show advocates and critics alike that agile is the better way. Hard data has always won arguments for me. I am sure that it will work for you as well.
Summary and Conclusions
In this white paper, I have shown that agile is here. The data shows that it is no longer an emerging technology. While government lags, that does not mean that they should standby. In summary, those organizations that are just starting on their agile path are behind the adoption curve. They need to move out aggressively or they will find it difficult to get their job done.
To help, I have also provided lessons learned for those on their journey to disciplined agile use. Both early adopters and those fanning the technology out will find that the suggestions that I have offered will make their lives easier.
Finally, for those considering moving agile, what’s keeping you? You are behind the competition. You are a laggard. You need to catch up. Else, the agile freight train will either leave you behind or run you over. Again, I have made suggestions that you might take to shave the amount of time and effort you will need to expend to catch up. So, do it.
- Geoffrey Moore, Crossing the Chasm: Marketing and Selling High-Tech Products to Mainstream Customers, Harper, 1999.
- William Riddle, “The magic number eighteen plus or minus three: a study of software technology maturation,” ACM SIGSOFT Software Engineering Notes, Vol. 9, Issue 2, April 1984, pp. 21-37.
- Donald Reifer, Software Change Management: Case Studies and Practical Advice, Microsoft Press, 2011.
- Mike Cohn, Succeeding with Agile: Software Development Using Scrum, Addison-Wesley, 2009.
- B. Chrissis, M. Konrad and S. Shrum, CMMI®: Guidelines for Process Integration and Product Improvement, Addison-Wesley, 2003.
- See the following web site for description: http://disciplinedagileconsortium.org/.
About Reifer Consultants LLC
Reifer Consultants LLC is a small business that specializes in agile and lean transformations. During the 30+ years we have been in business, we have helped our clients make the leap forward. We have been involved with agile adoption efforts since 2004. To demonstrate success, we believe in using the numbers to make decisions and keep the momentum rolling. If interested, we also offer a variety of agile training and consulting services.
Reifer Consultants LLC
14820 N Dragons Breath Lane, Prescott, AZ 86305-5644
Erik Oltmans, an Associate Partner from EY, Netherlands, spoke at the Software Intelligence Forum on how the consulting behemoth uses Software Intelligence in its Transaction Advisory services.
Erik describes the changing landscape of M & A. Besides the financial and commercial aspects, PE firms now equally value technical assessments, especially for targets with significant software assets. He goes on to detail how CAST Highlight makes these assessments possible with limited access to the targetâ€™s systems, customized quality metrics, and liability implications of open source components - all three that are critical for an M&A due diligence.