What draws me to Anaheim, Calif., in October is not the walking Disney characters (though there are plenty of those), but instead the STARWest, the West Coast’s largest conference on software testing analysis and review.
After a day listening to James Bach teach critical thinking for testers, I wake up extra early on Tuesday to attend the opening keynote -- Michael D. Kelly and Jeanette Thebeau discussing “What Executives Value in Testing.”
I know Michael well; he is a former president of the Association for Software Testing who worked as a test manager at Liberty Mutual, a Fortune 500 company, then went on to start his own company that is going well. His partner, Jeanette, is a bit of wild card; her background is in working with executives to shape the business message.
With this pair, I’m not quite sure what to expect.
The pair explain their research for the presentation, which includes checking public data sources, conducting a formal survey, and a series of informal interviews. The executives they focused most on were not directors of quality, or vice presidents of software development, but instead the chief -- the startup founder or established company CEO.
Between founder and established CEO, the two found a real difference of opinion; the startup founder wants speed, to ‘get the demo to work‘. He probably does not hire testers at all, nor think of testing as a distinct activity. The established CEO, on the other hand, is more interested in protecting existing markets and revenue. According to their research, the terms an established CEO was most likely to use to talk about testing were test-driven development (TDD), continuous integration, user experience, and a formal user feedback process.
After describing a little bit about company leaders, Michael switched to talk about how the large company perceives the test group. Suddenly, things got interesting.
The Large Software Organization
Michael tells the story of a friend, interviewing for a director of QA position of a large, globally distributed company. The candidate for the position, who asked to remain anonymous, said the interview was ... odd.
The people doing the interviewing were asking about how to move staff and resources around to reduce cost. The questions were about what teams to offshore, what to consolidate, what to eliminate -- but at no point did anyone say anything about what the testers actually do. In frustration, the candidate asked what the role of testing and quality was at this company; to be blunt, what do the testers actually do?
No one at the table could answer that question.
That made testing a cost center, one best restricted, perhaps eliminated. Michael’s friend, our anonymous candidate, was not particularly interested in interviewing for that kind of role, so he turned things around: Say testing was eliminated tomorrow, what business processes would break?
That question opened the floodgates, and allowed our candidate to talk about test strategy.
The Test / Executive Disconnect
To understand the differences in perspective between roles, the speakers created a very simple survey, aiming it at two groups: Test/quality people and the executives that they served.
The survey found that most testers and executives agree on their #1 role, that testers test to make sure the software solves a business problems. For testers, the #2 thing of importance was compliance with regulations -- creating documentation, or following a process, in order to comply with Sarbanes-Oxley, HIPPA, FDA, or some other sort of government or auditor standard.
Compliance with regulation was the thing least desired by senior executives! When Michael and Jeanette pushed back on this, asking why, the executives reported that they had a compliance department, full of staff members whose job it was to make sure the company complied with external requirements; what the executives wanted from the technical staff was the work, not a stamp of approval.
The second insight from the survey was the #2 value for business executives was testing to contain operational costs. This means things like compatibility, ease of migration, the ability to upgrade and rollback easily, or reducing the chance (and impact) of regression issues. Testers rated testing to control costs as the least important.
Let’s look at that survey again: While testers may be creating documentation and allowing painful work (that will continue after go-live) to meet an initial deadline, the executives surveyed viewed it as their goal to contain long-term costs.
While most of the talk is about established business executives, and a little discussion of startup founders, the two introduce a third-size company: The profitable, fast-growth company, which they describe as the “sweet spot” for software testing.
Startups, after all, don’t need testers, they need developers, building that first chunk of code. Established companies run the risk of being so large the left hand doesn’t know what the right is doing, and testing becomes just another hoop to jump through to get the code to production.
How To Fix The Disconnect
The presentation reminds me of the dangers of having conflicting expectations between staff and management. It gives me a few tools (surveys and hypothetical questions) to figure out how wide the gap is, which can lead to conversations about how to fix things.
It suggests that the best place to start is with a fundamental shift of how we think about the executives; the more we know about what they want, the more successful we can be. In order to do this, we need to understand what’s at stake, and we need to start speaking the same language.
Jeanette says that we need to remember that we can provide a lot of value, but it is our responsibility to initiate the move from “I don’t know what you do” to “I wouldn’t think of doing this without you.”
We also need to recognize and take advantage of the opportunities that present themselves, such as getting control of operational costs, playing a larger role in helping with feature scalability, and acting as a facilitator.
Aligning goals between testing and business? Well, OK, it is certainly not a new idea.
But it is most definitely a good one.