IT Vendor Management Trends: Four Unexpected Benefits of Measuring Software Quality


We’re well into into a serious—and positive–shift in IT Vendor Management best practices. Since 2015, Gartner’s IT Leadership Vision reports have been studying a decision-making climate in which business-side stakeholders are gaining more clout than ever in making technology outsourcing budget decisions. Gartner analysts advise Vendor Managers to “communicate expectations for current and future vendors,” and to use tools and analytics to manage software risk.

To achieve objectivity with vendors, CIOs must set standards for software quality that clear the air of opaque, subjective judgments being made by indirect stakeholders. Following standards, like those put forward by the Consortium for IT Software Quality (CISQ) can help CIOs make decisions on issues like build vs. buy, sunsetting applications or just cutting a project altogether.

Amex and CISQ Raise the Bar

“Outsourcing allows you to bring in a global talent pool, but there is a lack of direction and objectivity in terms of software quality standards. Are you receiving the quality you expect?” asked Marc Cohen, Vendor Management Practitioner at American Express, in a recent webinar.

Cohen’s presentation detailed how Amex is using CISQ software quality standards, including those addressing efficiency, security, and robustness, with its vendor community to raise the bar on software development and testing. Working with a partner that adheres to CISQ standards, he says, removes ambiguity. “We had different vendor teams: one for development, one for testing. Each team had very different approaches and viewpoints in terms of quality expectations.”

By implementing CISQ software quality measures and an automated software measurement platform, Cohen’s Amex program has established a single means of measurement for all teams.

Surprise, Surprise: What Cohen Didn’t Expect

Amex created a monthly code-scanning and analysis process that included onboarding vendors, base-lining all original code, and communicating analysis results to facilitate teams’ revisions. And while Cohen and his Amex colleagues certainly expected to achieve their software quality goals, they also realized some unexpected benefits: 

Unexpected benefit #1: Eager vendors. “They absolutely welcomed the opportunity for deliverables to be measured in an objective perspective, to avoid finger pointing and subjective judgments,” says Cohen. “We trained them to ensure that their resources could understand and use the measurements on our platform of choice.”

Why the enthusiasm? "Some were having more difficulty than others in managing their resources, so the standardization was welcomed,” Cohen explains.

Unexpected benefit #2: More opportunity to focus on development. With this analytic standardization and revision process in place, Cohen and his teams can now focus on security, maintainability, reliability, performance efficiency—and foster more meaningful development that mapped to business outcomes.

“We gained a greater ability to focus our IT labor efforts, based on organizational needs. We can reduce risk, and make code more secure and more robust, while increasing reliability and maintainability. This was especially true as we moved towards an Agile sprint environment in which we’re releasing code continuously into the environment.”

Unexpected benefit #3: A “win win win” atmosphere for everyone. Amex has always fostered partnerships and a team spirit. Now, with the new metrics in place, these objectives became that much more attainable. “Whether you’re part of a development team or a full time employee, you have the same expectations and goals,” says Cohen.

Unexpected benefit #4: A large and sudden lift in resource quality. “We used to think that we were a training ground, but with these measures in place, we got highly skilled resources on our development and testing work,” says Cohen. We thank the CISQ standards for making this possible.

Successful narratives such as Cohen’s–including these great takeaways–underscore the serious business benefits of software quality standards, especially when managing outsourcers. It’s like building a home: you can outsource its construction, but you can’t outsource your responsibility. While it’s your vendor’s responsibility to provide secure, reliable work, monitoring their work and its quality is ultimately on you. So why not protect your assets from the start? It’s your house.

Learn more about getting insight in the true effort, work and quality delivered by your vendors here

Lev Lesokhin
Lev Lesokhin EVP, Strategy and Analytics at CAST
Lev spends his time investigating and communicating ways that software analysis and measurement can improve the lives of apps dev professionals. He is always ready to listen to customer feedback and to hear from IT practitioners about their software development and management challenges. Lev helps set market & product strategy for CAST and occasionally writes about his perspective on business technology in this blog and other media.
Load more reviews
Thank you for the review! Your review must be approved first
You've already submitted a review for this item