Establish a quality metrics strategy at scale

By Milli d'Hangest d'Yvoy | 2/16/17

Today we are excited to bring you a guest blog post from Kevin Dunne, VP, Strategy and Business Development at QASymphonyTestplant announced a technology partnership with QASymphony on July 12, 2016. To read more about the partnership, click here.

Every engineering organization will encounter struggles in trying to get reliable and repeatable insights into their processes.  The world now relies heavily on interconnected software systems to do things as simple as sending an email to something as complex as finding cures for rare diseases.  

When software doesn’t work as expected, teams need to know what the root cause of the particular failure was.  More importantly, they need to understand what they can do going forward to detect a similar risk before it is able to cause harm.  This problem only continues to grow in complexity as companies scale and need to manage various release trains, development processes, and tool sets.  

While every team will find they need slightly different metrics in order to operate successfully, there are certainly key factors that go into building a winning strategy around managing metrics. Through discussing both successful and unsuccessful approaches with hundreds of key engineering leaders over the years, the key factors to a scalable metrics strategy that works are:

1. Showing quick value: metrics programs that work well are able to solve key problems early on and build up capabilities and insights over time.  The longer it takes for the solution to roll out, the more time there is for different teams and divisions to build their own solutions that conflict with the enterprise solutions.

2. Flexible to where and how teams work: Many solutions provide brilliant metrics but they require teams to work in unusual ways that cause conflict with the established internal processes.  Implementing a solution that allows teams to continue to work in the tools they have become accustomed to is ideal.

3. Single source of the truth: Too often, there are multiple solutions implemented to provide metrics and analytics across the various teams and tools in place.  Ideally, there can be one centralized analytics solution for key metrics that reduces confusion and standardizes the charts and data that is shared.

4. Provides value across the organization: Many teams implement new metrics without understanding the requirements across all stakeholders, both across the SDLC and from entry level to senior executive.  By increasing the number of stakeholders getting valuable metrics from the system, you will increase buy-in around maintenance and data cleansing needed to keep the system accurate.

5. Allows quick action and low maintenance: No matter the metrics system put in place for measurement, it is critical to make sure it can support a low effort process to add new metrics and pull the latest reports.  As the SDLC process and tools are certain to change as time goes along, it’s important to choose a solution that can adapt to fit the changing needs of the organization.

Though engineering processes and the metrics needed to effectively track and optimize them may always be changing, a sound strategy about how to collect, interpret, and communicate data should be comparatively future proof.  Teams should nail down a plan and gain agreement across the various stakeholder before getting deeply involved in specific tools for metrics collection and scoping individual reports.  When everyone is aligned on what the overarching goals of the metrics strategy are, the reports themselves should flow easily.

Find out more about the importance of having the right software test metrics and how to get started with improving your metrics in our latest eBook, Better Data, Better Software: The Essential Guide to Improving Software Quality with Data and Analytics.

 

Milli d'Hangest d'Yvoy

Written by Milli d'Hangest d'Yvoy

Milli is Marketing Manager at TestPlant, overseeing campaigns, communications, PR, design and events. South African small town girl, working in the big City of London, she loves surfing, coffee, cooking, rooibos tea and Land Rovers.