The Quality Aspect Heatmap: Quantifying quality, the Zyro way

Let’s talk about quality strategy. 

At Zyro, the way we measure quality is essential to us. A quality strategy helps us understand where we are with our product and its change over time. 

It is a tool that connects all the teams here and challenges everyone to think about what can be done to improve the targeted area and make an impact.

The backstory

The Zyro life cycle is based on OKRs (Objectives and key results (OKR). It is a goal-setting framework for defining and tracking objectives and their outcome. 

Our year is divided into quarters, and for each quarter, we set OKRs, take ownership of a given set of responsibilities, and do our best to deliver results. At Zyro, planning is also an engaging and inspiring activity. 

There are team-level OKRs that support company-level OKRs, and we always strive to use this time to create inspiring improvements in any place that we feel can add value. There are no bad ideas, and creative planning gives rise to a better quality strategy overall.

Still, the trickiest part here is finding out which improvements are the most important and most effective and might make an even more significant impact than we expect.

 It’s also the moment that we have to take a step back and ask ourselves how we can show our teammates and other colleagues that it is worthwhile to focus their energy on a specific area that we have defined as the most important for improving quality at Zyro. 

The problems

Each OKR, the task of interrogating our objectives, methods, and ability to negotiate with our colleagues is a primary theme. 

In adopting new OKR concepts, we have to analyze the places where our initial concepts and objectives might have failed and why that was? Is the idea itself faulty, or is it our ability to communicate it to our teammates holding us back?

I’ve started asking myself why? The feeling that each team’s OKR sheet is inspiring and leads to a better product is natural at first. Still, in creating a coherent quality strategy, it is important to understand that we simply would not succeed if we try to put all of them into one big picture. 

They are all too different to contribute in a meaningful way to company-wide OKRs, so we have to hone them down into a new form.

The birth of a solution

As the Quality Assurance lead, it is my role to make the QA team act as the glue in a model airplane kit. We need to bring all the pieces of Zyro together and make them fit naturally with one another. So, I started questioning what type of glue we need in any given situation. 

Our solution? Something we like to call “quality shift-left.”

The concept of quality shift-left is to get people talking more and to create cross-functional teams as a natural part of the planning of each OKR. 

To implement this effectively, we organized a few workshops with the following goals:

• Discussing quality as a company developing an understanding of the way that we see it at Zyro
• Defining what our most important quality aspects are
• Ensuring that those aspects match our overall objectives and are working well
• The changes we would like to make in the near future

We voted on each initiative on a sliding scale of up to 7 (where one is the least important and seven is the most important).

After each round of workshops and discussions, it is my task to take the time to look at my online board, which is full of sticky notes and votes on ideas, and try to find the ideal model to move forward.

I quickly understood that I needed to find a unique solution and that it simply was not enough to give yet another Excel spreadsheet that summarized the outcome of the round of voting.

In sorting all those sticky notes, an idea came into my head. I wanted to try a thermometer approach. 

Using all those initiatives with all their numbers, I could create a heat-map that would show the relative importance of each idea from each team to the needs and objectives of the company as a whole. 

Creating a Quality Aspect Heat-map

A Quality Aspect Heatmap is a tool that allows you to measure quality from different angles without biases.

You can use it to evaluate the status of any quality aspect in a matter of minutes. 

And, most importantly, you can track the changes and make decisions accordingly. 

It’s a tool for creating a genuinely context-driven and results-focused quality strategy. It aids prioritization and helps align cross-functional teams.

Struggles:

• Creating a Quality Aspect Heatmap takes time. It is worth it as a way of finding areas of correlation in the numbers
• Courage. Be a proactive listener and communicate your ideas clearly
• Developing cross-functional teams. Engaging teammates to work towards goals outside of their immediate area is difficult. A quality heat-map should unify and spread a shared mindset that is driven by a constantly changing context.

Value:

• Quality is measured effectively
• A clear understanding of practical areas and places that need improvement
• A roadmap of the way that quality matures or regresses over time and insights on how to make timely decisions
• All other team representatives are included in the action and can contribute to the quality strategy
• Collecting brilliant minds into one place generates even more great ideas and improvements (including the heat-map itself)
• Help to align on focus areas by with unbiased metrics rather than relying on gut feeling
• Reflect, understand, and measure which elements are most effective

Concept:

• Identify a list of quality aspects/areas your team/company wants to focus on (the most common examples would be functionality, user-friendliness, performance, maintainability, suitability)

• Create a proof of concept. You could follow the Zyro path and identify the essential initiatives supporting each aspect, and vote on each quality aspect.

• Iterate your proof of concept. Ask relevant teams to help with the extraction of relevant metrics. You could use metrics such as the Net Promoter Score (user feedback on your product), user conversion, or churn rate. Performance metrics could contain Cumulative Layout Shift (CLS), First Input Delay (FID), Largest Contentful Paint (LCP), Lighthouse performance score, or you could choose any other aspect that you would like to improve.

The Quality Aspect Heatmap

So, what is the main idea of a heat-map?

Well, ideally, you want to move the colors from red to yellow and then green. 

In the process, it should help you to understand the areas that require the most attention and the ones that are evolving smoothly. 

For example, to implement an MVP visualization of this idea, you need to align all metrics on the same scale.

If you choose a scale, like Zyro, grading every idea out of seven, it is important to measure all aspects out of seven to attain cohesive results.

Tips & tricks, secrets from Zyro

So, let’s now examine how we can implement the Quality Aspect Heatmap at Zyro.

Since we need to make multiple calculations, I’ve chosen Excel as the tool for building the visual side of our Quality Aspect Heatmap. 

I would advise choosing a colleague as the DRI (directly responsible individual) for each quality aspect. Ideally, you are looking for a teammate interested in the area that you are trying to improve and can help you identify metrics, generate ideas, and push to get things done. 

You can consistently achieve more with the support of your team than you can alone. 🚀

By grouping metrics into different lines and columns, you can collapse and expand the results depending on which level you are interested in. 

If you want to dive deep and understand why the exact value of each quality aspect you are looking at got this number and what slices does it have, expand again.

Finally, if you are interested in all metrics, expand once more.

Tip 💡 Don’t be afraid of excel formulas. Some of them might become quite long, but the logic is simple:

• Check that the value does not step out of the boundaries
• Convert all values to one scale to unify all metrics and draw comparisons
• Calculate the average metric representing each quality aspect area

What’s next? 

Do not forget to iterate. Review metrics and put new values every OKR (once per 3 months).

Make some changes to each metric to understand the impact of each metric in relation to the final quality aspect evaluation score.

If you are interested in this idea and want to know more details, email me at asta@zyro.com. 

I would be glad to guide you through the concept, discuss this topic, and find your feedback.

If you liked the idea and want to try it – be creative, innovate and invent your variant of Quality Aspect Heatmap