Nonprofit Case Study:

How Workshops Helped AARP Get Aligned on a KPI Measurement Model

Challenge: No Consensus on KPIs

A business unit at AARP had recently launched a high-profile digital platform targeting Americans 50 and older. The site offered a personalized set of life-management tools and programs to help Baby Boomers tackle their new life stage.

Over 1 million people were using the platform within its first year. The business unit was eager to increase adoption and engagement.

As the next calendar year approached, the team wanted to have key performance indicators (KPIs) for the platform and target metrics for each KPI. Yet they were having trouble reaching consensus internally and wanted an objective outsider to take a look at their business and help them identify KPIs that could withstand scrutiny from senior management.

Solution: Collaborative, Workshop-Based Process

AARP partnered with Marketade to help develop a KPI framework for the platform. We led 2 workshops with senior team members and reached alignment on a KPI model with business objectives, supporting web metrics, and target metrics by quarter for the upcoming calendar year.

Here’s the process that we used.

Getting Buy-In on Collaboration

First, we captured buy-in on using a collaborative, workshop-based process. We knew that if we went off and did our own research in a vacuum — even if it involved stakeholder interviews — and came back to them to present our recommended KPIs, the process would not work well. We’d have trouble getting buy-in and consensus if the AARP team was not part of the process. And because we wouldn’t be tapping into their context and expertise, we’d be more likely to pick the wrong KPIs.

Once we explained the collaborative process, the AARP team was excited about it and they were able to get the right people in the room for 2 sessions spread over 2 weeks. This group was eager to have a say.

Workshop #1

The 1st workshop was a 2-hour session and was the pivotal part of the project.

10 AARP stakeholders participated. Most were from the business unit running the new platform along with a couple of team members from central corporate offices who oversaw strategy. Specific roles from the business unit included the chief marketing officer, marketing managers, and the research/analytics manager.

We gathered around a table in a room that had the right supplies: lots of wall space, sticky notes of different colors, markers, etc. We started off by giving an overview of the process, which was a combination of Avinash Kaushik’s KPI measurement model process and Jared Spool and UIE’s affinity mapping process (aka KJ method).

Avinash’s approach provides the overall structure and goals (we used a lighter-weight version than what he uses), but it doesn’t tell you how to reach consensus with a diverse group of opinionated stakeholders. When we tried that in the past using traditional brainstorming and discussion techniques, it failed.

So we borrowed a page from our user research playbook, where after doing things like usability testing, we’d seen how effective affinity mapping was at reaching consensus and prioritizing the most important findings. This was one of the first times we applied affinity mapping to a KPI exercise.

* * *

First, we used affinity mapping to identify core business objectives for the platform, answering the question “why do you exist?”. We reached consensus on 3 goals pretty quickly.

Then we used affinity mapping to identify 1 or more KPIs for each business objective, answering the question “how will we measure progress against this objective?”. This part took a while because we had a huge volume of potential measures on sticky notes. As we often see, people have a hard time remembering the “Key” aspect of KPIs, and so we had a lot of metrics like visits and time on site that lacked outcome focus. But affinity mapping worked its magic, and because we’d stressed “outcomes” early and often, the group was able to reach tentative consensus on a solid list of KPIs.

The specific affinity mapping steps that we used in this case were:

• Brainstorm individually and add each idea you have (for a business objective or a KPI) to a sticky note
• Everyone adds their stickies to the wall at once (no names attached)
• Group the notes into thematic categories
• Name the groups on separate sticky notes above the groups (this part is always hard, but it is critical for getting people to process all the notes and groups)
• Vote on the top groups
• Order the top groups based on votes and identify the top 3

Nearly all of these steps were completed in silence. A big part of the reason affinity mapping works is that we’re avoiding a shouting match where loud voices and senior voices win. We were able to achieve this here, but it took a lot of reminding, including reminding some senior-level people.

If we hadn’t done this a lot before and realized how important this part is, we wouldn’t have been as disciplined and it would have devolved into the typical frustrating meeting. And there’s no way we would have reached consensus within 90 minutes.

Workshop #2

By the second session, the hard work was over.

We recapped where we’d arrived on business objectives and KPIs, and gave people an opportunity to raise questions and concerns. We discussed the large questions on the spot; for the small ones, we identified a plan to investigate them after.

Then we covered how to track and count the KPIs. Which data source or analytics tool? Was it set up to track this behavior already? If not, could it be?

Finally, we covered initial KPI targets. Since we didn’t have good baselines in place for many of the KPIs, we created a plan to quickly set up tracking and develop baselines so our targets would be better informed.

​More Case Studies

How a Website Survey Gave UVA Health Actionable UX Insights

How do you launch a website survey that captures actionable data without annoying users? Here’s the step-by-step process we used for a regional health system along with the keys to our survey’s success.

Brainstorming GEICO Content Improvements with Remote Research Workshops

A B2B product team wanted to increase its KPIs through better digital content. Marketade conducted 1:1 customer interviews and led 2 workshops to analyze the research and ideate practical content ideas.