UX Research Case Study:

Usability Testing of a Platform Supporting Ethical Supply Chains

Background

In late 2016, a coalition of environmental and human rights organizations began work on an initiative to support companies in achieving ethical supply chains. The coalition consisted of major players like WWF, National Wildlife Federation, Rainforest Alliance, and Nature Conservancy.  They called their effort the Accountability Framework initiative (AFi).

Underpinning the initiative were a few realities:

• Commodity supply chains are a major driver of many of today’s biggest environmental and social problems, from climate change to human exploitation.
• Hundreds of global companies have pledged to transform their agriculture and forestry supply chains to become more ethical.
• Turning those pledges into results, however, is challenging.
• Companies have struggled to answer and agree on core questions: What commitments should we make? What do our commitments actually mean? How do we take actions do achieve them? How do we monitor and demonstrate progress?

By 2019, the group launched a new digital platform for the Accountability Framework. The platform provided an interactive set of principles, definitions, and guidance to companies.

Fast forward to 2021. Forbes declared that:

“Two years on and the AFi is now everywhere. Some of the biggest companies in the world, such as McDonald’s and Nestlé, are now using the Framework. … Most major assessments of corporate performance on deforestation and related issues are now aligned with the Framework.”

Goals

Back in the fall of 2019, however, the future of the Framework’s digital platform was less promising.

While many companies were trying the new platform, the AFi team believed that it was too hard to use — and that its user experience barriers were hurting adoption and engagement.

AFi partnered with Marketade to conduct usability testing of the platform and to identify user experience problems, gaps, and opportunities.

AFi’s specific research questions included:

• What do users expect this site and tool to provide for them?
• What are users’ specific goals when arriving to the site?
• How do users navigate through the journey tool to meet their goals? What works well and what are pain points?
• Can users find the key information they need to succeed on their journey to establishing a more ethical supply chain journey?

The AFi team needed to complete the study by the end of 2019.

Approach, Part 1: Research

Marketade conducted moderated usability research with 7 participants whose work primarily involves managing supply activities.

The goals of the research were to capture users’ general understanding of how to use this site and the journey tool. We also gathered feedback on users‘ ability to complete key tasks, including:

• Setting commitments and benchmarking against existing company policies.
• Finding definitions and guidance on specific topics.
• Understanding how to credibly report progress.

During the sessions, participants:

• Told us about their work and how they normally begin engagements in supply chain analysis;
• Performed key tasks on the site (finding key definitions, setting commitments, monitoring progress, and demonstrating progress to stakeholders);
• Explained how they address smallholder inclusion and broader challenges outside of the supply chain and looked for information to support those scenarios on the site;
• Commented on the general challenges of their work in regard to creating ethical and responsible supply chains.

Approach, Part 2: Workshop

After completing the research, we facilitated a one-day workshop with the AFi team to build a shared understanding of how people used the site and to identify the key barriers for users of the platform. During the workshop, we led AFi through a collaborative analysis of the research to identify where users experienced frustration or confusion on the site.

During the one-day workshop, the AFi team defined actionable next steps to address top-prioritized issues, including reorganizing the site and reconsidering how to display information.

* * *

Workshop participants spent the morning viewing recordings of these sessions. As they watched, they recorded their observations on individual sticky notes.

Following viewing the user interviews, the team was led through a collaborative research analysis session, ending with brainstorming solutions to top-prioritized issues.

Below is an outline of the process we followed:

• Participants individually took notes as we viewed user sessions, using one sticky note per observation.
• They posted their observations/notes on a wall and then worked to categorize the insights into logical groups.
• Each participant voted on what they felt was the most critical category to address in solutions brainstorming.
• The team discussed the groups, and evaluated any that could and should be combined. We identified 4 top-priority categories, moved out-of-scope topics to a parking lot, and solved “quick win items” in under 2 minutes. We noted next steps for those solutions to the “quick win items” in a chart.
• Two working groups were formed to delve into the top four findings. Each group took two of the priority categories and defined the problem, supporting the problem with actual observations, listed solutions, and identified action items.
• Each team presented their findings to the whole group, and then engaged in a brief discussion.
• The group discussed specific action items and next steps.

Outputs

The AFi team walked away from the project with:

• 4 “quick win” findings and 1-2 solution ideas for each finding — along with owners and deadlines
• 4 high-priority findings and multiple solution ideas for each
• 10 next-step action items to support the solution ideas — along with owners and deadlines

Here are a few examples of insights from the research:

• Users preferred to download the PDFs rather than navigate the site to find the information they needed.
• Users found the site search tool helpful, but still struggled to find key definitions and understand what tools the AFi provides.
• Users sometimes even created their own benchmarking tools based on the information they found in the PDFs.

After the project, AFi stakeholders shared the following feedback:

“The guided problem analysis really managed to pull out core issues and potential solutions. The workshop facilitation triggered so much thinking — I felt really engaged.”
— Nadine Syarief | Program Associate

“The workshop was great! It allowed for analysis and discussion on next steps.”
— Melissa Lazaro | Communications Associate

“Great process – very effective and we came out with a great set of solutions.”
— Adrienne Stork | Sr. Program Manager

“While it will clearly mean a lot of effort to make this work, there’s no doubt these changes will lead us (iteratively) towards ensuring the Framework delivers value to our audience. Effective and clear process. Clear next steps.”
— Ryan Thompson | Communications Manager

​More Case Studies

Iterative Usability Testing of NFPA’s Web Tool for Building Engineers

A global nonprofit was struggling to get on the same page about how to improve its new web app. NFPA partnered with Marketade to conduct 2 rounds of usability testing and on-site workshops, with rapid design changes in between.

UX Audit for Search Inside Yourself Leadership Institute

The marketing team at this national nonprofit was eager to increase B2B leads for its enterprise training service. With the help of our UX and content audit recommendations, they saw a 100% increase in leads and a 500% lift in hand-raisers.