Ongoing Research Program Case Study:

Building a UX Research Program for an Enterprise Service Application

Overview

Years ago, a GEICO team wanted to improve the user experience of the internal web application used by its call center support reps. With thousands of reps helping millions of customers, support task efficiency was a key driver of both business KPIs and customer satisfaction.

Many companies invest heavily in the UX of their customer-facing software yet ignore the usability of employee-facing systems.

The GEICO team was ahead of the curve. They recognized the potential ROI of a redesigned support app built on a deep understanding of the reps’ tasks, needs, and pain points.

The company’s digital experience team partnered with Marketade to launch and operationalize an ongoing user research program for its call center software. Together we built from scratch a repeatable research program that has:

• Transformed the user experience of call center staff
• Shown a strong ROI and become a point of pride for the company
• Continually expanded and inspired similar research programs

Process

Here’s an overview of the key steps we followed to create this program.

1. Discovery & Pilots

We started with a discovery research process that included stakeholder interviews, industry research, and most importantly, call center field visits. At the call centers, we observed and interviewed 20+ service reps, helpdesk staff, supervisors, trainers, and more — providing critical data for the following steps.

Based on what we learned, we quickly launched a few pilot usability studies with reps. We captured key learnings about recruiting, moderating, and working with the design and development teams.

2. Research Cycle Design

By now, we’d learned enough to design the first version of a repeatable, 2-week testing process — one that mapped to the scrum team’s agile development cycle. The 10-day cycle looked like this:

• 3 days to prototype, design the research, and recruit participants
• 3 days to test (primarily qualitative testing of prototypes supplemented by quantitative testing)
• 3 days to analyze the research, identify problems, generate solution ideas, and collaboratively iterate on those solutions
• 1 day to present the findings and changes to stakeholders

3. Program Design

We also designed and planned an initial version of the larger research program, mapping out:

• Research topics and methods for the first 2 months
• A repeatable process for topic and method selection going forward
• A target number of program participants
• Key stakeholders and management check-ins
• Program KPIs and measurement methods
• A detailed rollout schedule

4. Phased Rollout

• With an MVP and early feedback in hand, we shifted our focus to building support from critical stakeholders — from senior management to call center supervisors to scrum team managers.
• We identified a couple of call centers for a full-blown pilot — testing the end-to-end process multiple times, gaining recruiting practice, and building ground-level support. We then prepared for a nationwide launch.
• After getting buy-in from management on a program name and brand identity, we built an internal marketing website that explained the program and benefits to reps and featured an application form. We also designed and wrote recruiting email templates and built out an email communication process. Then we launched the program across all call centers.

5. Panel Building

We coordinated with call center management to send out program invitations to curated lists of reps. We also built a process for capturing key attributes of program applicants (job role, tenure, etc.) — and used these as screening criteria to select a representative panel that would support different types of studies. With this process, we recruited well over 1,000 participants into the panel.

6. Story Sharing

A few months into the program, we had settled into a steady cadence and had completed over 10 studies. To accelerate program recruitment, we needed stronger support from management across all call centers.

We decided that the best way to do this was to show the impact this research was having in concrete ways. We designed a case study template for the marketing site. For each study completed to date, we produced a story that showed what we tested, learned, and changed. We then promoted the case studies to call center managers.

7. Constant Iteration

As the program matured and surpassed KPI targets, we maintained a focus on continuous experimentation and evolution. Improvements included:

• Shifting from prototype testing to live environment testing
• Creating more realistic user tasks by modeling them on real call flows
• Expanding the research cycle to match changes in the development cycle
• Transitioning panel recruitment and maintenance to new team members

Results

UX Transformation

In 3 years, we completed over 60 research studies through this program — across a wide range of software features/flows and supported by a diverse set of research methods. Even high-quality, recurring UX research often gets forgotten in a report and fails to drive change. By creating a highly collaborative research process that syncs with the company’s agile development cycle, we’ve helped the UX team achieve a high implementation rate for research study recommendations.

Corporate Visibility

The program received praise throughout the company, from executives to HR. It was a finalist for a company-wide quality award and was promoted in public-facing employee recruitment materials.

New Units, New Programs

As word has spread, new business units have approached the UX team about rolling this program out to their applications and teams. We’ve helped to launch spinoff programs for other call center applications. The program has also served as a poster child and a starting framework for new research programs in other product areas.

Business ROI

Best of all, the UX team has shown a return from the program that resonates with metrics-focused executives: significant cost savings from faster and more productive call center reps, tied back to research-driven usability improvements.