Rapid Iterative Content Testing for a Complex GEICO Product

Goals

Insurance is a hard product to understand in general, but certain types of insurance are harder than others. Think term/whole life insurance, disability insurance, umbrella insurance, cyber liability insurance, or identity protection.

A few years back, a GEICO team picked one of these types of insurance and launched an initiative to improve its digital experience and business KPIs. Anecdotal evidence and spot research suggested that many online consumers were confused about what the product provided, why someone might need it, and how it worked — if they had even heard of it.

A major thrust of the project was to tackle these awareness and comprehension challenges through better digital content across multiple channels. Greater visibility and consumer understanding could improve the customer experience as well as key business metrics such as qualified quotes, online conversions, and self-service.

GEICO partnered with Marketade to lead content strategy and research for this initiative.

Part 1: Discovery

While no team had conducted content or usability testing for this product, there was some useful past data scattered across the company. We started by reviewing and analyzing what we could find:

• Sales call recordings
• Interviews with users who requested quotes but did not buy policies
• An online survey of recent purchasers

We also interviewed 3 GEICO phone counselors for this product line — 2 sales reps and 1 service rep. Each of these reps had talked with hundreds of customers and potential customers of this product.

This combination of discovery stage research gave us a great starting picture of the product’s key personas and their common scenarios, questions, and pain points. Now we were ready to pull in GEICO stakeholders.

Part 2: Alignment

We designed and facilitated a 2-hour workshop with GEICO stakeholders from it product, IT, UX, CX, and communications teams. We led the team through a series of steps:

• We brainstormed project goals and, through a voting exercise, picked the 3 most important goals.
• We presented findings highlights from our discovery research review and showed 20 minutes of clips from our phone rep interviews.
• Workshop participants took notes through the lens of the 3 goals identified earlier.
• After reviewing the research, the team participated in an affinity mapping exercise to reach consensus on the biggest barriers and opportunities.
• We aligned on 4 top barriers, brainstormed solutions for each barrier, and voted on the most impactful solution ideas.

This workshop gave us the focus areas and guidance we needed to move forward with content changes and testing. It also helped different teams within the organization get on the same page, which was critical given the number of channels that we’d touch later on.

Part 3: Content Changes

Based on the workshop outputs, we worked with GEICO to identify 5 channels (e.g. mobile app) that were good candidates for content improvements.

We met separately with stakeholders from each of these channels to discuss relevant solution ideas generated in the workshop and to explore others. For each channel, we picked the most promising page or section or tile to change.

Then our content expert went to work. In some cases, we made major changes; in others, we made tweaks.

At the same time, we began recruiting participants for an initial phase of moderated content testing. We used recruitment criteria established earlier in the project to create campaign messaging and a 13-question screener survey.

We recruited candidates via Craigslist ads in 3 metro areas and through social media. Out of 500 responses, 22 candidates qualified based on their responses. We selected the 10 who were most qualified.

We split the participants into 2 groups so that we could run 2 rounds of testing.

Part 4: Iterative Testing

Our goals for this phase of research were to:

• Learn which new content elements are effective — and which ones are ineffective — at educating consumers about the product and helping them to learn if it was a good fit for them
• Quickly make changes to the content based on problems observed with the 1st batch of users, and test those revisions with the 2nd batch.

Interviews were remote, one-on-one, and about 30 minutes each. We showed participants draft or test versions of content from the 5 channels and then asked questions.

We conducted our 1st batch of sessions on a Tuesday afternoon. The 2nd batch was scheduled for Wednesday morning — giving us just a few hours to make changes. The keys to pulling off this rapid iteration:

• We put the content in Google Docs, allowing for easy and fast revisions
• Our content writer observed all of the sessions in real-time
• Our research moderator and content writer compared notes at end of the sessions and quickly aligned on the problems to address

The 2nd batch of testing showed further improvement in some areas while continuing to expose some problems.

We made additional changes on Thursday based on what we learned from the 2nd batch.

On Friday, we delivered a 4-page report that summarized our testing process, findings, final content changes for this phase, and next steps.

Outcomes

This cycle was our 1st phase of content testing and improvements for this product. We would follow it with two more phases that followed a similar approach of iterative testing and changes.

This 1st phase led to A/B tests in 2 of the 5 channels, and before-vs-after tests in the other 3 channels.

By the end of the project, GEICO saw meaningful improvements in key metrics across these channels. More importantly, they found a scalable process to make cross-channel content improvements. In the months and years to come, we would team up with GEICO to repeat this same process for a number of other products.

About the Project

• Industry: Financial services
• Audience type: Consumers
• Research area: UX
• Methods: Content testing
• Length: 3 weeks (for this phase)
• Primary stakeholder: Product and UX teams
• Company size: Over 25,000 employees

​More Case Studies

Prototype UX Testing of a New McDonald’s Mobile Ordering Flow

A McDonald’s product innovation team was redesigning a key app experience. To uncover friction points prior to rollout, McDonald’s partnered with Marketade on 2 rounds of UX testing with rapid design changes in between.

Moderated Usability Testing Unlocks Problems in a Bank Account Onboarding Flow

A bank’s design team had redesigned its savings account onboarding flow. Through qualitative and quantitative prototype testing, we identified the top UX friction points and generated 40 improvement ideas.

How a Content Audit Helped Travel Leaders Improve its Web Experience

A business unit of one of the travel industry’s largest service companies was eager to improve the content on its flagship website. We conducted a rigorous content audit with quantitative data, qualitative assessment, and page-by-page recommendations.

Additional Project Resources