In this lesson, let's run through a few common examples of A/B tests that have worked for other brands, and can help you drive higher engagement. In particular, these examples will showcase how to:
- Test email designs (and use personalized campaign AI).
- Refine high-impact SMS flow copy.
- Optimize your sign-up form display timing.
Test new email designs
A common use case for A/B testing is comparing how 2 different visuals catch a recipient's attention and spur them to click. Let’s examine how an online pet supply company determined what to test, how they ran their test, and what outcome it resulted in.
Here’s how this brand approached its campaign testing strategy:
- Problem: Last month’s benchmarks are in, showing that recent email campaign click rates have performed as Fair. They’d now like to increase clicks going forward, landing in the Excellent or Good categories.
- Hypothesis: They believe that more zoomed-in product images can improve click rate.
- A/B test: They created 2 variations of the content for a new leash product launch campaign. The first variation shows a picture zoomed in on the leash, while the other variation is a zoomed-out image focusing more on the dog itself.
- Results: Their hypothesis was correct! High-resolution images of the leash up close spurred more clicks. They take note for the next product launch, and plan to test this strategy again for next week’s summer sale campaign to see if the results carry in a different themed email send.
Experiment with high-impact flows
Our friends at the pet supply company are now facing a new obstacle: their SMS flow messages are not seeing the click rates they’d like. This is particularly true for their SMS welcome flow, which introduces new subscribers to their brand and text communication.
Here’s how an A/B test can improve that flow:
- Problem: Low SMS click rate for their SMS welcome series.
- Hypothesis: They think adding an image to their text, making it an MMS, will improve their flow performance.
- A/B test: They create 2 variations of the first SMS message, adding an image to 1 variation and leaving the other as is.
- Results: The click rate for their test did not alter dramatically and their results were not statistically significant. As a result, they take that as a sign that the image does not spur more clicks, so they scrap that idea and start over. They form a new hypothesis: perhaps they need to test the offer within their SMS to add a greater incentive to click. They’ll start a new experiment to see if this drives impact.
Show your sign-up form at the right time
At the end of the year, the pet supply company determines that a goal for the new year is to gain more new marketing subscribers. While they feel their sign-up form content is relatively successful, they aren’t positive that it is appearing to website visitors at the best time.
Here’s how they conduct their form A/B test:
- Problem: They do not feel confident in the display time settings for their main website pop-up.
- Hypothesis: They believe finding the optimal time for their form to appear will lead to more sign-ups.
- A/B test: They launch a form optimization A/B test. In this test, Klaviyo uses AI to automatically find the best time for their pop-up to appear for website browsers, taking the guesswork out of their form display settings.
- Results: When this test concludes, Klaviyo automatically updates its pop-up to appear using the winning display timing settings. They review the finalized test results and are excited to already see a 15% lift in form submissions.