Understand A/B testing best practices
A/B testing, also referred to as split testing, allows you to compare different versions of your marketing content against each other to see which performs best.
It involves building specific content variants (e.g., Variant A and Variant B) and then showing them to different portions of your audience at the same time, measuring which version drives higher conversions. Variant A is typically the control, or the “original version” of the content, while Variant B includes a change to 1 specific element (e.g., the main image, background color, or call-to-action (CTA) text). That way, you can quickly gather data around what resonates best with your customer base.
Before you begin testing content, review the video below to follow our best practices for A/B testing.
Determine what to test for each channel
Now that you understand a few best practices for A/B testing, let’s discuss which elements are commonly tested across marketing channels. While these are not the only things you can test, they are a great starting point if you are new to A/B testing.
Website
Elements to test:
- Sign-up form calls-to-action (CTA)
- Sign-up form timing and display types
- Headlines
- Navigation menu
- Product images
- Page layout
- Product pricing structures
- Button colors
Elements to test:
- Subject line
- Imagery
- CTA button text
- Type of discount
- Colors and fonts
- Design layout
Text messages
Elements to test:
- Images (SMS vs. MMS)
- CTA text
- Type of discount
- Emoji use
- Message length
Social media
Elements to test:
- Graphics
- CTA buttons or text
- Ad types
- Ad placement
- Lead magnets
- Type of discount
Follow key steps for A/B testing
Let’s walk through a few core steps to get an A/B test up and running. Explore each step in the dropdowns below.
Identify your testing goal
First, define a goal for your A/B test so that you know what success actually looks like. Having a goal lets you build a “hypothesis"; in other words, what you think you can change to help you achieve this milestone.
Here's how to identify a clear A/B testing goal:
- Align with your current business objectives
How will your A/B test support larger business initiatives, like increasing revenue or improving the customer experience? Tie your goal directly to relevant KPIs like attributed revenue, conversion rate, or engagement rates. - Solve for current problems or opportunities
Identify areas that need improvement (e.g., cart abandonment, high bounce rates, etc.) or support new opportunities you have uncovered (e.g., better ways to showcase new products).
For example, your goal may be to increase email click rates by 5% to drive higher conversions attributed to email campaigns and flows.
Choose 1 element to test
Your next step is to identify 1 element of your marketing content to test. Choosing just 1 makes it easier for you to attribute positive or negative results to a specific change to your marketing content.
For example, if your goal is to improve email click rates, then you may choose to test your CTA button. You historically use the copy: “Shop now” so you keep that as Variation A. You make a second Variation B with the copy “Take 10% off”, for the purpose of testing these 2 CTAs against each other.
Set up your test variations
It is best practice to create anywhere from 2 to 4 content variations; that said, as you get started, test just 2 variations at once to get used to this process.
As we discussed previously, Variant A is typically the control, or the “original version” of the content, while Variant B includes a change to 1 specific element of the content. It’s crucial to have a control in place, so that you can verify that a change to your current strategy outperforms your usual content and is worth investing in further.
Run your test
Your next question will inevitably be: How do I conduct an A/B test for each channel?
The answer depends on what platform you're using and what type of message you’re testing. In Klaviyo, you can test elements of your marketing, like email or text campaigns, automated flows, and sign-up forms. That said, you can also test your webpages or paid ads on the platforms that you use to power those tools.
To access the best information for your use case, here are quick links to documentation on how to A/B test common marketing content on popular platforms:
- How to A/B test an email campaign in Klaviyo
- How to A/B test a text message campaign in Klaviyo
- How to A/B test your website with:
- How to A/B test your social media content with:
Analyze and iterate
The message channel, platform, and type will also affect what “good” looks like for A/B test results. Use the goals you mapped at the start of this process, as well as the insights and benchmarks that the tools (linked in the dropdown above) share on their brand sites.
Based on what you find, your final steps are to:
- Determine your winner (which variation outperformed the other?)
- Identify if this was statistically significant (do you have enough data and a large enough test audience to confidently say this test variation is best?)
- Improve your future content (what lessons can you take away from this test for any future content you build via this channel and content type?)