A/B testing (split testing) is a controlled experiment in which two versions of a webpage, email, or ad — identical except for one changed element — are shown to separate portions of your audience simultaneously. Version A is the original (control); version B is the variation. Traffic is split randomly between the two, and the version that achieves a higher conversion rate on the primary metric wins. A/B testing removes opinion from design and marketing decisions, replacing gut feel with evidence from your actual audience.
A/B testing is the most reliable method for improving website performance because it controls for external variables — both versions run simultaneously in the same market conditions, removing seasonal effects, traffic quality changes, and other confounding factors that plague before/after comparisons. A change that appears to improve conversions by 30% in a before/after analysis might be entirely explained by seasonal demand variation; a properly run A/B test isolates the change's effect.
How to run a valid A/B test
- Define one primary metric — the conversion event you are optimising (purchases, form submissions, clicks, sign-ups)
- Change only one element — the headline, the CTA button colour, the hero image, the form length; multiple changes make results uninterpretable
- Calculate the required sample size — use a statistical significance calculator to determine how many visitors each variant needs before you can trust the result (typically 1,000–5,000 per variant)
- Run the test until statistical significance — do not stop early because one version is winning; wait until the significance threshold (typically 95%) is reached
- Implement and document — roll out the winner, record the learning for future tests
- Plan your next test — CRO is continuous; what is the next-highest-priority conversion barrier to address?
What to A/B test on your website
- Headlines — the single element with the most influence on whether visitors stay or leave
- CTA button copy and colour — 'Get started free' vs. 'Start your free trial'; button colour relative to page background (contrast drives clicks)
- Hero images — lifestyle images vs. product images vs. illustrated graphics; people outperform objects in most consumer contexts
- Form length — removing non-essential fields; every additional field reduces form submission rate
- Social proof placement — testimonials above vs. below the CTA; specific vs. generic testimonials
- Pricing presentation — monthly vs. annual default; price anchoring; emphasising value vs. features
- Page length — long-form with full details vs. short-form with key points linking to more information
A/B tests should run until statistical significance is achieved (typically 95% confidence), not for a predetermined time period. The required duration depends on your traffic volume and the conversion rate uplift you're detecting — higher traffic and larger differences reach significance faster. A rough guide: tests on pages with 500+ daily visitors can reach significance in 1–4 weeks; pages with 100–500 daily visitors may require 4–8 weeks. Never stop a test early because the winning version appears obvious — early leaders frequently reverse as more data accumulates.
Free and low-cost tools: Google Optimize has been deprecated, but Microsoft Clarity (free) offers basic A/B testing. Paid tools with more functionality: VWO (Visual Website Optimizer) starts at around £150/month, Optimizely for enterprise, AB Tasty, and Convert are all well-regarded. For e-commerce on Shopify, the platform's built-in theme section testing and apps like Intelligems cover product page and pricing tests specifically. For most UK SMEs, VWO or Convert provide the right balance of capability and cost.