Skip to main content
CRO

Most A/B tests fail because the research stage was skipped

The short answer
A conversion test is only as good as the hypothesis behind it. Research — analytics review, session recordings, heatmaps, user interviews and form analytics — should precede any experiment, because the constraint is usually not the element teams instinctively want to test.

R. Nakamura

Head of Conversion

6 min

A conversion test is only as good as the hypothesis behind it. Research — analytics review, session recordings, heatmaps, user interviews and form analytics — should precede any experiment, because the constraint is usually not the element teams instinctively want to test.

Why this comes up so often

In India, this question tends to surface at the point where a business has outgrown its first marketing setup but has not yet built the measurement discipline to know what to change. The instinct is to add another channel. The more useful move is usually to establish a defensible baseline first.

What good looks like

  • Every claim states its timeframe, baseline and attribution method
  • Decisions are documented, including the ones that turned out wrong
  • Leading indicators are reported between the lagging ones
  • Underperformance is raised the month it happens, not at quarter end

Where to start

Document what you currently believe to be true, then check it. In our experience roughly a third of what a business assumes about its own funnel does not survive that exercise — and finding out early is considerably cheaper than finding out after a year of optimising the wrong constraint.

Frequently asked

How do we apply this in our own business?

Start by documenting your current baseline before changing anything. Most of the value in this approach comes from being able to prove what changed and why.

CRO

Apply this to your own situation

We will work out which part of this is actually relevant to your constraint — and which part is not.