A campaign lands in your lap and the numbers are flat. The button says "Submit." The page is three paragraphs of feature copy above the fold. You have one experiment slot this sprint, one shared traffic budget, and two candidate changes that both look obviously wrong.
So which one do you ship first?
The useful answer is neither "test both" nor a coin flip. The two changes answer different questions, and one of them is usually wasted if you run it in the wrong order.
The two problems look identical in a dashboard
Both failures surface as the same line: campaign traffic arrives, signups don't happen. To separate them, split the funnel into two questions.
- Does the page earn attention? Can a first-time visitor understand the offer within the first screen?
- Does the ask earn commitment? Once the value is clear, is the action obvious, specific, and low-friction?
The first question is about the landing page. The second is about the call to action. A dashboard that only tracks sessions and signups cannot tell you which one broke.
A diagnostic table before you write any code
What the data showsLikely causeWhere to startMost sessions end within a few seconds; low scroll depthThe page does not match the promise in the adLanding pageDeep scroll, pricing or docs section viewed, few form startsThe ask is vague, buried, or costlyCall to actionMany form starts, few completionsForm frictionNeither. Fix the formLow engagement and low action, traffic source unclearTargeting or message mismatch upstreamNeither. Fix the campaign
That last row matters more than it looks. If the audience arriving from the campaign was never a fit, no amount of button rewording or hero-copy polishing will rescue the page. Fixing the targeting is cheaper than any experiment.
Why the order matters more than the individual change
A call to action is read in the context of the headline, the proof, the price, and the surrounding copy.
That produces two ordering rules.
Rule one: a button cannot fix a page that never establishes value. If visitors leave before they understand what they are being asked to do, a sharper label has nothing to attach to. You will measure noise and conclude that labels do not matter.
Rule two: do not optimize a component you are about to replace. If a page rewrite is already on the roadmap for this quarter, running a careful label experiment on the current page is a throwaway. The result will not transfer to the new layout, because the context that produced it no longer exists.
The second rule is the one teams break most often, because label experiments are fast to ship and feel productive.
Do the math before you commit to either
A small change usually produces a small effect, and a small effect demands a large sample. A page rewrite can plausibly shift behavior by several points; a button label change often moves it by a fraction of one.
A quick calculation, runnable in any Python environment, shows how steep that difference is:
import math
def sessions_needed(p_control, p_treatment, alpha=0.05, power=0.80):
z_a = 1.959963985
z_b = 0.8416212336
p_bar = (p_control + p_treatment) / 2
spread = (z_a * math.sqrt(2 * p_bar * (1 - p_bar))
+ z_b * math.sqrt(p_control * (1 - p_control) + p_treatment * (1 - p_treatment))) ** 2
print(math.ceil(spread / (p_treatment - p_control) ** 2))
sessions_needed(0.04, 0.045) # a 0.5-point lift on a 4% baseline
sessions_needed(0.04, 0.06) # a 2-point lift on the same baseline
On a 4% baseline, detecting a 0.5-point improvement at 80% power takes roughly 25,000 sessions per arm. Detecting a 2-point improvement takes roughly 1,900. Same campaign, same page, a thirteen-fold difference in required traffic.
Two conclusions follow. First, if your campaign delivers a few thousand sessions a month, a micro-optimization of the button label is probably not measurable, no matter how good the label is. Second, the size of the expected effect should drive the choice of experiment far more than how easy the change is to ship.
A default order that holds up in practice
- Verify the instrumentation. One exposure event per session, assignment logged server-side rather than inferred from a query parameter, internal traffic excluded, bots filtered. Every downstream conclusion inherits the quality of this step.
- Check message match. Put the ad headline next to the page headline. If they describe different things, stop here.
- If message match fails, run the landing page experiment first. Change the hero, the offer framing, or the proof: the elements that determine whether a visitor stays long enough to be asked anything.
- If message match holds and engagement looks healthy, run the call-to-action experiment. Label, placement, and the one or two lines of supporting copy next to it.
- Re-run the call-to-action experiment after any page rewrite. Treat the old result as historical, not as a default.
Step 4 is where a lot of teams want to start, and it is a legitimate starting point, but only when the diagnostic in the table above says the page is already doing its job.
Track what happens after the action, not just the action
A call to action that increases signups while filling the pipeline with people who never come back is a net loss. Before you run either experiment, decide on guardrail events and log them alongside the primary metric:
// One exposure event per session, emitted once and cached client-side
trackEvent("experiment_exposure", {
experiment_id: "cta_copy_q3",
arm: "specific-verb",
campaign_id: "launch_aug",
});
// Downstream quality signals, tied to the same session
// activation_started, first_key_action, repeat_usage_7d
If the primary metric moves up and a guardrail moves down, that is a trade-off that needs a decision. Guardrails are especially important for landing page experiments, because a page that over-promises will inflate the action rate and damage downstream quality.
Do not change both at once
Running a page experiment and a call-to-action experiment simultaneously on the same traffic makes the result unreadable. You cannot tell whether the change came from the page, the ask, or the interaction between them.
If you genuinely need both answers and have the traffic to support it, use a factorial design: four arms, so every page version is paired with every call to action. The interaction term is often the most interesting number in the output, because it tells you whether a strong page makes the label irrelevant. The cost is roughly four times the traffic of a two-arm experiment.
For most campaigns, sequential experiments are the practical choice: page first, then the ask, with each experiment consuming a pre-committed block of traffic. Avoid switching between them mid-run based on how the numbers look.
Link hygiene when campaign traffic is split
If you are routing campaign traffic between page builds and call-to-action experiments, the link you publish becomes part of the workflow. Campaign copy already competes for space with a headline and a button, and long parameter-heavy URLs wrap badly in chat apps and social posts. A short link such as LinkBit keeps the destination compact and readable in the creative. That is a formatting convenience. Your experiment assignment should live in your own instrumentation, not in the link.
The short answer
Run the landing page experiment first when the funnel shows that visitors never engage, when message match is broken, or when a rewrite is already planned. Run the call to action first when the page is doing its job and the ask is the weak link, and only when your traffic is large enough to detect the smaller effect you are likely to see.
Before either, confirm what your instrumentation measures. Most experiment programs that stall are running careful changes on a funnel they cannot see.#experimentation #conversionrate #landingpages #webanalytics #software #coding #development #engineering #inclusive #community
Should You A/B Test Your Call to Action or Your Landing Page First?
