Skip to main content
Run an A/B test on any page of your site and keep the version that converts best. A/B testing with Experiments splits the visitors to one page between the original and up to 4 new versions, counts how many of each group reach your goal, and tells you when one version has won. Visitors keep the same URL, so your ads and links don’t change.

​
Before you begin

  • Publish your site. Visitors are only split on your published site. Version pages must be published before the experiment can launch, and Replo offers to publish them for you at launch.
  • Check that results can be counted. The Replo pixel records which version each visitor saw and what they did next. If the A/B tests tab in Insights shows A/B tests need the Replo pixel, finish the pixel step in the Insights Setup tab first.

​
Set up an experiment

1

Open Experiments

In the Site Builder, open the page you want to test. Click Settings in the top bar and, under This page, choose Run an experiment. The setup opens in its own tab with that page already picked.To start from the list of all experiments instead, choose Experiments under This site in the same menu, then click Create experiment.If the page is already in a draft or live experiment, the menu item reads View experiment and opens that experiment.
2

Pick the page to test

Under What do you want to test?, search for the page and select it. It becomes the Original.A page can be in only one draft or live experiment at a time. Pages that already are show In an experiment; click one to open its experiment.For a dynamic page template, choose whether to test every page it renders or pick up to 1,000 of them. The pages you don’t pick keep the original.
3

Make version B

On the Version B card, choose how to make it:
  • Copy the original: Replo duplicates the page. Under What should version B change?, describe the change (optional). After you create the experiment, Replo makes that change in chat, on the copy only.
  • Use another page: pick a page you already built.
To test more than one change, click Add version C. An experiment holds the original plus up to 4 versions (B to E).
4

Name the experiment

Experiment name fills in as “[Page] experiment”. Change it to something you’ll recognize in the list later.
5

Set the traffic split

The Traffic split card gives every version an even share by default. Type a percentage for each version (1 to 99) or click Split evenly to reset.Once a goal is set, the card estimates how long the test will take, for example About 14 days to a result at your traffic. The version with the smallest share sets the pace, so an uneven split takes longer.
The shares must add up to 100%. If they don’t, the card shows These add up to 90%. Make them 100%. and the experiment can’t be created.
6

Choose the goal

Under Goal, pick what counts as a conversion:
  • Visits a page: then pick the page next to Counts when a visitor reaches, for example a thank-you page.
  • Clicks a button
  • Adds to cart
  • Starts checkout
  • Buys
Every version is judged on this one goal.
7

Write a hypothesis (optional)

In Hypothesis, note what you expect and why, for example “If we move the bundle higher, more visitors will add it to their cart.” It shows on the experiment’s Goal card later.
8

Create and launch

The main button reads Launch when every version is already a published page and your site is ready to split traffic. Otherwise it reads Create experiment, which saves a draft and opens it. If the button is greyed out, the reason shows next to it, for example Pick the page that counts as a conversion.In the draft, click Launch. If a version isn’t published yet, the button reads Publish B and launch (or Publish versions and launch): Replo publishes your site, then launches when the publish finishes.
If you asked Replo to change a copied version, check the change in chat before you launch. Until Replo finishes, that version is an exact copy of the original, and the test compares two identical pages. Open it from the Versions tab with Edit page.
What success looks like: the experiment shows a Live status, and a message confirms Visitors see it in a minute or two.

​
How visitors are split

Each visitor is assigned a version using your traffic split. They keep that version on every visit, and the address bar shows the page’s normal URL while they see their version. Copied versions live under /v/ and are hidden from search engines. To see a version the way visitors do, open the experiment’s Versions tab, open the menu on a version’s card, and choose Preview on your live site. Preview visits are left out of results.

​
Read the results

Open the experiment from Settings > Experiments. A live or ended experiment opens on its Results tab, which shows:
  • Leader, Goal, Lift and Chance to win at the top.
  • A table with each version’s Visitors, Conversions, Conversion rate and Chance to win.
  • A banner that says what to do next, such as Gathering data or Version B won, so you can end this experiment.
Click Open in Insights to open the same report in Insights, or Write up results to have Replo add a written summary to it. Every experiment is also listed in the A/B tests tab in Insights.

​
How a winner is called

Numbers show from the first visitor, but Replo doesn’t call a result until every version has at least 100 visitors and 10 conversions and the experiment has run for 7 days. Until then the banner says Gathering data and estimates how long is left. After that:
  • A version wins when it has at least a 95% chance of beating the original.
  • The original wins when every other version has a 5% chance or less.
  • If neither happens, the banner tells you to keep it running. If detecting a 10% lift would take more than 60 days at your traffic, it says No clear difference, so you can end this experiment: try a bolder change.

​
End an experiment

1

Click End experiment

On a live experiment, click End experiment.
2

Pick the version to keep

Select the version every visitor sees from now on. If you keep a version other than the original, visitors to the page’s URL get that version.
3

Record the outcome

Outcome fills in for you: “Version B won” when you keep a version, No clear winner when you keep the original. Change it to The original won or Stopped early if that fits better, and add a note under What did you learn?. Then click End experiment.
The experiment moves to the Ended tab of the list. To test the same versions again, open its menu and choose Run again.
You can delete a draft or an ended experiment, but not a live one. Deleting an ended experiment where you kept a version other than the original sends visitors back to the original.

​
Troubleshooting

Hover the button to see why. Version B’s page is missing means the version’s page was deleted: create it again or delete the draft. Version B will be deleted at your next publish means the page is set to be removed: restore it first.
The experiment saved, but visitors aren’t split until it’s connected. Click Try again on the message.
The publish that launch started failed. Fix the publish (see Publishing), then click launch again. If you see Your site is already publishing, wait for that publish to finish and launch again.
Visitors aren’t being split while a version has no live page. Create the page again or end the experiment.
Replo has no traffic for this page to estimate from, so the card shows no time estimate. It doesn’t block the experiment; you can still create and launch it.

​
Next steps

Insights

See every A/B test with your other reports.

Publishing

Publish your site so versions can go live.

Act on Your Analytics

Decide what to test next from your traffic and sales.