A/B Testing Popups
Popup A/B tests let you run two or more variants of a Popup simultaneously to find which performs better. Turnjo splits traffic between variants, tracks conversions, and can automatically conclude the test when a winner is found.
How it works
-
In a Popup's editor, open the A/B test section and click Create A/B Test.
-
Name the test and choose what it's judged on:
- Became a lead (the default)
- Clicked a button or link
- Reached a particular screen
- Finished — reached the end of the popup
The goal can't be changed once the test starts.
-
Check How long will it take? (below). Then set the stopping rules: confidence, minimum improvement (default 20%) and maximum duration (default 30 days).
-
Add variants and set their traffic weights. The original Popup content becomes the reference variant.
-
Click Edit on a variant to design it on the canvas — the same live preview you use for the Popup itself. A variant changes only the first screen and the theme; form fields and every later screen are shared with the live Popup.
-
Click Start Test.
Variant edits are saved with an explicit Save variant button (no autosave) — while a test is running, a content change is a deliberate act, and you can optionally reset the variant's data at the same time if the change was meaningful enough to invalidate earlier results.
How long will it take?
The New A/B Test form estimates how many visitors each variant needs to find the minimum improvement you've set, and how many days that is at your Popup's traffic over the last two weeks. It starts from your current rate for the chosen goal, which you can change.
If the estimate is longer than the maximum duration, the form tells you the smallest lift the test could find in time. Small lifts need a lot of traffic: at 200 visitors a day and a 5% rate, finding a 20% lift takes about 100 days, while a 40% lift takes about a month. Testing a bolder change, such as a new offer or headline, is usually the quickest way to a result.
Stopping rules
Turnjo checks running tests every 15 minutes. It concludes a test automatically when:
- A winner is found — a variant beats the reference at your confidence level (default 95%) and improves on it by at least your minimum improvement (default 20%). Each side needs at least 100 visitors first.
- Maximum duration is reached — the test has run for N days without a winner.
Because results are checked so often, the bar a variant must clear starts high and falls as the test approaches its maximum duration. With several variants it's higher for each one. This keeps the chance of a false winner at your confidence level, however often the results are checked and however many variants you test.
You can also pause or conclude a test manually at any time.
Promoting a winner
When a test concludes, open the A/B test section in the Popup's editor and click Promote Winner. The winning variant's blocks and style replace the Popup's default content and the test is archived.
Viewing results
Open View full results from the A/B test section. For each variant you'll see:
- Visitors — each person counted once, however many times they saw the Popup.
- Converted — visitors who reached the test's goal. For leads, duplicates of an earlier lead aren't counted.
- Rate and uplift over the reference, with a likely range — for example "+18% (likely −4% to +40%)". While the range includes zero, the result isn't clear yet.
- The z-score and the z-score needed to win right now.
If the split between variants is further from their weights than chance would explain, the page warns you. That usually means something other than the test is deciding who's counted, such as a cached page or bots, and the results should be treated with caution.
Tests started before visitors were counted show impressions instead.
The Analytics page for a Popup shows each variant's impressions, leads and rate for the current date range.
Traffic splitting
Variants are assigned to visitors randomly at impression time using the configured weights. The assignment is sticky — a visitor who sees variant B will always see variant B on that property (stored in the browser's localStorage).
More docs
Getting Started
Popups
Surveys
Guided Tours
Experiments
Visitor Feedback
Integrations
Team & Account
Billing & Plans
Privacy & Compliance