One experiment, start to finish. The sticky add to cart bar.
Product behaviour described here was checked against the shipped code on 7 September 2026. Plan tiers reflect the published pricing page on the same date.
Vendor demos show you a dashboard. This shows you an experiment: the brief a CRO team would actually write, the code that implements it, the targeting and goals underneath it, the checks before and after launch, and the specific places where this gets built wrong. The pattern is deliberately ordinary, because an ordinary experiment is the honest test of a platform.
1. The brief
This is the experiment as a CRO team would write it up, before anyone opens a testing tool. Every row below has to be answered by the platform, and the rows that platforms usually make hard are the last three.
| Field | The brief |
|---|---|
| Observation | On mobile product pages the add to cart button scrolls out of view as soon as a visitor reads the description or the reviews. Getting back to it costs a deliberate scroll up. |
| Hypothesis | Keeping price and an add to cart button persistently in view on mobile product pages will increase the rate at which visitors add an item to the cart. |
| Change | A fixed bar at the bottom of the viewport carrying the current price and an add to cart button that submits the same form as the page's own button. |
| Who sees it | Mobile visitors on product detail pages only. Not tablet, not desktop, not category or search pages. |
| Primary goal | Add to cart. Counted once per visitor, in both arms, on the same action. |
| Secondary goals | Reached checkout, and revenue per visitor. |
| Guardrails | The two goals that attach to every experiment automatically, plus page speed on mobile. |
| Split | 50 / 50, with the whole of the qualifying audience entering the test. |
| Duration | Decided before launch from the baseline add to cart rate and the smallest effect worth acting on. Not decided by watching the dashboard. |
That last row is the one worth arguing about. Run the numbers in the A/B test calculator before you build anything: if the page cannot produce enough traffic to detect the effect you care about, the most faithful implementation in the world will not save the experiment. No platform can fix that, and any platform that offers you a duration without asking for a baseline rate and a minimum detectable effect is guessing.
2. The variation, in code
ABTestly is code first, so a variation is JavaScript and CSS you write and review in a Monaco editor. There is no visual editor recording your clicks into generated selectors. That trade is the whole product decision: it excludes people who do not write code, and in exchange the variation is a thing you can read, diff, and paste into your own repository.
1/* Variation 1 - sticky add to cart, mobile product pages */ 2/* ABTESTLY_EXP_ID is bound for you in the variation scope. */ 3window.abtestly.onApply(ABTESTLY_EXP_ID, function () { 4 var BAR_ID = 'abx-sticky-atc'; 5 if (document.getElementById(BAR_ID)) return; 6 7 // waitFor polls for the element, then gives up. The PDP renders 8 // client side, so the button is not there on first paint. 9 window.abtestly.waitFor('[data-pdp-atc]', function (atc) { 10 if (!atc || document.getElementById(BAR_ID)) return; 11 12 var price = document.querySelector('[data-pdp-price]'); 13 var bar = document.createElement('div'); 14 bar.id = BAR_ID; 15 bar.setAttribute('role', 'region'); 16 bar.setAttribute('aria-label', 'Add to cart'); 17 18 var amount = document.createElement('span'); 19 amount.className = 'abx-sticky-price'; 20 amount.textContent = price ? price.textContent.trim() : ''; 21 22 var button = document.createElement('button'); 23 button.type = 'button'; 24 button.className = 'abx-sticky-btn'; 25 // The SAME hook the goal matches. See section 4. 26 button.setAttribute('data-pdp-atc', ''); 27 button.textContent = 'Add to cart'; 28 button.addEventListener('click', function () { atc.click(); }); 29 30 bar.appendChild(amount); 31 bar.appendChild(button); 32 document.body.appendChild(bar); 33 }); 34}); 35 36// Runs on route change in a single page app, and on teardown. 37window.abtestly.onCleanup(ABTESTLY_EXP_ID, function () { 38 var bar = document.getElementById('abx-sticky-atc'); 39 if (bar) bar.remove(); 40});
Six decisions in that file are worth naming, because each one is a bug that a visual editor would have hidden from you:
- The insertion is guarded twice. Once before waitFor and once inside its callback. On a single page app the apply cycle can run again on a route change, and an unguarded append gives you two sticky bars stacked on top of each other.
- The button delegates to the page's own button rather than reimplementing the add to cart request. Reimplementing it means owning the cart payload, the variant id, the stock check and the error handling, and getting any of those wrong turns a conversion experiment into an outage.
- Nothing is built from an HTML string. Elements are created and text is set with textContent, so a product title containing an apostrophe or an angle bracket cannot break the markup or inject anything.
- The hooks are data attributes, not structural selectors. A selector like .pdp > div:nth-child(3) button survives exactly until the next front end deploy, and then fails silently while the experiment keeps collecting.
- There is a cleanup callback. Without one, the bar from the previous product page persists across a client side route change and shows the wrong price.
- The new button carries the same hook as the original. That is not tidiness. It is what makes the measurement valid, and section 4 explains why.
1#abx-sticky-atc { 2 position: fixed; left: 0; right: 0; bottom: 0; 3 /* Above the page, below a modal. Not 2147483647. */ 4 z-index: 40; 5 display: flex; align-items: center; gap: 12px; 6 /* Clears the iOS home indicator. */ 7 padding: 10px 14px calc(10px + env(safe-area-inset-bottom)); 8 background: #fff; 9 border-top: 1px solid rgba(0, 0, 0, 0.12); 10} 11.abx-sticky-price { font-weight: 600; white-space: nowrap; } 12/* 44px is the minimum comfortable tap target. */ 13.abx-sticky-btn { flex: 1; min-height: 44px; border-radius: 8px; }
The z-index and the safe area inset are the two lines that decide whether this ships or gets rolled back on the first morning. A bar at z-index: 2147483647 covers your own cookie banner and your own cart drawer; a bar without the inset sits under the iOS home indicator and loses roughly a thumb's width of its tap target on every recent iPhone.
3. Targeting: who qualifies
Targeting in ABTestly is authored as saved Locations (where a test may run) and saved Audiences (who is eligible), then attached to the experiment. They are reusable objects rather than rules retyped per experiment, which matters the third time you build a mobile only test.
| Requirement | Type | Rule |
|---|---|---|
| Product pages only | Location | URL contains /products/. New URL rules default to ignoring case, so /Products/ matches too. |
| Mobile only | Audience | Device equals Mobile. Desktop and Tablet are separate values, so a tablet does not quietly count as mobile. |
| Exclude the team | Audience | Optional. A cookie rule, or the built in visitor opt out at ?abtestly_optout=1, which excludes that browser from every experiment and every beacon on the site. |
Both rule types here are on every plan. Device, browser, operating system, language, referrer, the UTM parameters, day of week, hour of day, cookies and a raw JavaScript condition are all available as audience rules, and URL, URL with query, query string, hostname and a JavaScript condition as location rules.
One note on targeting, because it is the sort of thing a walkthrough should not paper over: the geo rules, country, region and city alike, need a Pro plan. Everything this experiment actually uses, the URL location and the device audience, is on every plan, so the build above runs on Starter.
4. Goals, and the trap that invalidates this experiment
This is the section that decides whether the whole exercise produces a number worth acting on, and it is where the sticky bar pattern is most often built wrong.
| Goal | Type | How it is defined |
|---|---|---|
| Add to cart (primary) | Click | Matches [data-pdp-atc], which both the page's own button and the sticky button carry. One definition, both arms, counted once per visitor. |
| Reached checkout | Page visit | A URL rule for the checkout step. Catches the case where the bar lifts add to cart but nothing reaches the till. |
| Revenue per visitor | Revenue | Reported with a confidence interval, and with outlier control available so a single very large order cannot decide the result. |
| Increase Engagement | System default | Attaches automatically. Fires on a click that resolves to a link, once per page. |
| Engaged session | System default | Attaches automatically. Qualifies on a second page view in the session or ten seconds on the page, whichever comes first. |
The two system goals are attached to every new experiment and cannot be detached. They are there so that an experiment always carries at least two behavioural guardrails, including on the entry plan, rather than only the metric you were hoping to move. If the sticky bar lifts add to cart while engagement collapses, you want that visible in the same panel rather than discovered a month later.
If a goal has to fire on something that is not a click or a page view, the runtime exposes a call you make from your own code, which is the honest answer for a cart that updates over a network request:
1// After your cart request actually succeeds, not on the click. 2window.__abtestly.trackGoal('add_to_cart', { value: 49.00 });
5. Traffic split
Two variants at 50 percent each, with 100 percent of the qualifying audience entering the test. The dashboard shows these as percentage sliders and tells you plainly whether the variants add up to 100; underneath, weights and traffic allocation are stored in basis points, so 5000 is 50 percent and the split cannot drift through rounding.
Bucketing is deterministic on a stable visitor id, so a visitor who qualifies sees the same arm on every subsequent visit. Worth knowing before you launch: a visitor bucketed into the control arm counts toward your tracked user allowance even though the page did not change for them. That is how the meter works everywhere, and the terms page states it rather than leaving you to find out from an invoice.
6. QA before launch
Nothing here is live yet. Two mechanisms carry this stage.
Preview links, on your real page
- A link that forces one specific variant on your live site, of the form ?abtestly_preview=<experiment>:<variant>.
- A hash form is supported for single page app routers that reject unknown query parameters.
- A draft or paused experiment needs a signed token alongside it; the token is scrubbed out of the address bar once the session starts, so a pasted URL does not leak it.
- An on page pill shows the live status, the registered goals and whether each has fired, which is how you confirm the goal matches the sticky button without waiting for data.
The variation code check
- Runs at save time. JavaScript is parsed with acorn, CSS with css-tree, against roughly seventy rules.
- Catches, among others, building HTML from strings, a missing cleanup callback, an uncleared interval, a fragile selector, an unguarded insertion, and an extreme z-index.
- It is warn only. It does not block a publish, and we are not going to claim it does.
- Issues persist across saves, keyed on the experiment, the variant and the rule, so an unresolved one stays visible until a later save stops producing it. Suggestions are shown once and not stored.
Check both variants on a real device, not only in a desktop browser's responsive mode. The safe area inset, the tap target and the interaction between a fixed bar and the mobile browser's own collapsing toolbar are three things that look fine in a simulator and wrong in a hand.
7. QA after launch, in the first hour
The first hour after starting an experiment is when instrumentation bugs are cheap to fix. Four checks, in order:
- Do the right visitors qualify? Exposures should be arriving from mobile product pages and nowhere else. Desktop exposures mean the audience is not attached, or is attached to the wrong experiment.
- Does exposure fire only after consent? If you operate under GDPR the snippet must be wired to your consent platform. An exposure recorded before consent is a compliance problem, not a data problem.
- Does the goal fire on the action you defined? Add to cart from the sticky bar and from the original button, in both arms, and confirm both register.
- Is the split arriving as configured? A 50 / 50 experiment that delivers 56 / 44 has a delivery bug, and every conclusion drawn from it is unsound.
That last one is checked for you. When the arrival ratio departs far enough from the configured weights, the results panel raises a sample ratio mismatch banner reading Sample ratio mismatch detected: results may be invalid, with a per variant observed and expected table. It flags at a chi square p value below 0.001 and needs at least 500 distinct users before it will say anything, so early noise does not produce a false alarm. Below that floor it reports that it is still collecting rather than showing a reassuring green tick it has not earned.
8. Reading the result
Frequentist statistics are on every plan and are the default engine. Two further engines exist and, on the published pricing page, sit on Pro and above:
| Engine | What it reports | When it suits this experiment |
|---|---|---|
| Frequentist | Significance on conversion rate, with a confidence interval. | The default. Fix the duration in advance and read it at the end. |
| Sequential | A procedure built to let you look at the data repeatedly without inflating the false positive rate. | When people will look at the dashboard daily regardless of what you asked them to do. Which is to say, usually. |
| Bayesian | A chance to win, an expected loss, and a credible interval on the difference against control. | When the decision is commercial rather than academic. A credible interval is not a significance test, and we do not describe it as one. |
Both of the additional engines apply to revenue as well as conversion. With three or more variants the leader reads optimistically under the Bayesian view, and the product says so on screen rather than leaving you to remember it.
One guardrail specific to this experiment: a fixed bar is new DOM on every product page, so it is worth watching what it does to loading performance. On Pro and above, the speed guardrail reports per variant largest contentful paint at the 75th percentile, split by mobile and desktop, as a band rather than a falsely precise midpoint. It needs 100 page views per variant per device class before it reports anything, and says nothing at all until then.
What a technical team can compare
This is the point of walking through one experiment rather than reading a feature grid. After building this once, in your own tool and then in ABTestly, you can compare the following from direct evidence.
Directly comparable
- Time to build. How long the same change took to express in each tool, including the QA loop, not just the first draft.
- Whether the variation says what it does. Explicit JavaScript and CSS you can review and diff, against generated selectors you cannot.
- What happens on a route change. Whether the tool gives you an apply and cleanup lifecycle, or whether you hand roll one.
- The goal definition model. Whether one definition can span both arms, or whether the tool pushes you toward the trap in section 4.
- The pre launch checking loop. Forced variant links on your real page, and what the tool tells you before any visitor is bucketed.
- What the results panel refuses to say. Whether it flags a bad split, and whether it admits when it is too early to decide.
- The price at your tracked user volume, against your renewal quote, for the scope you actually use.
Not comparable, and we will not pretend otherwise
- A head to head conversion rate race. Run this in two tools at once and some visitors land in both, some in neither, and two scripts fight over the same DOM. The numbers stop meaning anything. A clean split has to be made upstream of both tools, on a stable visitor id.
- An exact numbers match. Even done properly, two tools will not agree to the decimal, for legitimate reasons: different identity, attribution, consent handling and goal definitions. Treat any cross tool check as a plumbing check, not a scoreboard.
- Your history. Nothing imports from any other vendor, and no tool can import it honestly. Export before you cancel.
- Whether the sticky bar wins. That depends on your traffic and your customers, and no walkthrough can tell you.
Where this experiment would not suit ABTestly
Three variants of this brief that we would tell you to build somewhere else. Each is a real product limit, not a feature waiting on a roadmap slide.
- If you want to test the bar's colour, copy and position as a factorial grid, that is multivariate testing and ABTestly does not do it. Test the version you believe in against control.
- If the bar should only appear for a personalised segment computed on your server, that is a personalization campaign or a server side experiment, and this is a browser runtime.
- If the person who owns this test does not write code, they cannot author or safely review that variation file, and no amount of tooling around it changes that. That is the deliberate trade this product makes.
Common questions
Is this a real experiment or a mock up?
It is a real experiment pattern, written out with real ABTestly configuration and real variation code, on a deliberately generic product page. No customer site, traffic figure or result is shown, because none of that is ours to publish. The parts that are specific to ABTestly, which is the point of the page, are exact.
Why does the sticky button carry the same data attribute as the original?
So one goal definition matches the add to cart action in both arms. If the goal only matched the new sticky button, the control arm could never convert on it, and the experiment would measure whether the bar exists rather than whether it helps. This is the single most common way a sticky element test is built wrong.
Which parts of this need a Pro plan?
The build itself runs on Starter: the variation code, URL and device targeting, the click goal, the preview links and frequentist statistics are all core. Sequential and Bayesian statistics, experiment heatmaps and the per variant speed guardrail are Pro capabilities on the published pricing page. Check the pricing page for the current tier of any capability before you rely on it.
Does the variation code check block a publish?
No. The check reports issues and suggestions at save time and they are warn only, so nothing stops you publishing code the check dislikes. Issues persist across saves, keyed on the experiment, the variant and the rule, so an unresolved one stays visible until a later save no longer produces it. Suggestions are shown once and not stored.
How long should an experiment like this run?
That is set by your conversion rate and the effect size you want to detect, not by the tool. Put your own numbers into the A/B test calculator before you launch. A tool that promises a duration without those two inputs is guessing.
Checked on 7 September 2026 against the shipped code. Product behaviour changes, so verify before you rely on a detail. Plans, trial terms and the tracked user definition: abtestly.com/pricing. What an evaluation costs: abtestly.com/evaluate-one-experiment. The longer form of the assisted offer: abtestly.com/switch-one-experiment. Runtime API, install and consent wiring: docs.abtestly.com.
ABTestly is an independent A/B testing platform for CRO developers and technical teams. Convert, VWO, Optimizely, AB Tasty and PostHog are trademarks of their respective owners. ABTestly is not affiliated with, endorsed by or sponsored by any of them.