A/B experiments let you test two versions of a step, such as a cancellation benefits page, a retention offer, or a reward journey, to see which one performs better before you roll it out to everyone. Subscribers are split into a control group (A) and a variant group (B), and we track key metrics for each group so you can compare results with data instead of guesswork. This article covers what A/B experiments are, where you can run them, and how to review and monitor results.
What are A/B experiments
A/B experiments show two different versions of the same step to different subscriber groups and measure which one performs better.
Control group (A): your current experience.
Variant group (B): an alternative experience you want to test.
When a subscriber enters a flow that has an experiment running, they are randomly assigned to either group A or group B based on the audience split you set. Once assigned, a subscriber always sees the same group until the experiment ends or you stop it. We track the key experiment metrics for each group and use your winning criteria to highlight the best-performing variant.
Where you can use A/B experiments
A/B experiments are available in two areas of Loop:
Cancellation flows
Subscription benefits page: test different content, videos, and calls to action on the page subscribers see before they cancel.
Retention offers: test different offer types and values, such as a gift against a discount.
Reward journey
Test different milestone rewards and the way each reward is communicated to subscribers.
If you are new to these modules, read Benefits page, Retention offers, or Reward journey before setting up an experiment.
Identifying where an experiment is running
Each experiment now displays the exact Loop module it is associated with, for example Cancellation benefit, Retention offers, or Reward journey. This makes it possible to tell at a glance where an experiment is running without opening its configuration.
Viewing experiment performance
We provide a dedicated place to track all your experiments.
Experiments list
Navigate to Loop admin > Tools & apps > Experiments.
Here you'll see every experiment across all modules, along with its current status (Active, Completed, Stopped), the module it belongs to, start date, and completion rule.
Experiment details page
Click any experiment to open its details page. The details page now includes the full experiment setup, so you no longer need to guess how an experiment was configured or lose visibility into a variant once the experiment ends.
The details page shows:
Experiment configuration: primary metric, audience split, duration, and winner calculation.
Control and variant groups: which subscribers were placed in each group.
Content for each variant: the complete content or design configured for both control (A) and variant (B).
Primary metric and all tracked metrics for each group, with day-on-day trends.
Start and end dates, so you can correlate results with other business events.
Which group, if any, is currently marked as leading or winner based on your winning criteria.
Previously, once an experiment concluded, you could only view the winning variant. Both the winning and losing variants now remain accessible, so you can compare experiences, review past tests, and document learnings.
Note: Experiment configuration is available only for experiments created on or after July 28, 2026.
Exporting experiment results
From the experiment details page, you can export raw data for your own analysis, including subscription IDs, which variant each subscriber was shown, and the outcome for that subscriber. You can combine this export with your own analytics, ESP, or BI tools for deeper analysis.
What happens when an experiment completes
Once an experiment completes, whether automatically or because you stopped it manually, control group (A) content starts showing to all eligible subscribers for that step. Subscribers no longer see variant B unless you explicitly choose to continue with it.
For 7 days after an experiment completes, the relevant configuration page still shows both the control (A) and variant (B) configuration, and you can select which variant to continue with as your long-term experience. After those 7 days, if you have not chosen a variant:
Subscribers continue seeing control (A) content.
You only see the control (A) configuration in the admin.
Content from variant (B) is no longer visible in the UI.
You can then continue using control (A), or create a new experiment to test a fresh set of hypotheses.
FAQs
How are subscribers assigned to control (A) or variant (B)?
We use your audience split, for example 50/50 or 70/30, and randomly assign eligible subscribers to either group when they reach that step. Once a subscriber is assigned to a group, their assignment is sticky: they always see the same group on subsequent attempts while the experiment is active.
Can I stop an experiment before it finishes?
Yes, you can manually stop an active experiment from its details page. Stopping an experiment marks it as Completed / Stopped, makes control (A) the active experience shown to all subscribers for that step, and keeps the results collected up to that point available in reporting and exports.
Can I edit the content of a variant once the experiment is live?
No, you can update the content of control (A) or variant (B), once the experiment is live. We recommend finalizing copy and creatives before starting the experiment. If you want to test a significantly different idea, complete or stop the current experiment and start a new one with the updated concepts.
Need help?
No worries - we're here for you!
If you have any questions or need assistance, feel free to email us at [email protected] or chat with us using the support beacon at the bottom right of your screen.
Regards,
Loop subscriptions team 🙂