A/B experiments for personalized upsell profiles let you run two upsell setups on the same profile at the same time, so you can see which one drives more add-ons from recurring orders. Subscribers are split into a control group (A) and a variant group (B), and we compare their upsell performance so you can pick a winner on actual results instead of guessing which discount, product mix, or banner works best.
This article covers what you can test, popular use cases, how to set up an experiment, and the metrics tracked.
Upsell profile experiments use the same Experiments list, creation flow, and results page as cancellation flows and reward journey. For what A/B experiments are and how to review results, see A/B experiments.
What can be tested
Control (A) and variant (B) can differ in any of the following:
Recommendation type: a list of specific items, best selling products, or smart recommendations.
"See more" behavior: show only the recommended items, or show all items from the general upsell profile.
Recommended items list: the items themselves, along with their one-time and subscription discount configuration.
Customer portal upsell banner: an uploaded image or custom HTML.
Popular use cases
Discount depth:
Control gives 10% off and variant gives 20%, either as a one-time add-on discount or a subscription discount. Many brands over-discount by default, so if the 10% version converts about as well, you keep the extra margin.
Primary metric: Conversion or Revenue added.
Intro discount vs flat discount:
Control gives a flat 10% on every order. Variant gives 20% for the first couple of orders, then drops to 10% using the Change discount after specific number of orders option. This tests whether a strong hook up front beats a smaller discount that never changes.
Primary metric: Revenue added. Revenue realised also shows whether the extra discount pays off over time.
One-time vs subscription add-on:
Control offers "One-time only" and variant offers "Subscription only". Subscription add-ons keep earning on every future order, so if subscription-only converts nearly as well, you earn more recurring revenue per customer.
Primary metric: Avg. upsell value per subscription.
Hand-picked vs automated recommendations:
Control shows a curated list of products. Variant shows best selling products or smart recommendations, and you can also add recommendation rules on the automated side. This shows whether your own curation beats the algorithm.
Primary metric: Items added or Revenue added.
Upsell banner copy:
Test an offer-led message against a benefit-led one, or a banner against no banner.
Primary metric: Conversion.
Setting up an A/B experiment for a personalized upsell profile
Navigate to Loop admin > Grow > Personalized upsell profiles, open the profile where you want to run the experiment, then click + Create an experiment.
Fill in the experiment details:
Experiment name: required.
Audience split: the percentage of eligible subscribers assigned to control (A) and variant (B). The default is 50/50.
Group tag and Subscription attribute: at least one of these fields is required.
Experiment duration: set it by time in days, by unique subscription count, or by views.
Metrics to track, Primary metric, and Winner threshold: select the metrics to measure for each group, choose one as the primary metric, and set the threshold for declaring a winner. All metrics are defined in the next section.
Save the experiment. The profile splits into side-by-side Control (A) and Variant (B) editors, in the same section order as the regular editor. Configure what you want to test in each one.
Save the profile. If the profile is in Draft, saving only creates the experiment, and the experiment goes live on the next save once the profile is Active.
Navigate to Loop admin > Tools & apps > Experiments, then open the experiment to track results.
Metrics tracked in upsell profile experiments
We track the following metrics for each group so you can compare performance.
Metric | What it means |
Conversion | Items added divided by the number of times the profile was served, multiplied by 100. |
Revenue added | Revenue from upsell items in recurring orders, excluding refunds. |
Items added | Number of items added through the upsell in recurring orders. |
Avg. upsell value per subscription | Average upsell revenue per subscription, counting only orders with at least one upsell item. |
Revenue realised | Revenue realized from the upsell items added. This is a lagging metric, so it can't be set as the primary metric. |
Experiment completion
An experiment completes automatically once it reaches the experiment duration you set, whether that is a number of days, a unique subscription count, or a number of views. You can also stop it manually from its details page.
Once an experiment completes, control (A) starts showing to all eligible subscribers. For 7 days afterward, the configuration page still shows both control (A) and variant (B), so you can choose which one to continue with.
After that, only control (A) remains visible in the admin unless you chose variant (B). For the full behavior, see A/B experiments.
Considerations
Set at least one of Group tag or Subscription attribute when you create the experiment.
Saving while the profile is in Draft creates the experiment only. It goes live on the next save once the profile is Active.
Revenue realised can't be the primary metric.
FAQs
Can I create an experiment while the profile is in Draft?
Yes, saving a Draft profile creates the experiment without starting it. The experiment goes live on the next save once the profile is Active.
Can I test the customer portal upsell banner in an experiment?
Yes, you can test an uploaded image or custom HTML banner as part of control (A) and variant (B), including a banner against no banner.
Need help?
No worries - we're here for you!
If you have any questions or need assistance, feel free to email us at [email protected] or chat with us using the support beacon at the bottom right of your screen.
Regards,
Loop subscriptions team 🙂