AI Agents · Experiments & A/B
Test variants; keep the winner.
Try more than one version of a campaign — the subject line, the copy, the creative, the offer, the channel or the send time. The platform splits your audience, measures real member response, and keeps the winner once there's enough response to be confident. And a small holdout proves the lift is real.
What you can test
Test the part of the campaign that matters most.
An experiment can be as simple as two subject lines or as meaningful as two different offers. Pick what you want to learn, and let real member response answer the question.
Subject line
Try two ways of opening the conversation and see which one earns the open — without guessing which tone your members prefer.
Copy
Test a short, punchy message against a warmer, longer one, and let real response decide which voice moves people to act.
Creative
Compare hero images, layouts or a bold banner versus a clean look, and keep the design that actually gets clicks.
Offer
Weigh 20% off against a gift-with-purchase, or points versus a coupon — find the incentive members respond to, not the one you assume.
Channel
See whether email, app push or the inbox reaches a given audience best, and lean into the channel that lands.
Send timing
Test a morning send against an evening one to find the moment your members are most likely to respond.
Experiments live inside the normal build — in the campaign canvas, different branches can be delivered differently, so a test is just part of the flow, not a separate tool.
How it works
From two ideas to one proven winner.
You set up the variants; the platform handles the fair split, the honest measurement, and the call on which version won.
Define your variants
Pick what to test — subject, copy, creative, offer, channel or timing — and set up two or more versions of the campaign. It's part of the normal build, right in the canvas.
Split the audience
The platform automatically divides your audience across the variants so each version reaches a fair, comparable slice of members.
Measure real response
As sends go out, real member response is measured — opens, clicks, redemptions — counted honestly, never double-counted or inflated.
Keep the winner
Once there's enough response to be confident, the winning variant is selected — so you scale what works instead of calling it early on a hunch.
The experiment
Two variants, one confident winner.
Variant A
WinnerMembers responded more
Variant B
Fewer responses
Prove it's real
A holdout proves the win is real, not noise.
Picking a winner tells you which variant did better. A holdout tells you whether the campaign did anything at all — because some members buy anyway.
A small group, deliberately left untouched
A holdout is a slice of your audience that gets nothing — no send, no offer. If the members who received the campaign respond more than the untouched group, the difference is incremental lift: real value the campaign caused, not sales that would have happened anyway.
The same rigor behind SoClaw's win-back proof
This is exactly how our autonomous win-back agent proves its worth: it only claims the lift it can show against an untouched control. So the number you report to the business is honest — it's the value you added, measured the same trustworthy way.
See how SoClaw proves win-back lift →Every experiment feeds the bigger picture.
Winners, response and lift roll up into honest measurement that makes your next campaign smarter. See the full measurement story on Campaign Intelligence.
Explore Campaign Intelligence →