AI can do the busywork of an experiment: draft a new version, pick a sensible goal, even keep testing on its own. You decide how much it may do. And Barry, the assistant in Studio, can explain results and set experiments up with you.
Choose how much AI does
In the setup step AI help (or later under Settings), pick an Autopilot mode:
| Mode | What happens |
|---|---|
| Off | You build versions and launch. You can still ask AI for a version at any time. |
| Suggest versions | AI drafts a version on its own. You review it and launch. |
| Full autopilot | AI drafts versions and launches them within the limits you set. It can also publish a proven winner, if you allow that. |
Decide what AI may change
Under What AI may change, tick the areas AI may touch. One area starts ticked, such as Wording for quizzes. AI cannot draft anything until at least one area is ticked.
| Surface | Areas you can allow |
|---|---|
| Quizzes | Wording, Layout and design, Questions, Question paths, Results page, Product matching |
| Search | Recommendation strategy, Ranking, Rules, Presentation, Quiz suggestions, Banners |
| Popups | Wording, Layout and design, Screens, Calls to action, Forms, Products |
| Collection pages | Wording, Layout and design, Ranking |
| Recommendations | Recommendation strategy, Ranking, Rules, Wording, Layout and design |
Some things always stay protected, whatever you tick: consent, which products are eligible, stock rules, required quiz questions and form fields. Keep specific items unchanged lets you lock individual questions, sections, audiences or even the whole setup, so AI leaves them and everything inside them alone.
Set the limits
- AI drafts per day: how often AI may draft a version, 1 to 4 times a day, whether you ask or autopilot does. Failed attempts count too.
- Publish a proven winner automatically: only on Full autopilot. When the evidence is clear and the safety checks pass, the winner goes live for all shoppers, not only those in the experiment.
- Start a new experiment after publishing: with autopilot on, the published version becomes the new original and the next round starts from there. Stop optimizing on a finished experiment ends the cycle, and Open the next experiment takes you to the round that followed.
After launch, goals and traffic are locked, but you can still change what AI may do with Settings → Change what AI may do.
Ask Barry
Open Barry from the editor you are working in and ask about your experiments in plain language. For example:
- “How is the shorter quiz test doing?”
- “Does this quiz have enough traffic for a test?”
- “Set up a test for a popup headline that leads with the discount.”
Barry reads the same results you see and explains them without jargon. It tells you honestly when a result is not proven yet, and when a quiz has so little traffic that a test would never finish, it suggests making the change directly instead.
When Barry proposes a change, such as creating an experiment or launching it, nothing happens until you click Apply. The only exception is what you already allowed in the experiment's autopilot settings. Barry never widens those permissions on its own. More about Barry in Meet Barry.
Tips
- Start with Suggest versions. You see what AI comes up with before any shopper does.
- Want to test one specific idea of your own? Keep autopilot Off and build that version yourself, or describe it under Ask AI for a version. See Build the versions you want to test.