Creating an A/B Test
Use experiments (the portal calls them A/B Test) to roll a configuration change out to a portion of traffic rather than all at once. Good for floor changes, timeout changes, new slots, new bidders — anything that could move revenue.
Before you begin
Define a clear hypothesis. A good hypothesis has:
- The change — what's in the variation
- The expected direction — what should happen
- The metric — how you'll measure it
Example:
"Raising the floor on sidebar slots from $0.75 to $1.25 will increase sidebar CPM by 20% with less than a 5% drop in fill rate."
How to create an A/B test
Step 1: Open the new-experiment page
- In Ad Manager (main) mode, confirm the right domain is selected.
- Go to Wrapper → Experiments.
- Click the A/B Test button (top-right).
Step 2: Setup tab — hypothesis and schedule
The new experiment opens on the Setup tab:
| Field | What to enter |
|---|---|
| Name (title field) | Short descriptive name, e.g. Sidebar floor test — Q2 2026 |
| Hypothesis | The hypothesis statement (what you're testing, expected direction, metric) |
| Start time | When the experiment begins |
| End time | When it's scheduled to stop |
Save at any point to create the experiment as a Draft. You can keep editing until you deploy it.
Step 3: Variations tab — control and tests
Switch to the Variations tab. The variations formset lets you define:
- Control — your current live configuration, captured when you save
- Variations — one or more test configurations
Each variation points at a source config (so you build the variation config in Wrapper → Configs, then reference it here). Typical setup for a single-change test:
- Control = current config
- Variation A = the same config with your one change applied
Step 4: Traffic allocation
Still on the Variations tab, the Traffic Allocation fieldset shows a progress bar. Set the traffic share for each variation:
| Split | When to use |
|---|---|
| 90 / 10 | Risky change — small exposure first |
| 80 / 20 | Moderate risk — more data, still low blast radius |
| 50 / 50 | Low risk — fastest to significance |
Shares must sum to 100% — the progress bar turns green when you're there.
Risky changes (floor hikes, bidder removals, new slots above the fold) deserve a 90/10 first pass. You can increase allocation after you see the variation isn't hurting metrics.
Step 5: Save and start the experiment
Starting an experiment is a two-step sequence — a "Create Release" from the experiment row, then a separate deploy of that release.
- Save the experiment. It stays in Draft — configuration is still editable, no traffic is affected yet.
- Return to Wrapper → Experiments and click the Create Release button (box icon) on the experiment row. This does two things:
- Creates a new Draft release that points at the experiment as its source
- Moves the experiment's status to RUNNING
- Go to Wrapper → Releases. Find the draft release you just created and Deploy it (rocket icon). Until this step, the variation is not actually serving on your site.
The experiment transitioning to RUNNING just means it's locked from further setup edits. Your variation only goes live once the backing release is deployed. If you skip the deploy step, the experiment sits idle.
Once deployed, the Setup and Variations tabs on the experiment form lock. Traffic allocation remains editable via a Manage Traffic button at the top of the experiment detail page.
Adjusting traffic while running
Running experiments expose a Manage Traffic button near the top of the experiment detail page:
- Open the running experiment.
- Click Manage Traffic.
- Set new shares for each variation (must sum to 100%).
- Add notes if helpful.
- Confirm.
Each change creates a new row in the Allocation History table, so you have a record of when you ramped up a variation and why.
Using the AI Assistant
The Assistant can scaffold experiments from natural language:
| You ask | What happens |
|---|---|
| "Create an A/B test for $1.50 floors on sidebar slots" | Drafts an experiment with the variation pre-built |
| "Ramp my floor test to 50/50" | Opens the Manage Traffic modal prefilled |
Review the drafted configuration before deploying — the Assistant is useful for the tedious scaffolding, not for deciding whether the change is a good idea.
Example experiments
Floor price test
Name: Floor price optimization
Hypothesis: Higher floor on sidebar slots improves blended CPM
Control: current config (sidebar floor $0.75)
Variation: current config (sidebar floor $1.25)
Split: 90 / 10
Duration: 14 days
Timeout reduction
Name: Timeout reduction
Hypothesis: 1200 ms timeout preserves revenue while improving LCP
Control: current config (1500 ms bid timeout)
Variation: current config (1200 ms bid timeout)
Split: 80 / 20
Duration: 7 days
New in-content slot
Name: Third in-content slot
Hypothesis: Adding a third in-content slot lifts revenue without hurting CWV
Control: current article layout (2 in-content)
Variation: article layout with 3 in-content
Split: 90 / 10
Duration: 14 days
Troubleshooting
Experiment won't deploy
| Check | Fix |
|---|---|
| Traffic allocation sums to 100%? | Adjust shares on the Variations tab |
| Start time in the past? | Push start time forward if relevant |
| Source config valid? | Open the variation's source config and confirm it saves cleanly |
No traffic on variation
| Cause | Fix |
|---|---|
| Release not deployed | Click Create Release from the experiment list |
| Full-page cache serving old HTML | Purge your CDN cache |
| Wrapper not on the page | Verify the tag is present on page source |
Common questions
How long should I run an experiment?
At minimum 7 days (captures a full weekday/weekend cycle). For most publishers, 14 days is the sweet spot. Very-low-traffic sites may need longer to see stable numbers.
Can I run multiple experiments at once?
Technically yes. Watch for:
- Overlapping changes — avoid, since you can't attribute the result cleanly
- Stacked traffic shares — a 10% variation of one experiment intersected with a 10% variation of another is only 1% of traffic
- Interaction effects — two changes together may behave differently than either alone
How do I stop a running experiment?
Archive it from Wrapper → Experiments (the archive action in the row). Traffic returns to the control configuration on the next release.