Video Ad Creative Testing on Amazon DSP
The instinct when a video campaign underperforms is to blame the creative first. Our own internal practice runs the opposite order for a delivering-but-not-converting line item: check the URL/placement report first, then bids, then creative, then the pixel — because creative is expensive and slow to iterate on, and it's usually not where the actual problem lives.
What this looks like in a real account
Why creative is the wrong first lever
There are two distinct DSP failure modes, and they need opposite diagnostic paths. A line item that won't spend needs base bids and caps checked first, then frequency, viewability and geography constraints, loosened with the advertiser's agreement since each is a brand-safety decision. A line item that spends and won't convert needs the URL report first — which specific sites or placements the money actually went to — then bids, then creative, then the pixel. Reversing that order and starting with creative is how a month gets spent solving the wrong problem, because a genuinely weak-performing placement can make even strong creative look like it's failing.
The reason placement comes first is simple: it's the fastest, cheapest check available, and it's a genuinely common cause of weak performance. A handful of low-quality third-party sites or apps absorbing a disproportionate share of spend is something a report reveals in minutes; a creative problem takes days to diagnose and longer to fix, so ruling out the cheap explanation before reaching for the expensive one is just good triage.
What a real creative test looks like once you get there
Once placement and bid issues are ruled out, a proper creative test needs real scale to mean anything — the same 27-advertiser, 31-day DSP book that shows a 5.12x blended portfolio ROAS also shows individual account returns ranging from 0.85x to 18.03x with a median of just 4.30x, which is a reminder that even a genuinely working creative can look weak on a small sample before the data settles. Run two creative variants against the same audience and placement mix, with enough budget behind each — generally several thousand dollars minimum per variant — to produce a completion-rate and later-conversion difference that isn't just noise.
Isolate one variable per test where possible — a different opening, a different length, a different call to action — rather than changing several elements at once in a single new cut. A test comparing two creatives that differ in three ways at once can tell you which asset performed better but not why, which limits what the next round of creative can actually learn from the result. Amazon's own DSP reporting supports this kind of line-item-level comparison directly, so the constraint is usually production discipline, not a platform limitation.
A worked example
A $15,000 monthly streaming line item showing a weak ROAS: first pull the URL/placement report and cut anything clearly mismatched to the audience or brand-unsafe. Second, check bid competitiveness against the CPM range for that specific supply source — an underbid campaign quietly loses the best inventory and keeps the worst. Third, if placement and bid both look clean, test creative — two 15-second variants at $3,000-$4,000 each inside the remaining budget. Only after all three, if the pixel-reported conversions still don't match what Amazon Marketing Cloud shows for the same period, check pixel implementation itself.
Document the test plan before launching it — which variable is being isolated, what budget each variant gets, and what result would count as a meaningful difference — rather than deciding after the fact whether a gap between two creatives is real or just noise. Deciding the bar in advance removes the temptation to call a result a win only once it's already known.
What to do when creative testing itself stalls
If two creative variants show no meaningful difference in completion rate or later engagement after a full month at real budget, that's a legitimate result — not every test produces a winner, and running a third variant on the same underlying concept rarely breaks the tie. Test a genuinely different creative approach — different hook, different pacing, different message — rather than a minor variation on the same idea, since small creative tweaks rarely move a metric enough to separate from noise at typical test budgets.
It's also worth checking whether the audience itself, not the creative, has simply been exhausted. A prospecting audience that's been running the same creative for months can show declining performance for reasons that have nothing to do with the asset's quality — the addressable pool has been reached repeatedly, and frequency is doing the damage a fresh creative won't fix. Checking reach growth and frequency distribution alongside a stalled creative test avoids mistaking an audience-saturation problem for a creative one.
The common mistake
The mistake is treating creative as the first and only lever whenever a video campaign underperforms, because it's the most visible, most discussed part of any campaign review. In practice, placement and bid problems are both more common and cheaper to fix than a creative problem, and checking them first saves real budget that would otherwise be spent producing new creative to fix a problem creative was never causing. reMKTR runs 109 live Amazon DSP advertiser seats and applies the URL report, then bids, then creative, then pixel — in that order — on every underperforming video line item, specifically because the order matters as much as the individual checks.
The reverse mistake also happens: a team convinced the creative is fine skips creative testing entirely even after placement and bid checks come back clean, and defends a flat-performing asset indefinitely because it was expensive to produce. Sunk cost in a creative asset isn't evidence it's working — once the cheaper diagnostic steps are exhausted, a genuine creative test is the next honest move, not an optional one.
| Symptom | Check first | Check last |
|---|---|---|
| Line item won't spend | Base bids and caps | Creative |
| Line item spends, won't convert | URL/placement report, then bids | Pixel implementation |
| Genuinely inconclusive after both | Real creative test, at real budget | A third variant of the same idea |
Which one you should actually pick
This diagnostic order suits any team running Amazon DSP video who defaults to blaming creative first when a campaign underperforms. It matters less for a brand that has already ruled out placement and bid issues through routine account management — for that brand, creative testing genuinely is the next lever, and the sequencing question above is already answered.
Shortlist on the job, not the feature grid. Pull your search-term report for the last 90 days and total the spend against terms that produced no orders — 33.6% on the account above. Then ask each vendor on your list what they would do about it in week one, and see who answers with a process rather than a screenshot.
Common questions
Should I test creative first when a video campaign isn't converting?
No — check the URL/placement report and bid competitiveness first. Both are more common causes of weak performance than creative, and both are faster and cheaper to diagnose than producing and testing new creative.
How much budget does a real creative test need?
Generally several thousand dollars per variant at minimum — individual account returns in our own DSP book range from 0.85x to 18.03x even at meaningful scale, so a test funded below that level risks reading noise as a real creative difference.
What if two creative variants show no real difference?
That's a legitimate result. Test a genuinely different creative concept next rather than a minor variation on the same one — small tweaks rarely produce a measurable difference at typical test budgets.
When should I check the pixel?
Last, and only if placement, bid and creative have all been ruled out and the DSP's reported conversions still don't reconcile with what Amazon Marketing Cloud shows for the same period.
Does this diagnostic order apply to a line item that won't spend at all?
No — that's a different failure mode with a different order: base bids and caps first, then frequency, viewability and geography constraints. The URL-report-first approach here applies specifically to a line item that's spending but not converting.
We show the method before the number.
Claim the free auditRead next
- Criteo Pricing: The Fee Stack Inside Your BudgetPricing · criteo pricing
- Acorn Review: Which Acorn, and What Is ProvenReview · acorn review
- Pacvue Pricing: No Public Number — What to AskPricing · pacvue pricing
- Skai vs Pacvue: Contracts, Not Feature GridsHead to head · skai vs pacvue