Seventy Percent of Marketers Don’t Trust Platform-Reported ROAS. Here’s How to Fix That.
A recent Forrester survey found that 70% of brand-side marketers question the accuracy of vendor-reported performance metrics — yet fewer than 15% have a formal process to independently verify them. As generative AI ad formats proliferate across Meta, Google, and TikTok, the gap between reported and actual incremental lift is widening. This generative AI ROAS verification playbook gives procurement and analytics teams a structured methodology to close that gap.
The stakes are enormous. Generative AI ad spend is projected to exceed $50 billion globally, according to Statista’s digital advertising data. Vendors are incentivized to report the rosiest numbers possible. Your job is to separate signal from noise — and protect the budget.
Why Vendor-Reported Lifts Are Structurally Inflated
Before diving into methodology, it’s worth understanding why platform-reported metrics so frequently overstate performance. This isn’t necessarily fraud. It’s incentive alignment — or rather, misalignment.
Platforms like Meta’s business tools and Google’s Performance Max use last-touch or algorithmic attribution models that favor their own ecosystem. When a generative AI ad format — say, a dynamically assembled creative variant on Meta Advantage+ — gets credit for a conversion, the platform rarely asks: would this conversion have happened anyway?
That question is the entire ball game.
Three structural biases inflate reported lifts:
- Selection bias: AI-optimized ad delivery targets users already likely to convert, then claims credit for those conversions.
- Attribution window padding: Longer click and view windows capture organic conversions and attribute them to paid activity.
- Missing counterfactuals: Without a true holdout group, there’s no baseline to measure incremental impact against.
If you’ve ever seen a vendor claim a 300% ROAS lift from their new AI creative tool and thought “that can’t be right” — your instincts were probably correct. Our companion piece on evaluating AI ROAS claims covers the vendor-side red flags in detail.
Control Group Design: The Non-Negotiable Foundation
No control group, no credible measurement. Full stop.
Yet most brand teams accept vendor lift studies that use synthetic baselines, historical comparisons, or — worst of all — no control at all. Here’s how to design controls that actually hold up to scrutiny.
Ghost ads / intent-to-treat holdouts. The gold standard. A randomly selected subset of your target audience is flagged to receive the ad but instead sees either a PSA (public service announcement) or a blank placeholder. These users are still “in market” and subject to all the same organic signals. The difference in conversion rates between exposed and holdout groups represents true incremental lift.
Geographic holdouts. When ghost ads aren’t feasible — TikTok still doesn’t support them natively — use matched-market tests. Select two to four DMAs with statistically similar demographics and baseline conversion rates. Run the AI ad format in test markets, suppress it in control markets, and compare. Tools like Google’s open-source CausalImpact R package can model the counterfactual time series.
Minimum sample requirements. This is where many teams cut corners. For a 5% detectable lift at 95% confidence, you need roughly 15,000-25,000 users per cell, depending on your baseline conversion rate. Anything less and you’re measuring noise.
If a vendor refuses to support a holdout group — or insists their internal lift study is “just as rigorous” — that’s the single biggest red flag in AI ad format evaluation. Walk away or demand independent measurement.
For teams building out their measurement infrastructure, understanding how CRM attribution and identity resolution work together is critical to connecting holdout group membership to downstream conversion events.
Attribution Window Settings That Don’t Lie to You
Attribution windows are the silent assassin of accurate ROAS measurement. A 28-day click, 1-day view window will capture conversions that had nothing to do with your ad. A 1-day click, 0-day view window might miss legitimate influence. The right setting depends on your product’s consideration cycle — not on what the platform defaults to.
Here’s a practical framework:
- Map your actual purchase journey. Use your CRM data to calculate median time from first touchpoint to conversion. If 80% of conversions happen within 3 days, a 7-day click window is reasonable. A 28-day window is padding.
- Kill view-through attribution for ROAS calculations. View-through conversions (someone saw an ad impression, then converted later) are legitimate for awareness measurement. They are wildly unreliable for ROAS. A user who was served a 0.3-second autoplay video in a feed and then purchased five days later was almost certainly influenced by something else.
- Run parallel windows. Report ROAS at 1-day click, 7-day click, and 28-day click simultaneously. If the “lift” from the AI ad format only materializes in the 28-day window, that’s organic conversion contamination — not genuine performance.
- Compare to non-AI control creative. The question isn’t just “did the AI format drive conversions?” It’s “did it drive more conversions than the standard format at the same spend level?” Always benchmark AI variants against your best-performing non-AI creative.
Teams running creator-driven ads across platforms should also calibrate these windows against their TikTok Shop attribution stack to ensure consistent measurement logic across channels.
Red Flags That Separate Real Lift from Vendor Theater
After running dozens of these audits, patterns emerge. Here are the red flags that experienced analytics teams watch for.
“Lift” that disappears when you shorten the attribution window. Genuine incremental impact from better creative shows up quickly. If tightening from 28-day to 7-day click attribution cuts reported ROAS by more than 40%, the AI format isn’t driving incremental conversions — it’s harvesting organic ones.
No holdout group in the vendor’s study. Many vendors present “lift studies” that compare performance before and after deploying their AI format. This is not a lift study. It’s a before-and-after comparison contaminated by seasonality, media mix changes, competitive activity, and a dozen other confounds.
Suspiciously consistent lift across segments. Real performance varies. If a vendor shows a 25% lift across every demographic, every creative variant, and every product category, someone smoothed the data. Ask for segment-level breakdowns with confidence intervals.
Refusal to share raw conversion logs. If a vendor won’t provide event-level data for your analytics team to independently analyze, the reported numbers cannot be trusted. Period.
ROAS calculations that exclude creative production costs. Generative AI ad formats often require setup fees, API costs, or platform-specific creative adaptation. If the vendor’s ROAS calculation only counts media spend in the denominator, it’s flattering the metric. Insist on fully loaded cost inputs — a principle that applies equally when evaluating AI video pricing models and their total cost of ownership.
The most reliable signal of genuine incremental lift: when your own team’s independently measured ROAS — using your attribution windows, your holdout design, and your conversion data — comes within 15% of the vendor’s reported number. Anything beyond that gap warrants a serious conversation.
Building the Internal Verification Muscle
This isn’t a one-time audit. It’s an operational capability.
The brands getting this right — Unilever, Procter & Gamble, and a growing cohort of DTC brands with strong data teams — embed verification into every AI ad format test from day one. They negotiate holdout requirements into vendor contracts. They standardize attribution windows across platforms. They maintain a “measurement readiness checklist” that procurement consults before approving any new AI creative vendor.
What does that checklist look like?
- Holdout group design agreed upon before campaign launch
- Attribution windows aligned to product consideration cycle, not platform defaults
- Raw event-level data access guaranteed in the contract
- Independent measurement partner (Measured, Incrementality, or similar) engaged for campaigns above $100K
- Pre-registered success criteria — what ROAS threshold constitutes a “win” — documented before results come in
That last point matters more than most teams realize. If you define success after seeing the results, you’ll unconsciously move the goalposts. Pre-registration eliminates that bias. Campaign analytics dashboards like the ones covered in our analytics dashboard evaluation guide can help standardize how your team tracks these pre-registered KPIs in real time.
Your Next Step
Pick your highest-spend AI ad format vendor. Request event-level conversion data and propose a 5% holdout test on your next campaign. If they push back, you’ve already learned everything you need to know about the reliability of their reported numbers. Start there — the rest of the playbook follows.
Frequently Asked Questions
What is generative AI ROAS verification?
Generative AI ROAS verification is the process of independently validating vendor-reported return on ad spend claims from AI-powered ad formats. It involves designing proper control groups, setting appropriate attribution windows, and analyzing raw conversion data to determine whether reported performance lifts represent genuine incremental value or inflated platform metrics.
How large should a holdout group be for AI ad format testing?
For detecting a 5% incremental lift at 95% statistical confidence, you need approximately 15,000 to 25,000 users per cell (both test and control groups). The exact number depends on your baseline conversion rate — lower baseline rates require larger samples. Undersized holdout groups produce unreliable results that can’t distinguish real lift from statistical noise.
Why do platform-reported ROAS numbers tend to be inflated?
Platform-reported ROAS is structurally inflated due to three main factors: selection bias (AI targets users already likely to convert), generous attribution windows that capture organic conversions, and the absence of true counterfactual measurement. Platforms use models that favor their own ecosystem, attributing conversions that would have occurred regardless of ad exposure.
What attribution window should brands use when measuring AI ad performance?
Brands should align attribution windows to their actual product consideration cycle based on CRM data, not platform defaults. For most consumer products, a 7-day click window is reasonable. View-through attribution should be excluded from ROAS calculations entirely. Running parallel windows (1-day, 7-day, and 28-day click) reveals whether reported lift is genuine or driven by organic conversion contamination.
What is the biggest red flag in a vendor’s AI ROAS lift study?
The single biggest red flag is the absence of a proper holdout or control group. Many vendors present before-and-after comparisons or synthetic baselines as “lift studies,” but these are contaminated by seasonality, media mix changes, and other confounding variables. If a vendor refuses to support a randomized holdout test, their reported performance numbers cannot be independently validated.
Top Influencer Marketing Agencies
The leading agencies shaping influencer marketing in 2026
Agencies ranked by campaign performance, client diversity, platform expertise, proven ROI, industry recognition, and client satisfaction. Assessed through verified case studies, reviews, and industry consultations.
Moburst
-
2
The Shelf
Boutique Beauty & Lifestyle Influencer AgencyA data-driven boutique agency specializing exclusively in beauty, wellness, and lifestyle influencer campaigns on Instagram and TikTok. Best for brands already focused on the beauty/personal care space that need curated, aesthetic-driven content.Clients: Pepsi, The Honest Company, Hims, Elf Cosmetics, Pure LeafVisit The Shelf → -
3
Audiencly
Niche Gaming & Esports Influencer AgencyA specialized agency focused exclusively on gaming and esports creators on YouTube, Twitch, and TikTok. Ideal if your campaign is 100% gaming-focused — from game launches to hardware and esports events.Clients: Epic Games, NordVPN, Ubisoft, Wargaming, Tencent GamesVisit Audiencly → -
4
Viral Nation
Global Influencer Marketing & Talent AgencyA dual talent management and marketing agency with proprietary brand safety tools and a global creator network spanning nano-influencers to celebrities across all major platforms.Clients: Meta, Activision Blizzard, Energizer, Aston Martin, WalmartVisit Viral Nation → -
5
The Influencer Marketing Factory
TikTok, Instagram & YouTube CampaignsA full-service agency with strong TikTok expertise, offering end-to-end campaign management from influencer discovery through performance reporting with a focus on platform-native content.Clients: Google, Snapchat, Universal Music, Bumble, YelpVisit TIMF → -
6
NeoReach
Enterprise Analytics & Influencer CampaignsAn enterprise-focused agency combining managed campaigns with a powerful self-service data platform for influencer search, audience analytics, and attribution modeling.Clients: Amazon, Airbnb, Netflix, Honda, The New York TimesVisit NeoReach → -
7
Ubiquitous
Creator-First Marketing PlatformA tech-driven platform combining self-service tools with managed campaign options, emphasizing speed and scalability for brands managing multiple influencer relationships.Clients: Lyft, Disney, Target, American Eagle, NetflixVisit Ubiquitous → -
8
Obviously
Scalable Enterprise Influencer CampaignsA tech-enabled agency built for high-volume campaigns, coordinating hundreds of creators simultaneously with end-to-end logistics, content rights management, and product seeding.Clients: Google, Ulta Beauty, Converse, AmazonVisit Obviously →