marketing experiments
Presentation: MSI Webinar: Meta Ad Testing Demystified: Divergent Delivery and What It Means for Your Results
Meta’s Lift and A/B tests are widely used to evaluate campaign performance, but they answer fundamentally different questions. Lift tests estimate true incrementality using a no-ad control, while A/B tests compare campaign variants without a control group.
A key challenge in A/B testing is “divergent delivery,” where Meta’s algorithms distribute each variant to different audience segments. This means observed performance differences may reflect both creative effectiveness and who saw the ads.
Drawing on large-scale evidence from thousands of Lift and A/B tests, this webinar shows when and why divergent delivery occurs, why it can be both informative and misleading, and how it compares to Lift test results. You’ll also learn practical ways to reduce imbalance—through campaign setup choices like targeting, budgets, bidding, and placements—to better isolate creative impact when that’s the goal.
PIE (Predicted Incrementality by Experimentation) Takes the Cake on Ad Measurement
Random controlled trials (RCTs) can assess causal effects of marketing but are expensive and incur opportunity costs by excluding control groups. Predicted Incrementality by Experimentation (PIE) uses samples of RCT-run campaigns to determine which characteristics map to causal outcomes and then applies that mapping to campaigns not run as RCTs.
Predicted Incrementality by Experimentation (PIE) for Ad Measurement
Predicted Incrementality by Experimentation (PIE) uses samples of RCT-run campaigns to determine which metrics map to causal outcomes and then applies that mapping to campaigns not run as RCTs, resulting in accurate predictions of advertising effects.
- Audience & Media Measurement
- Article
Learning Across Marketing Experiments: A Bayesian Approach to Improving Targeting
Companies run many marketing experiments, but most A/B tests are analyzed independently—limiting what firms can learn about how customers respond to interventions over time. This research introduces a hierarchical Bayesian framework that integrates data from many experiments simultaneously to estimate customer-level responsiveness to marketing. Using large-scale field experiments, the model decomposes treatment effects into customer, campaign and timing components and uses these insights to improve targeting decisions. The results show that most variation in marketing effectiveness comes from persistent differences in customer responsiveness, enabling firms to better identify who to target and when.
Member Only Access- Past Event Highlights
- Article
Proving Influencer Success with RCTs and Brand Lift Data
Tyler Montague – VP of Client & Research Success, Swayable
Maddie Perkins – Director, Strategic Planning, Influential
Tyler Montague from Swayable and Maddie Perkins of Influential explored whether influencer marketing truly drives business outcomes, positioning it as a measurable, full-funnel performance channel rather than just an awareness tactic. Swayable applies an AI-enabled randomized controlled trial (RCT) framework to pre-test creative, comparing exposed vs. control groups across thousands of respondents to isolate causal impact on key brand metrics (e.g., favorability, purchase intent). This is complemented by Influential’s large-scale campaign data and platform integrations, enabling meta-analyses across campaigns, audiences and verticals to evaluate performance at scale. The approach addresses a core industry challenge: proving effectiveness beyond engagement metrics (likes, clicks) and linking influencer content to meaningful business outcomes. Member Only Access
- Past Event Highlights
- Article
Proving Influencer Success with RCTs and Brand Lift Data
Tyler Montague – VP of Client & Research Success, Swayable
Maddie Perkins – Director, Strategic Planning, Influential
Tyler Montague from Swayable and Maddie Perkins of Influential explored whether influencer marketing truly drives business outcomes, positioning it as a measurable, full-funnel performance channel rather than just an awareness tactic. Swayable applies an AI-enabled randomized controlled trial (RCT) framework to pre-test creative, comparing exposed vs. control groups across thousands of respondents to isolate causal impact on key brand metrics (e.g., favorability, purchase intent). This is complemented by Influential’s large-scale campaign data and platform integrations, enabling meta-analyses across campaigns, audiences and verticals to evaluate performance at scale. The approach addresses a core industry challenge: proving effectiveness beyond engagement metrics (likes, clicks) and linking influencer content to meaningful business outcomes. Findings show that influencer marketing consistently outperforms traditional content across both upper- and lower-funnel metrics, challenging the perception that it is primarily an awareness driver. The results also highlight the importance of trust and authenticity as performance drivers, as well as the scalability of influencer effectiveness across demographics and categories. The research reframes influencer marketing as a core strategic lever—not a test-and-learn channel—while emphasizing the need for rigorous, experimental measurement to guide investment decisions. Key Takeaways:- Influencer marketing is rapidly scaling, with the global market reaching ~$32 billion and projected to hit ~$60 billion, and 83% of brands investing in the space.
- RCT-based measurement enables causal attribution, using controlled exposure vs. control groups to isolate lift in key metrics like favorability and purchase intent.
- Influencer content outperforms traditional creative, delivering ~6.7-point lift in favorability (vs. ~2.8 baseline) and higher purchase intent lift (~3.5 vs. ~3.1).
- Effectiveness extends across the funnel, disproving the notion that influencer marketing is only upper funnel. It also drives meaningful lower funnel outcomes (purchase intent).
- Trust and authenticity are key performance drivers, with high-trust creators driving up to +67% increases in lower-funnel performance.
- Influencer marketing works across demographics, with strong performance not only among Gen Z but also Millennials, Gen X and even Boomers.
- Effectiveness spans categories, including strong results in CPG and non-traditional influencer categories (e.g., finance, healthcare), expanding its applicability.
Watch the Presentation
Member Only AccessCombine and Conquer: Integrating Experiments for Better Targeted Marketing
A Bayesian model integrating multiple marketing interventions over time combines the causal-inference advantages of randomized experimentation with the longitudinal richness of repeated observational data. Results reveal that marketing effectiveness depends much more on customer receptivity than campaign design or timing, pointing to the need for—and means of—better targeting.