Sameer Kothari (PepsiCo) and Todd Kirk (Middlegame Marketing Sciences) examined the application of a transformed rendition of marketing mix modeling created through the development of a proprietary system called the “ROI Engine.” Sameer indicated the desire to harness the “true potential of marketing mix models even beyond measuring past campaigns and using it for strategic planning looking forward.” Sameer discussed this system as having a more “predictive ROI outcome-based approach” by “leveraging an ecosystem of leading indicators for before and during a campaign flight.”
Matt Weinman (TelevisaUnivision) and Spencer Lambert (datafuelX) shared the methodology and results from testing TelevisaUnivision’s initiative that, with datafuelX’s technology, enabled their advertising partners to choose their preferred currency in forecasting both long- and short-term audiences for their programming. Implementation involved adjusting the business flow for multi-measurement sources but with each source ingested, validated and normalized to the tech standard separately.
Brendan Kroll of Nielsen and Anne Ori and Daniel Sacks, both of Google, explained that their study’s objective was to identify potential improvements to marketing mix models by utilizing enhanced prior beliefs (priors) based on sales lift studies and exploring the resulting changes in campaign-level sales lift once those priors were incorporated.
Marketers are faced with many challenges such as smaller budgets, targeting decisions and the quantification of media effectiveness, but Harvey Goldhersz of Circana explained that in-flight measurement and optimization provides opportunities to overcome these challenges. Knowledge gained from in-flight measurement can strengthen the optimization of campaign media by enabling maximum impact with lower cost.
Michael Siewert (Colgate-Palmolive) and Souptik Datta (GroupM) presented how their companies worked together to combine measurement data for building custom solutions around bidding in the programmatic space. Colgate wanted to create their own quality definition for their inventory and be able to benchmark at a scalable cost benefit. Building a framework around their definition of quality and creating their own “qCPM” metric allowed them to understand the details of performance at a baseline and optimize with machine learning across 80+ markets and varying formats.
Attention, Emotion and Memory (ESG) are seen as the key elements in building brands.
The “Attention Break-out Session” during the second day of the AUDIENCExSCIENCE conference showed that attention metrics are typically used within a larger framework. Case in point, a presentation by Realeyes and Mars showed how Mars sees attention to marketing messages as a trigger of emotions that leads to memory and (hopefully) brand building.
Source:
AUDIENCExSCIENCE Break-out Session, Attention Metrics: Validity, Reliability & Predictive Powers. Using Attention AI to Predict Real-World Outcomes – Max Kalehoff, VP Marketing Growth, Realeyes; -Johanna Welch, Global Mars Horizon Comms Lab Senior Manager, Mars.
Matt Weinman – Senior Director of Product Management, Advanced Advertising Product, TelevisaUnivision
Matt Weinman (TelevisaUnivision) and Spencer Lambert (datafuelX) shared the methodology and results from testing TelevisaUnivision’s initiative that, with datafuelX’s technology, enabled their advertising partners to choose their preferred currency in forecasting both long- and short-term audiences for their programming. Implementation involved adjusting the business flow for multi-measurement sources but with each source ingested, validated and normalized to the tech standard separately.
Forecasting incorporated a programming schedule imputation process which was then fed into a mixed model estimation (MME), and then optimized with linear granular data.
Their model revealed gaps that they addressed with a variety of tactics including a ratings adjustment approach that updated network viewership trends, a proportional weight method for advanced audiences, recency weighting to avoid stale rate cards and relying less on forecasting viewers rather than scheduled content. The MME drove strong predictive forecasts and increased the use of long-tail inventory.
Key Takeaways
Forecasting should always be done based on content.
In reviewing the accuracy of predicting exact programming, the forecast to actuals had a 71% program match. In predicting programming type, there was 94% accuracy.
Results for the long-term audience forecasts had 42% MAPE (mean absolute percentage error) improvements overall using big data sources.
Daniel Sacks – Incrementality Lead, US Agency, Google
Brendan Kroll of Nielsen and Anne Ori and Daniel Sacks, both of Google, explained that their study’s objective was to identify potential improvements to marketing mix models by utilizing enhanced prior beliefs (priors) based on sales lift studies and exploring the resulting changes in campaign-level sales lift once those priors were incorporated.
Incrementality experiments are widely accepted as the gold standard for causal measurement. Calibrating individual channels via experimentation ensures optimization of model outcomes. However, the results of incrementality experiments are often not part of marketing mix model (MMM) design.
Nielsen utilized NCS sales lift studies as the source of the experimental data for this analysis. NCS determined the causal effects of advertising on incremental sales while controlling for targeting and other co-variates.
The study design involved 10 brands with existing MMMs and available NCS results for corresponding periods, model re-estimation using NCS lift priors, refinement of the priors and scaling. This study showed that applying this methodology to a YouTube campaign resulted in significant sales lift, as well as revenue and ROAS increases, including a 2.6x median increase in the effectiveness in the adjusted model.
The adjusted model showed greater marketing contribution overall; therefore, marketers are at risk of undervaluing their overall marketing if experimental results are not included.
Key Takeaways
Brands can effectively leverage experiment-based priors to strengthen marketing mix models.
For nascent channels, the inclusion of experimentation results proved fundamental, especially if those campaigns showed strong initial results, since MMMs cannot rely solely on historical anchoring to measure true impact.
When experiments reveal high performing channels or campaigns, the use of testing can aid more accurate MMM measurements as investment scales.
Even for channels with long histories and relative stability, experimentation can serve as a way to validate models and may give models a chance to remain flexible in case of strategic shifts and/or changes in consumer behavior.
Michael Siewert – Programmatic Director, Colgate-Palmolive Co.
Michael Siewert (Colgate-Palmolive) and Souptik Datta (GroupM) presented how their companies worked together to combine measurement data for building custom solutions around bidding in the programmatic space. Colgate wanted to create their own quality definition for their inventory and be able to benchmark at a scalable cost benefit. Building a framework around their definition of quality and creating their own “qCPM” metric allowed them to understand the details of performance at a baseline and optimize with machine learning across 80+ markets and varying formats.
Key Takeaways: Mapping their quality journey involved
Defining the quality metric for Colgate led Souptik (GroupM) to create a menu of foundational elements (verification, clarity) and Colgate’s CPMs, KPIs, goals and values that constituted quality for them, which then informed a custom formula. Their qCPM metric is formulated from cost, quality and business effectiveness KPIs.
Reporting and benchmarking: Using multiple DSPs, clean rooms and viewability partners to build a global reporting scalable cloud-based system, Colgate’s interactive dashboard was able to measure benchmarks and put dollar values to opportunity sizes (underperforming and extra mile) for the first time.
Automating custom bidding to optimize for quality: Because it’s not possible to optimize manually, Colgate used its own AI algorithm to spot inefficiencies, seeing a nearly 20% lift in small-scale tests. They also implemented their own custom bidding tests to monitor and analyze to see how much improvement they could make.
