Greg Dolan (Keen Decision Systems) and Mark Bennet (Johnsonville Milwaukee) examined how to navigate in uncertain and volatile times in the current marketplace using next-generation marketing mix solutions. In the opening, Greg explored the progression of marketing from the late 90s, through what he dubbed “The Roaring 20s.” He noted that we went from a minimally complex, slower-paced “top-down” approach in the 1990s to a very fast-paced, complex environment with a shift to Retail Media and a unified approach where we apply next-generation predictive analytics in the 2020s. Greg discussed the intricacies of their approach of combining historical data with predictive/prescriptive plans to address drastic changes in the current environment, leveraging ML. He provided a case study that demonstrated the successful application of the next-generation marketing mix. In addition, Mark gave a client perspective on how they are handling market uncertainty.
Sameer Kothari (PepsiCo) and Todd Kirk (Middlegame Marketing Sciences) examined the application of a transformed rendition of marketing mix modeling created through the development of a proprietary system called the “ROI Engine.” Sameer indicated the desire to harness the “true potential of marketing mix models even beyond measuring past campaigns and using it for strategic planning looking forward.” Sameer discussed this system as having a more “predictive ROI outcome-based approach” by “leveraging an ecosystem of leading indicators for before and during a campaign flight.”
Liz Riley (OLLY) and Mark Garratt (In4mation Insights) explored rebuilding and reimagining marketing mixed modeling (MMM) to better handle fragmented data, in the era of retail media networks. Mark lauded MMM as an effective technique that has contributed to financial success for many businesses. In light of data becoming increasingly fragmented, he suggested that “some reinvention of the fundamental model framework is going to be required in order to move this old venerable method into the future.” Mark and Liz examined the Bayesian approach to MMM in handling fragmented data. Mark noted that there will not be a situation “where all the data is the same granularity in one place at one time.” The Bayesian approach can “fill in the blanks” of missing or fragmented data using reasonable estimates, creating a more accurate picture, which traditional MMM falls short of in the retail media environment.
Peter Cain of Marketscience covered shortcomings of marketing mix modeling (MMM). It is hampered by the endogeneity problem (selection bias) in things like online media. MMM is also only a short-term approach. The standard long-term method accompanying MMM is also flawed and ignores the time-series properties of the data.
In this session, Elea McDonnell Feit (Drexel University) led a panel discussion with the day’s speakers on innovations in experiments in marketing and referred to these experiments as a “mature part of the measurement system.” In this discussion panel members brought up ideas and examples of how to effectively employ randomized controlled trials (RCT) and the benefits of using experiments for attribution. They examined the lack of patterns stemming from advertising incrementality and credited this to the changing nature of the consumer journey and unique factors in strategy, the business life cycle and the product being sold. The panel also explored processes to ensure the deployment of a successful and effective experiment. In addition, geo-based tests were also considered. Other topics discussed were the cost-effectiveness of running experiments and the value of failed experiments.
The MRC’s Ron Pinelli outlined the scope of the Outcomes and Data Quality Standard, recently completed in September 2022. Part of MRC’s mission is setting standards for high quality media and advertising measurement, and Ron walked through the phased approach and iterative process that included the ANA, the 4A’s and other industry authorities.
Brendan Kroll of Nielsen and Anne Ori and Daniel Sacks, both of Google, explained that their study’s objective was to identify potential improvements to marketing mix models by utilizing enhanced prior beliefs (priors) based on sales lift studies and exploring the resulting changes in campaign-level sales lift once those priors were incorporated.
Daniel Sacks – Incrementality Lead, US Agency, Google
Brendan Kroll of Nielsen and Anne Ori and Daniel Sacks, both of Google, explained that their study’s objective was to identify potential improvements to marketing mix models by utilizing enhanced prior beliefs (priors) based on sales lift studies and exploring the resulting changes in campaign-level sales lift once those priors were incorporated.
Incrementality experiments are widely accepted as the gold standard for causal measurement. Calibrating individual channels via experimentation ensures optimization of model outcomes. However, the results of incrementality experiments are often not part of marketing mix model (MMM) design.
Nielsen utilized NCS sales lift studies as the source of the experimental data for this analysis. NCS determined the causal effects of advertising on incremental sales while controlling for targeting and other co-variates.
The study design involved 10 brands with existing MMMs and available NCS results for corresponding periods, model re-estimation using NCS lift priors, refinement of the priors and scaling. This study showed that applying this methodology to a YouTube campaign resulted in significant sales lift, as well as revenue and ROAS increases, including a 2.6x median increase in the effectiveness in the adjusted model.
The adjusted model showed greater marketing contribution overall; therefore, marketers are at risk of undervaluing their overall marketing if experimental results are not included.
Key Takeaways
Brands can effectively leverage experiment-based priors to strengthen marketing mix models.
For nascent channels, the inclusion of experimentation results proved fundamental, especially if those campaigns showed strong initial results, since MMMs cannot rely solely on historical anchoring to measure true impact.
When experiments reveal high performing channels or campaigns, the use of testing can aid more accurate MMM measurements as investment scales.
Even for channels with long histories and relative stability, experimentation can serve as a way to validate models and may give models a chance to remain flexible in case of strategic shifts and/or changes in consumer behavior.
“The clock is ticking. In September 2024, the currency we’ve all used for decades is going to disappear. That means no more year-over-year comps. Sorry, no more media mix models based on outdated TV metrics that are simple, siloed counting and was yesterday’s consumers.”Read more »
A growing number of privacy regulations and data deprecations are creating a highly complicated landscape for marketers to maneuver through. To help us better understand how these forces affect marketing and marketing effectiveness measurement, the ARF recently gathered a group of experts on privacy-related topics from both practitioner and academic backgrounds, to discuss the key issues and outline ways forward. This report is the result of that effort. It includes an overview of the latest privacy-related macro-trends, implications for marketing and measurement and actionable recommendations.
