Carl Mela (Duke University) helmed a panel of the day’s presenters to further review the “vanguard work of MMM in 2022.” Granularity inspired the most debate among the panelists, with other topics including causality, cadence of modeling vs. decision-making, false trust in priors, marketing mix model (MMM)’s worst mistakes and lack of precision, and methods for long-term ROI and branding meriting discussion.
Analytic Partners’ Trent Huxley interviewed client Mallory Fetters of Cox Communications on the telecom’s marketing measurement strategy. Dealing with constantly evolving challenges, both common and specific to its industry, Mallory expanded on how data deprecation, shifting consumer media behaviors, demand for faster speeds and growing consumer choice and competition rapidly accelerated Cox’s learning curve.
Greg Dolan (Keen Decision Systems) and Mark Bennet (Johnsonville Milwaukee) examined how to navigate in uncertain and volatile times in the current marketplace using next-generation marketing mix solutions. In the opening, Greg explored the progression of marketing from the late 90s, through what he dubbed “The Roaring 20s.” He noted that we went from a minimally complex, slower-paced “top-down” approach in the 1990s to a very fast-paced, complex environment with a shift to Retail Media and a unified approach where we apply next-generation predictive analytics in the 2020s. Greg discussed the intricacies of their approach of combining historical data with predictive/prescriptive plans to address drastic changes in the current environment, leveraging ML. He provided a case study that demonstrated the successful application of the next-generation marketing mix. In addition, Mark gave a client perspective on how they are handling market uncertainty.
Sameer Kothari (PepsiCo) and Todd Kirk (Middlegame Marketing Sciences) examined the application of a transformed rendition of marketing mix modeling created through the development of a proprietary system called the “ROI Engine.” Sameer indicated the desire to harness the “true potential of marketing mix models even beyond measuring past campaigns and using it for strategic planning looking forward.” Sameer discussed this system as having a more “predictive ROI outcome-based approach” by “leveraging an ecosystem of leading indicators for before and during a campaign flight.”
Liz Riley (OLLY) and Mark Garratt (In4mation Insights) explored rebuilding and reimagining marketing mixed modeling (MMM) to better handle fragmented data, in the era of retail media networks. Mark lauded MMM as an effective technique that has contributed to financial success for many businesses. In light of data becoming increasingly fragmented, he suggested that “some reinvention of the fundamental model framework is going to be required in order to move this old venerable method into the future.” Mark and Liz examined the Bayesian approach to MMM in handling fragmented data. Mark noted that there will not be a situation “where all the data is the same granularity in one place at one time.” The Bayesian approach can “fill in the blanks” of missing or fragmented data using reasonable estimates, creating a more accurate picture, which traditional MMM falls short of in the retail media environment.
Walgreens’ three-year change management evolution, in partnership with Ipsos MM, embedded unified marketing measurement (UMM) into tactical and strategic decisions at the retailer. This approach to change management included simulated impact testing, validated results and provided proof points. Douglas Brooks of Ipsos MMA provided a current overview of UMM at Walgreens, which consists of monthly marketing mix models, weekly attribution models and continuous validation through testing. Jenny Peelle of Walgreens stated that this approach impacts decision making, including what businesses to support, reach and frequency management and an understanding of attribution for individual KPIs as well as for enterprise business growth objectives. The resulting shift to UMM increased revenue at both the business unit level and for the entire enterprise.
In this session, Leslie Wood of NCSolutions and Patrick McGraw of Colgate-Palmolive walked the audience through the process their machine learning (ML) program takes to discover the baseline sales for exposed households. The ML platform consists of a double robust causal framework that they began training in 2014, a super learner to do modeling, a database of results and a decision-tree – where no levers are thrown, as everything is preset. She also defined a counterfactual or the results, which in this case is incremental sales.
In this session, Elea McDonnell Feit (Drexel University) led a panel discussion with the day’s speakers on innovations in experiments in marketing and referred to these experiments as a “mature part of the measurement system.” In this discussion panel members brought up ideas and examples of how to effectively employ randomized controlled trials (RCT) and the benefits of using experiments for attribution. They examined the lack of patterns stemming from advertising incrementality and credited this to the changing nature of the consumer journey and unique factors in strategy, the business life cycle and the product being sold. The panel also explored processes to ensure the deployment of a successful and effective experiment. In addition, geo-based tests were also considered. Other topics discussed were the cost-effectiveness of running experiments and the value of failed experiments.
Sudeshna Sen (Dentsu International) and Aarti Bhaskaran (Snapchat) examined the unique role experiments play in marketing performance and decision-making. The increased importance of experiments was discussed, particularly in light of App Tracking Transparency (ATT), which created a challenging shift in the way Dentsu and other marketing firms collect user data. The discussion around the growing importance of experimentation in marketing included ways to incorporate experimentation into a client’s marketing goals and best practices for testing and experimentation. Performance collaborations with platforms were examined as a method to produce solid testing results based on client needs and business goals. In addition, the importance of aligning relevant business goals and KPIs were acknowledged as a key component in identifying the appropriate testing and measurements, to create more meaningful outcomes. Next steps include progressing from “anecdotal incidents of performance to a proof of concept.”
In this session, Ross Link (Marketing Attribution) and Jeff Doud (Ocean Spray Cranberries) examined a large-scale experiment conducted with Ocean Spray. They applied randomized controlled trials (RCT) to approximately 10 million households (30 – 40 million people) in which ads were consumed by their participants via a variety of devices. Jeff explained that the experiment was done to measure the impact of when certain ads were suppressed for some of their participants. Additionally, they examined an MTA (multi-touch attribution) logit model that was subsequently applied, which yielded KPIs such as ROI. Information from this MTA-RCT experiment supplied refreshed results monthly. Daily ROI results from the campaign were collected from the MTA-applied modeling. Outcomes from this experiment revolved around retargeting and recent and lagged buyers. In addition, the study also explored creative treatments and platform effectiveness.