On June 26, Sequent Partners, with their panoramic perspective on the most important developments in marketing analytics, analyzed the most disruptive and revolutionary analytic applications and techniques impacting our world. Plus, attendees got a sneak peek of the themes featured in this year’s upcoming Marketing Analytics Accelerator.
Member Only Access
On June 25, OptiMine dove into the specifics of Google’s privacy changes and revealed how these changes will impact the marketing and measurement landscape. Traditional forms of measurement will no longer be viable as privacy-driven data loss continues to diminish accuracy, coverage and reliability. Attendees gained applicable strategies on how their brands can adapt before it’s too late.
Member Only Access
At SHOPPER 2024, practitioners and academics shared research-based insights on retail media networks (RMN), AI, influencer marketing and live shopping. The industry’s leading experts revealed which tools, technologies and trends are shaping the ever-evolving shopper landscape and what brands need to know to stay ahead.
Member Only Access
Sophie MacIntyre – Ads Research Lead, Marketing Science, Meta
Randomized Control Trials (RCTs) are the gold standard for unbiased measurement of incrementality according to Sophie MacIntyre (Meta). However, there are situations where RCTs are not available so Meta explored other methods to improve the measurement of incrementality. Meta’s researchers wanted to know how close they could get to the experimental result by using non-experimental methods. The researchers were unable to accurately measure an ad campaign’s effect with sophisticated observational methods. Additionally, traditional non-experimental models like propensity score matching and double machine learning were difficult to use and resulted in large errors. Sophie presented incrementality as a ladder of options that get closer to measuring true business value as the ladder is ascended. The different rungs of the ladder are based on how well a particular measurement approach can isolate the effect of a campaign from any other factors. This research was undertaken in collaboration with the MMA and analyzed non-incremental models, quasi-experiments with incrementality models and randomized experiments. Meta revealed that incrementality could be achieved with modeling if the research included some RCTs. Using PIE (predictive incrementality by experimentation) estimates for decision making led to results similar to experiment-based decisions. Sophie stated that academic collaborations provide quantitative evidence of the value of incremental methods. Key takeaways:
Brett Mershmann – Sr. Director, Research & Development (R&D), NCSolutions
Brett Mershmann’s (NCSolutions) discussion focused on how to quantify incremental advantages of some more modern contemporary machine learning (ML) frameworks, over more traditional measurements for incrementality. Beginning the presentation, Brett provided an overview of both traditional modeling techniques as well as more contemporary ML campaign measurements. To understand the differences, Brett detailed an 11-experiment process, using real observational household data, intersected with real campaign impression data but with simulated outcome and with a defined outcome function. The experiments measured accuracy, validity and power. Additionally, they compared ML with randomized controlled trials (RCTs), noting that RCTs are the gold standard but are not always feasible. To accomplish this, they ran both an RCT and an ML analysis, by creating test-control groups on real, limited data. This experiment applied the same outcome function to each, depending on a larger set of variables. In closing, Brett shared feedback from these experiments, which supported ML as a powerful method of measurement and a viable alternative to RCTs. He highlighted the importance of getting the correct data into these models for optimum results. Key takeaways:
Joe Zucker – Senior Manager, Marketing Analytics, Indeed
Clean room experiments are challenging in an online marketplace, such as Indeed’s job site for employers and employees, due to potential online experimentation biases, including activity bias, ad server bias and base rate bias, according to Joe Zucker (Indeed). Control groups can be created in multiple ways with different degrees of technical setup or in some cases, external modeling. The five variations of control groups are ghost ads, publisher house ads, PSA ads, propensity score matching and intent to treat. A comparison indicated that each option has both pros and cons, including cost, the need for additional data or publisher support. Joe reminded the audience that there is “no free lunch.” Ghost ads would be preferred by Indeed to create the control group; however, this option has high technical set-up requirements, few publisher partners have this capability and there is low control over the analysis. There are also challenges related to interpreting experimental results, which include low match/conversion rates and the need to analyze experiments with different control group construction. Indeed was able to measure aggregate incrementality for their campaign metrics and prove the value of their advertising as a result of these clean room experiments. Key takeaways:
The Attribution & Analytics Accelerator returned for its eighth year as the only event focused exclusively on attribution, marketing mix models, in-market testing and the science of marketing performance measurement. The boldest and brightest minds took the stage to share their latest innovations and case studies. Modelers, marketers, researchers and data scientists gathered in NYC to quicken the pace of innovation, fortify the science and galvanize the industry toward best practices and improved solutions. Content is available to event attendees and ARF members.
Member Only AccessManaging business risk involves having a rational, data-driven view of the future while simultaneously being as prepared as possible for external shocks — from a global pandemic and the ensuing supply-chain disruptions, to inflation, data signal losses, war, and great power competition. At our annual Forecasting event, held virtually on July 18, leading experts shared how businesses can adapt forecasting techniques to manage risk.
Managing business risk involves having a rational, data-driven view of the future while simultaneously being as prepared as possible for external shocks — from a global pandemic and the ensuing supply-chain disruptions, to inflation, data signal losses, war, and great power competition. At our annual Forecasting event, held virtually on July 18, leading experts shared how businesses can adapt forecasting techniques to manage risk.
Member Only AccessTristan Webster and Kenneth Wilbur showcased their most recent collaborative work examining attention and frequency in advertising: the impact of multiple exposures on people’s attention to TV ads. They applied CTV data which TVision has collected natively in the field to provide insight into the long-examined question, “Is there an optimal frequency for TV ads?”, but more granularly: “What is happening in the media environment while viewers see ads, and how does that affect their attention?”