Change has been the only constant of the past 18 months, but for brands that know how to adapt, this change represents opportunity. Holistic measurement – from understanding incremental impact across channels to building in considerations for macro factors like consumer behavior shifts and the pandemic – is key to sustained growth.
Dennis Buchheim has a unique perspective from “two sides” of the ad industry. He leads Facebook’s Ads Ecosystem team after serving as CEO of IAB Tech Lab. Dennis shared a view on how the industry is grappling with the shifting regulatory, platform, and technology landscape. Only together can the industry understand these changes and create paths forward. As opportunities are evaluated to evolve how data is used, research will be critical in refining the industry’s foundational knowledge and providing tactical guidance.
An impressive body of work is building in attention measurement. The three winning-papers sessions preceding this panel revealed a work in progress with shared goals as well as differences in approaches. Moderator Earl Taylor of the ARF’s MSI division asked the speakers about their views on barriers to the process, and opportunities for further improving attention measures.
Xandr’s Peter Doe reinforced the omnipresence of bias in TV measurement as he outlined four key areas of bias in assessing DirecTV’s (DTV) set-top box (STB) data for its national data-driven linear TV advertising. Noting DTV’s relatively low sampling size (7M STB homes) has a high level of bias when measuring for national TV viewing, Peter provided a top-line overview of Xandr’s viewership data methodology relevant to advertisers and marketers working with big datasets.
The academic study at the heart of this presentation compared 13 hierarchy-of-effects (HoE) advertising models to determine which model matters the most, what moderators are most prominent, and what factors and sequence are most important in driving sales. Understanding the sequence of effects is most important for advertisers and marketers as they build their campaigns.
In this discussion for the track, Attribution & Approaches, session chair, Paul Donato (ARF) asked the speakers for their key insights on the drivers of short-term and long-term sales, the role of match control, and whether testing control should be part of attribution and ROI.
In this presentation, Pinkney illustrated several examples of ways in which missing data can be imputed when measuring audiences across platforms through analysis of “data dependencies.” Those dependencies in one known set of data can be leveraged through a model to generate estimates of unknown data.
Diana Saafi, Data Science Lead at Discovery, built on Cliff Young’s comments about the importance of multiple indicators for forecasting. Saafi forecasts linear television audiences in segments of interest to advertisers (beyond age-sex segments), rather than political outcomes. She and her team have found that their models have benefited from using multiple sources of data. She recommended identifying signals that are most predictive, experimenting with different types of models (such as ARIMA models and AI models), continually refreshing the data in the models, and continually updating the models. While this process is now automated at Discovery, there are people who monitor changes in the predictions, which she referred to as “human-in-the-loop automation.”
Rex Briggs, Founder & Executive Chairman of Marketing Evolution, who had developed forecasts of US COVID infections and deaths early in 2020, before it had been declared a pandemic, talked about the difficulties in “exponential forecasting” of events like a pandemic, compared to “stable state” forecasting of the impact of media, marketing, and creative. Just as companies might not have anticipated the impact of COVID-19 as it spread, they may be caught “flat-footed” in forecasting the exponential decay of the disease as vaccinations spread in 2021. Business forecasts of the impact of COVID in 2021 should take into account vaccination rates by age and surveys on acceptance of the vaccine.
Cliff Young, President of US Public Affairs at Ipsos, proposed that multiple indicators be used to forecast elections, not just data from the horse-race question alone. In particular, leaders’ approval ratings are strong predictors of their probabilities of winning, and Trump’s approval rating exceeded his horse-race preference in several swing states. Taking this variable and others (incumbency, perceptions of the most important problems facing the country) into account, Ipsos created a model based on results of over 800 elections across the globe. The model had predicted that a narrow Biden win with Republicans retaining control of the Senate was more likely than a Blue Wave.