On May 17, the ARF Analytics Council explored the groundbreaking concept of Virtual IDs (VIDs) and their potential to revolutionize cross-media measurement. The essential mechanics of VIDs were explained in a non-technical manner to help professionals across media and advertising understand it better. Panelists shared how VIDs could overcome barriers in calculating cross-media and device reach and frequency.
Matt Weinman – Senior Director of Product Management, Advanced Advertising Product, TelevisaUnivision
Matt Weinman (TelevisaUnivision) and Spencer Lambert (datafuelX) shared the methodology and results from testing TelevisaUnivision’s initiative that, with datafuelX’s technology, enabled their advertising partners to choose their preferred currency in forecasting both long- and short-term audiences for their programming. Implementation involved adjusting the business flow for multi-measurement sources but with each source ingested, validated and normalized to the tech standard separately.
Forecasting incorporated a programming schedule imputation process which was then fed into a mixed model estimation (MME), and then optimized with linear granular data.
Their model revealed gaps that they addressed with a variety of tactics including a ratings adjustment approach that updated network viewership trends, a proportional weight method for advanced audiences, recency weighting to avoid stale rate cards and relying less on forecasting viewers rather than scheduled content. The MME drove strong predictive forecasts and increased the use of long-tail inventory.
Key Takeaways
Forecasting should always be done based on content.
In reviewing the accuracy of predicting exact programming, the forecast to actuals had a 71% program match. In predicting programming type, there was 94% accuracy.
Results for the long-term audience forecasts had 42% MAPE (mean absolute percentage error) improvements overall using big data sources.
Daniel Sacks – Incrementality Lead, US Agency, Google
Brendan Kroll of Nielsen and Anne Ori and Daniel Sacks, both of Google, explained that their study’s objective was to identify potential improvements to marketing mix models by utilizing enhanced prior beliefs (priors) based on sales lift studies and exploring the resulting changes in campaign-level sales lift once those priors were incorporated.
Incrementality experiments are widely accepted as the gold standard for causal measurement. Calibrating individual channels via experimentation ensures optimization of model outcomes. However, the results of incrementality experiments are often not part of marketing mix model (MMM) design.
Nielsen utilized NCS sales lift studies as the source of the experimental data for this analysis. NCS determined the causal effects of advertising on incremental sales while controlling for targeting and other co-variates.
The study design involved 10 brands with existing MMMs and available NCS results for corresponding periods, model re-estimation using NCS lift priors, refinement of the priors and scaling. This study showed that applying this methodology to a YouTube campaign resulted in significant sales lift, as well as revenue and ROAS increases, including a 2.6x median increase in the effectiveness in the adjusted model.
The adjusted model showed greater marketing contribution overall; therefore, marketers are at risk of undervaluing their overall marketing if experimental results are not included.
Key Takeaways
Brands can effectively leverage experiment-based priors to strengthen marketing mix models.
For nascent channels, the inclusion of experimentation results proved fundamental, especially if those campaigns showed strong initial results, since MMMs cannot rely solely on historical anchoring to measure true impact.
When experiments reveal high performing channels or campaigns, the use of testing can aid more accurate MMM measurements as investment scales.
Even for channels with long histories and relative stability, experimentation can serve as a way to validate models and may give models a chance to remain flexible in case of strategic shifts and/or changes in consumer behavior.
Nielsen compared the implied ratings from ACR data and STB data in homes where they also have meters. The correlation was quite high, though panel adjustments raised the rating levels by about 1%. Big Data are limited in different ways: not all sets in a house provide ACR or STB data, they are devoid of persons information and STB’s are often powered up but the TV is off. Nielsen presented how a panel of 40,000 homes can be used to correct those biases. A critical finding was that projection of MVPD data outside of its geographic footprint significantly changed network shares. That said, Big Data can significantly improve local market data where samples are necessarily much smaller.
Key Takeaways
Nielsen presented their view on how to best use Big Data. Nielsen uses a wide variety of data sets as part of its Big Data solution. It identified several gaps in the use of ACR data, including when native apps are used and in cases where channels are not monitored.
Nielsen has about 30 million RPD and ACR homes and a panel of 40,000 homes which it uses to adjust its Big Data. Where possible, it matches its panel homes to RPD/ACR in those same homes. The r2 with the panel data is .98 and .96 respectively, which is quite good. However, using the panel to adjust anticipated missing data, raises the overall viewing levels by about 1%.
However, there are other TVs in the homes that are not ACR capable and there is no persons data associated with ACR only homes. The panel and Experian are used to model who is in the household and the panel and Gracenote are used to model who among them would be viewing.
Similar modeling and correction procedures are used for STB data. However, one of the most significant adjustments of STB data is modeling when the TV is off but the STP is still powered on.
Nielsen uses a fusion process to conduct its viewer assignment, the process of modeling who is viewing when the set is known to be tuned. A simple way to understand this is to look for a similar panel home and assign the viewing characteristics of the panel home to the ACR or RPD home. The use of Gracenote has made a significant improvement in the viewer assignment model.
Nielsen showed data on the shifts in share that occur when you take a Big Data set like an MVPD’s RPD data and project it beyond the footprint of that MVPD. The share shifts are much greater than when one limits the projection to the geographic footprint of that MVPD.
Statistically, when one compares the differences between the shares projected within and not within the footprint, 28% of the shares when projected beyond the footprint of the MVPD were different from the panel at the .05 level of statistical significance.
However, Nielsen showed how much Big Data can eliminate quarter-hours with zero ratings within local market (where zero ratings reduced from 62 to 0 in one market) by including Big Data in local market ratings.
Sergey Fogelson – Head of Data Science, TelevisaUnivision
Edouardo Vitale – Data Scientist, TelevisaUnivision
Sergey Fogelson and Edouardo Vitale, both from TelevisaUnivision, outlined their motivations for developing a custom lookalike model (LAM) to expand Spanish-language audiences, which were under-represented:
Misidentification: 4 in 10 Hispanics are excluded from 3p datasets.
Waste: 70% of impressions targeted at Hispanics are wasted.
Scale: The true scale of the Hispanic population within a given brand’s 1p dataset is hard to identify without extensive validation.
In order to address this audience underrepresentation, data sources were leveraged to create a household graph incorporating 1P (first-party) viewership, 3P (third-party) viewership and demographics on age, gender, income and education from TelevisaUnivision’s partner. The combination of these sources created a robust household level representation of approximately 17MM Hispanic households in the U.S.
Embeddings were used to create a latent space that allowed for the comparison of user similarities. Mathematically, very similar users have very similar embeddings. Similarities between individuals may be based on what content they viewed, where the content was viewed (zip code) or demographics of the viewer. Additional details of the embedding process as well as the autoencoder architecture steps and validation process were presented.
Key Takeaways
Developing a Lookalike Model (LAM) to expand Spanish-language audiences, corrected for the underrepresentation of this consumer target.
Expanding an audience with LAM identifies individuals who look and act just like a given target audience. These look-alike models are used to build larger audiences from smaller segments in order to create reach for marketers and advertisers and enable them to transact on an expanded audience.
Use of LAM can overcome the challenges of misidentification, waste and scale. LAM plus the household graph achieves significant increases in overall audience scale.
Editors at the Journal of Advertising Research have launched two new calls for papers on timely themes — one on immersive technologies and the other on prosocial advertising messages. Interested in contributing your robust work in these areas? Read on to learn about goals for each call and the journal’s resources and support toward submitting a successful paper.
While privacy regulations such as the European Union’s General Data Protection Regulation (GDPR) can potentially benefit society, they must strike an effective balance between firms’ need to target their messages and consumers’ need for privacy and security. Modeling “what if” scenarios can help marketers and policymakers find that equilibrium.
On October 13, 2022, industry leaders joined us for an inaugural Creative Effectiveness event on how to stimulate great creative and measure it. Panelists shared innovative and experiential formats as well as new approaches and tools in support of building and measuring campaigns. The event was followed by the ARF David Ogilvy Awards reception.
On July 12, 2022, forecasting, and product experts shared frameworks and strategies for participants to consider as they plan amid disruptions in the industry. Presenters discussed techniques marketers could use to drive consumer action and advocacy — as well as econometric models for search trends, insights on holistic analytics programs, reflections on gold standard probability methods — and new forecasting techniques in the wake of the pandemic and more.
