modelling

How to Calculate Reach and Frequency Using Virtual IDs (VIDs)

Current existing methods used to calculate reach and frequency of a campaign or media schedule are known to have deficiencies in measuring cross-device ad exposure. Restrictions to protect digital privacy complicate cross-platform exposure measurement even further. Multiple global research organizations have turned to a concept known as “virtual people,” to overcome these limitations in order to produce aggregate reach and frequency estimates. This report by the ARF Analytics Council provides a foundational overview of VIDs for a broad audience, providing ARF members with a stronger understanding of this vital topic.

Member Only Access

The ARF Attention Measurement Validation Initiative: Phase 1 Report Updated

  • ARF ORIGINAL RESEARCH

Attention metrics have drawn a high degree of energy in the last few years, for many reasons, including the loss of behavioral signals due to privacy restrictions, growing frustration with ad viewability and its perceived limitations, attention metrics’ impact on the cross-platform measurement debate and that biometric technologies can now be applied “in the wild,” rather than just in labs. The ARF’s Attention Measurement Validation Initiative aims to describe the attention measurement space in detail, illuminating this nascent sector. The Phase One findings include a comprehensive literature review and a report that maps out the vendor landscape in this increasingly diverse specialty. The report includes two sections. The first section describes what methods are being used, what these companies report and how and what they measure, be it ad creative or the media environment. The second section includes in-depth overviews of the 29 participating attention measurement companies. The Phase One Report is a must-read for anyone interested in attention metrics or what companies are operating in the space.  

Member Only Access

Business Outcomes in Advertising Powered by Machine Learning

Brett MershmannSr. Director, Research & Development (R&D), NCSolutions

Brett Mershmann’s (NCSolutions) discussion focused on how to quantify incremental advantages of some more modern contemporary machine learning (ML) frameworks, over more traditional measurements for incrementality. Beginning the presentation, Brett provided an overview of both traditional modeling techniques as well as more contemporary ML campaign measurements. To understand the differences, Brett detailed an 11-experiment process, using real observational household data, intersected with real campaign impression data but with simulated outcome and with a defined outcome function. The experiments measured accuracy, validity and power. Additionally, they compared ML with randomized controlled trials (RCTs), noting that RCTs are the gold standard but are not always feasible. To accomplish this, they ran both an RCT and an ML analysis, by creating test-control groups on real, limited data. This experiment applied the same outcome function to each, depending on a larger set of variables. In closing, Brett shared feedback from these experiments, which supported ML as a powerful method of measurement and a viable alternative to RCTs. He highlighted the importance of getting the correct data into these models for optimum results. Key takeaways:
  • A survey from the CMO Council indicated that 56% of marketers want to improve their campaign measurement performance in the next 12 months.
  • Traditional campaign measurement techniques use household matching (Nearest-Neighbor), household matching (Propensity) and inverse propensity weighting (IPW), based on simple statistical models applied uniformly. This method simulates balanced test and control groups to estimate the group-wise counterfactual.
  • The ML measurement technique, using NCSolutions’ measurement methodology, is computationally robust for large, complex data sets, understanding that data is not one-size-fits-all and estimates counterfactual for individual observations.
  • Simple A/B testing does not capture the true effect, while the counterfactual approach uses a "what-if model" approach to estimate the true effect.
  • The experiments comparing ML to traditional methods, measuring accuracy, validity and power showed that:
    • Accuracy: Machine learning outperforms on accuracy 55% compared to inverse propensity weighing (9%), propensity match (27%) and nearest-neighbor match (8%).
    • Validity: Percentage of scenarios with true effect in confidence interval (validity) found that ML gave valid estimates most often (91%), compared with inverse propensity weighing (82%), propensity match (64%) and nearest-neighbor match (73%).
    • Power: Machine learning is more statistically powerful. The average width of confidence interval using machine learning was 1.48, compared to inverse propensity weighing (1.56), propensity match (1.78) and nearest-neighbor match (1.72).
  • Results from ML vs. RCTs: Both ML and RCT are accurate in campaign measurement, both methods are generally valid, but ML is more powerful.
    • Overall, ML can be an adequate substitute for RCTs providing meaningful estimates when running an RCT is not a possibility.

Download Presentation

Member Only Access

The Power of AI for Effective Advertising in an ID-free World

Rachel GantzManaging Director, Proximic by Comscore

Amidst heightened regulations in the advertising ecosystem, Rachel Gantz of Proximic by Comscore delved into a discussion of diverse AI applications and implementation tactics, in an increasingly ID-free environment, to effectively reach audiences. Rachel highlighted signal loss as a "massive industry challenge," to provide a framework for the research she examined. She remarked that the digital advertising environment was built on ID-based audience targeting, but with the loss of this data and the increase in privacy regulations, advertisers have placed their focus on first-party and contextual targeting (which includes predictive modeling). In her discussion, she focused on the many impacts predictive AI is having on contextual targeting, in a world increasingly void of third-party data, providing results from a supporting experiment. The research aimed to understand how the performance of AI-powered ID-free audience targeting tactics compared to their ID-based counterparts. The experiment considered audience reach, cost efficiency (eCPM), in-target accuracy and inventory placement quality. Key takeaways:
  • Fifty to sixty percent of programmatic inventory has no IDs associated with it and that includes alternative IDs.
  • Specific to mobile advertising, many advertisers saw 80% of their IOS scale disappear overnight.
  • In an experiment, two groups were exposed to two simultaneous campaigns, focused on holiday shoppers. The first group (campaign A) was an ID-based audience, while the second group was an ID-free predictive audience.
    • Analyzing reach: ID-free targeting nearly doubled the advertisers’ reach, vs. the same audience, with ID-based tactics.
    • Results from cost efficiency (eCPM): ID-free AI-powered contextual audiences saw 32% lower eCPMs than ID-based counterparts.
    • In-target rate results: Significant accuracy was confirmed (84%) when validating if users reached with the ID-free audience matched the targeting criteria.
    • Inventory placement quality: ID-free audience ads appeared on higher quality inventory, compared to the same ID-based audience (ID-free 27% vs. ID-based 21%).

Download Presentation

Member Only Access

Inside the Journal of Advertising Research: Sonic Branding, ASMR Engagement, and Who Wins in Activist Messaging?

  • JOURNAL OF ADVERTISING RESEARCH

At this Insights Studio, researchers in Europe, the U.K. and the U.S. presented work in relatively new fields that have high-impact potential for the advertising industry. Starting with a forthcoming paper on sonic branding, the authors described their ground-breaking framework for measuring the implicit effects of sonic branding using music to manipulate visual scenes in video, film and TV. Next, a deep dive into autonomous sensory meridian response (ASMR)—a sensory-inducing device in ads—included strategies for helping brands collaborate with successful ASMR influencers. Lastly, a preview of an article to be published in the March Prosocial Advertising Special Issue showed how brand activism influences attitudes and purchase intentions, revealing a credibility gap between established activist brands and brands emerging in that space. Taking questions from Paul and from attendees, panelists in the concluding Q&A explored links between sonic branding and ASMR, the demographics of ASMR followers, ways for emergent activist brands to close the credibility gap with established activist brands, and future research possibilities.

Member Only Access

2023 Attribution & Analytics Accelerator

The Attribution & Analytics Accelerator returned for its eighth year as the only event focused exclusively on attribution, marketing mix models, in-market testing and the science of marketing performance measurement. The boldest and brightest minds took the stage to share their latest innovations and case studies. Modelers, marketers, researchers and data scientists gathered in NYC to quicken the pace of innovation, fortify the science and galvanize the industry toward best practices and improved solutions. Content is available to event attendees and ARF members.

Member Only Access

Navigating the Evolving Media Landscape

  • OTT 2023

The media landscape continues to evolve, arguably at a faster rate than ever. Leading media and measurement experts presented research-based insights on how viewers use different forms of TV/video on various platforms. Attendees joined us at the Warner Bros. Discovery Studios in California and via livestream to understand the latest data and discussions of the data’s implications.

Member Only Access

Advertising Effectiveness: Performance Measurement in the New World of Privacy and Tools

On July 26, measurement practitioners discussed how to adapt to this new era of privacy with tools for measuring ad performance effectiveness. Panelists explored new considerations for existing methods, such as marketing mix modeling (MMMs) and multi-touch attribution (MTAs), and discussed the pros and cons of various privacy enhancing technologies (PETs), including multi-party computation, clean rooms, and more.