In this session, Tina Daniels and Nicole Gileadi examined Google’s principles for charting the course for third-party cross-media audience measurement. Tina acknowledged more third-party measurement companies were expressing interest in working more closely with Google, given their stature as the world’s largest video provider. In her discussion, she acknowledged that this interest generated the need for Google to create a set of principles to offer to both measurement companies and key clients to guide the process. After reviewing these principles Tina and Nicole held an open discussion regarding these principles. Topics of the discussion included premium and high-quality content, long-form versus short-form video and the measurement of this content. In addition, Nicole touched on the importance of content and the context surrounding an ad. Other areas included the idea of exposure metrics (e.g., Where is my audience? Did I reach them?) in addition to providing signals to conduct an impact analysis.
Scott McDonald opened the session by discussing how the Census uses sample to correct for issues like undercounts in big data. Pete Doe (Nielsen) responded by commenting on persons who ask: do you have a Big Data solution or a panel solution? He doesn’t see it that way but rather you take all the signals you have and put them together in the best way for the problem at hand.
Jon Watts (CIMM) led a conversation with the CEOs of an organization that is helping to manage the JIC (OpenAP) and one that participates in it (the VAB), the EVP of an organization that does not belong to the JIC but has met with it and the CEO of the MRC. The participants clarified their relationships with each other, discussed Nielsen and expressed their hope for the future of television measurement.
Adopting AI usage into business functions seems to be the new trend, but can it be used to increase efficiency in market research? This study finds that it can help reduce costs and improve speed by automating some aspects of the process. One such way is to use Large Language Models (LLMs) as stand-ins for human survey respondents. The pool of respondents is shrinking. Moreover, the research finds that LLMs can realistically reflect consumer preferences because they have been trained on extensive online data. This process is also a fraction of the cost and time required with conventional methods.
Brittany Slattery (OpenAP), who opened this discussion, explained that the new JIC was created by national programmers and media agencies for three main purposes: (1) To bring buyers and sellers to the table with equal voices; (2) To create baseline requirements for cross-platform measurement solutions and (3) To create a harmonized census-level streaming service data set across all of the programmers in the JIC. Fox, NBCU, Paramount and Warner Brothers Discovery are all JIC members, as are Dentsu, Group M, IPG Mediabrands, OMG and Publicis. The members hope to foster competition among multiple ad video measurement currencies. After her introduction, Danielle DeLauro (VAB) moderated a discussion with the representatives of three networks and Group M.
This ARF Pharma Council event followed up on the Council’s podcast episode on “Inclusive Futures of Humancare,” focusing on the importance of inclusiveness in pharma research and marketing with respect to both demographic characteristics and health conditions. Four speakers delivered brief presentations, followed by a discussion moderated by Pharma Council Co-Chair Marjorie Reedy of Merck.
There are limitations in terms of measuring the emotional impact of video. Facial expressions are powerful. Neuroscience has proved that special parts of our brain is dedicated to faces. Development psychology also demonstrates the power of facial expressions. Consumer neuroscience also demonstrates the importance of faces. The question is whether Facial Action Coding System (FACS) is the best tool to measure consumers emotional response when viewing ads?
Nielsen IQ found that when you measure people’s facial expressions you need high quality videos, good lighting, algorithms. Nielsen IQ tests across over 2,000 ads in 15 countries. Initial R&D, global deployment, became a standardized diagnostic integrated with eye tracking. First phase found that facial expressions are only reproducible in strong and consistent moments. Happy is reproducible if it is really big, namely includes strong smiles. Negative emotions—reliability and reproducibility were as low as .25%. People watching TV, however, are mostly neutral. Reproducible expressions don’t occur often. Brain measure EEG, however, fluctuates much more and is highly dynamic. Put differently, there is a gap between facial expressions and EEG waves. Emotional expression does not equal emotion in the brain r=0.07.
Is FACS predictive of anything meaningful? NO! No significant relation between ad-driven sales lift and any emotional expression. Facial coding is not the right tool for the job of emotional response to video ads. It is not reproducible—when measured continuously, don’t get same answer twice; it is not sensitive—at reproducible levels, expressions are rare; it is not meaningful—expressions do not equal emotion and are not reflective of dynamic emotion in the brain; and not predictive—smiles do not equal sales and don’t correlate with outcomes.
Key Takeaways
Facial coding is not a good measure of measuring emotional response to video ads.
Only smiles were found to register in somewhat reliable manner with facial coding.
There is no significant relation between ad-driven sales lift and any emotional expression.
Johanna Welch – Global Mars Horizon Comms Lab Senior Manager, Mars
Mars and Realeyes prove connection between creative attention and sales performance. Mars’ Agile Creative Expertise (ACE) tool tracks visual attention and emotion responses to digital video ads. Visual attention AI and facial coding measures how long participants watch a video and how their attention changes as they watch. Proven this model to work—optimizing content, lifting sales up to 18% in 19 markets, $30 million in ad optimizations in 18 months.
ACE can 1. Predict sales accurately while learning how consumers behave and think. 2. Optimize—improve performance through creative selection. 3. Scale—establish a fast scalable solution. The model links attention, emotion and memory. Accordingly:
1. Attention is access to the brain and enables the brand to enter into consciousness.
2. Facial reactions—build memories.
3. Impact—higher consideration, conversions and sales.
ACE solution:
1. Participant exposure: 24-48 completion, 150-500 viewers from pool of +200m people. Observe people.
2. Attention detection: deep learning, collect viewer attention through natural viewer experience.
3. Actionable scores: ML and AI analytics to assess performance and deliver scores.
Company impact: validated predictions proved connections to behavioral and sales data via over 4,000 sales/ads data points and benchmarks. They also used ACE to improve performances for TikTok, Facebook, Instagram, YouTube, achieving 18% cumulative sales lift. Global scale—scored over 1,000 creatives in 18 months. In conclusion, ACE is the biggest attention database, received U.S. patent for visual attention detection. Mars hopes to share ACE with other companies. And the next step is how to take a pre-testing tool to in-flight content and to examine brand equity.
Key Takeaways
Establishing the connection between creative attention and sales performance is key.
Mars’ Agile Creative Expertise (ACE) tool tracks visual attention as well as emotional responses to digital video ads.
Proven this model to work—optimizing content, lifting sales.
How to measure attention? This presentation introduced MindProber—a tool that includes biometrics, physiology, conscious/dial AND survey methods to assess attention. MindProber measurement is passive—second by second emotional engagement is measured through electrodermal activity (EDA), aka galvanic skin response (GSR) and also active—measures cognitive response via an optional feature to indicate like/dislike content through app. N= 1,500 and growing to 3,000 by end of year.
Pedro Almeida (MindProber) emphasized three indicators of the value of metrics:
1. Validity—measures what it is supposed to measure. GSR is linearly related to perceived arousal in video formats. Measures emotional intensity (example of soccer game and spikes during key moments such as goals).
2. Reliability—metrics are stable across samples with 98.3% accuracy in ranking high vs. low impact ads.
3. Predictive power—metrics predict events of interest. Carry-over—if you’re more involved with the content you will respond more to ads. Premium content leads to higher advertising impact. Engagement with contents will carry over to engagement with ads. Emotional impact is indicator of likelihood of remembering the brand.
Generating macro insights through a robust taxonomy—analyzed 150+ sessions, watched 250+ hours of programming, monitored 20,000+ individual hours, 10,000+ individual participations, 22,700+ events tagged and 8,500+ ads. Ninety-eight percent of commercial activity is as engaging as the content.
Findings show 1. validity exists. 2. Reliability—98.3% accuracy in ranking high vs. low impact ads. 3. Predictive validity: A) Higher emotional impact for content = higher emotional impact for advertising. Carry-over—premium content leads to higher advertising impact; B) Higher emotional impact score = higher brand recall. Emotional impact is an indicator of likelihood of remembering the brand.
Lauren Zweifler (NBCU) showed that MindProber successfully identified the most crucial moments on both scripted and unscripted shows (Iconic holiday TV moments for the first and Top Chef and America’s Next Top Model for the latter) WITH validity, reliability and predictability.
What’s next?
How can we best predict lower funnel outcomes?
What elements of creative really matter?
How do we optimize for CTV?
Key Takeaways
New tool that captures emotional engagement through skin response.
Three fundamental metrics are validity, reliability and predictability.
Generated macro insights through a robust taxonomy. Power of premium video content yield strong emotional engagement (EE).
Because of the complexity of attention, Duane Varan proposes to focus on inattention and regard attention as absence of inattention. Attention is the threshold above which the cognitive processing of stimuli occurs. It is not linear; it’s best understood as occurring above a threshold (inattention) after which other variables can deliver their value. There are many different kinds of attention, and the different measures capture the different types of attention. Inattention, on the other hand, is a constant construct.
Phase 1 of the study focused on literature-proven measures (except fMRI) and found the following to be the four best methods: Heart rate (HR), visual fixations, blink duration, EEG (Alpha Wave). MediaScience and the Ehrenberg-Bass Institute concluded that eyes-on-screen is not best-in-class because 1. People are often paying attention even if not looking at screen; 2. Even when people look at the screen, they aren’t always paying attention, at least with fixations you see movements.
Phase 2 used ads as stimuli and found two measures worked best: heart rate and EEG.
Duane Varan pointed out that as an industry we have been fixating on eye tracking. But many ways of measuring eye tracking are not accurate AND they lose accuracy over time. There is a misalignment between what we are measuring (on which platforms) and what this means for different platforms. Even if accurate, eye tracking measures are not the best-in-class measure for attention.
While EEG is a great measure, it is not scalable. Heart rate, on the other hand, can scale.
Pilot project—massive in scale—50 programs, n>4000 using Stream Pulse platform; 1,576 in lab sessions and 2,437 in home sessions. Results demonstrate that HR via fitness trackers is viable with correlations as high as .81, but overall, this was only .32. Importantly, while promising, HR is a complex measure that necessitates more work to clean noise and harness full potential.
Next steps:
1. Continue mining existing data to devise strategies for identifying and cleaning noise factors.
2. Large pilot “in-the wild” using natural fitness tracker data combined with ACR.
Open questions: Where is attention being oriented to? Towards brand? If not, what’s the value in that? Then, thinking about what effect this has.
Finally, not all measures lend themselves to AI. Trained against what reference data—is it measuring attention or something else, this depends on the reference data. A lot of the companies are using sales lift measures—you need attention to get sales lift, but you don’t measure this as attention. Finally, with what validation?
Key Takeaways
Inattention should be the focus.
Eye tracking is not the best measure; heart rate and EEG are better but only heart rate is scalable.
Heart rate is viable and scales so there is promise in its application.
Be skeptical (but hopeful) of AI measure of attention. Proper scientific validation is necessary before we adopt them.
