research methods

In-Home Psychophysiological Television Measurement

In-Home Psychophysiological Television Measurement

Pedro Alemida – CEO, MindProber

Lauren Zweifler – SVP Insights & Research, Client Strategy & Insights Org, NBCU



How to measure attention? This presentation introduced MindProber—a tool that includes biometrics, physiology, conscious/dial AND survey methods to assess attention. MindProber measurement is passive—second by second emotional engagement is measured through electrodermal activity (EDA), aka galvanic skin response (GSR) and also active—measures cognitive response via an optional feature to indicate like/dislike content through app. N= 1,500 and growing to 3,000 by end of year. Pedro Almeida (MindProber) emphasized three indicators of the value of metrics: 1. Validity—measures what it is supposed to measure. GSR is linearly related to perceived arousal in video formats. Measures emotional intensity (example of soccer game and spikes during key moments such as goals). 2. Reliability—metrics are stable across samples with 98.3% accuracy in ranking high vs. low impact ads. 3. Predictive power—metrics predict events of interest. Carry-over—if you’re more involved with the content you will respond more to ads. Premium content leads to higher advertising impact. Engagement with contents will carry over to engagement with ads. Emotional impact is indicator of likelihood of remembering the brand. Generating macro insights through a robust taxonomy—analyzed 150+ sessions, watched 250+ hours of programming, monitored 20,000+ individual hours, 10,000+ individual participations, 22,700+ events tagged and 8,500+ ads. Ninety-eight percent of commercial activity is as engaging as the content. Findings show 1. validity exists. 2. Reliability—98.3% accuracy in ranking high vs. low impact ads. 3. Predictive validity: A) Higher emotional impact for content = higher emotional impact for advertising. Carry-over—premium content leads to higher advertising impact; B) Higher emotional impact score = higher brand recall. Emotional impact is an indicator of likelihood of remembering the brand. Lauren Zweifler (NBCU) showed that MindProber successfully identified the most crucial moments on both scripted and unscripted shows (Iconic holiday TV moments for the first and Top Chef and America’s Next Top Model for the latter) WITH validity, reliability and predictability. What’s next?
  • How can we best predict lower funnel outcomes?
  • What elements of creative really matter?
  • How do we optimize for CTV?

Key Takeaways

  • New tool that captures emotional engagement through skin response.
  • Three fundamental metrics are validity, reliability and predictability.
  • Generated macro insights through a robust taxonomy. Power of premium video content yield strong emotional engagement (EE).

Download Presentation

Member Only Access
Why Visual Attention is a Flawed Measure of Attention

Why Visual Attention is a Flawed Measure of Attention

Duane Varan, Ph.D. – CEO, MediaScience



Because of the complexity of attention, Duane Varan proposes to focus on inattention and regard attention as absence of inattention. Attention is the threshold above which the cognitive processing of stimuli occurs. It is not linear; it’s best understood as occurring above a threshold (inattention) after which other variables can deliver their value. There are many different kinds of attention, and the different measures capture the different types of attention. Inattention, on the other hand, is a constant construct. Phase 1 of the study focused on literature-proven measures (except fMRI) and found the following to be the four best methods: Heart rate (HR), visual fixations, blink duration, EEG (Alpha Wave). MediaScience and the Ehrenberg-Bass Institute concluded that eyes-on-screen is not best-in-class because 1. People are often paying attention even if not looking at screen; 2. Even when people look at the screen, they aren’t always paying attention, at least with fixations you see movements. Phase 2 used ads as stimuli and found two measures worked best: heart rate and EEG. Duane Varan pointed out that as an industry we have been fixating on eye tracking. But many ways of measuring eye tracking are not accurate AND they lose accuracy over time. There is a misalignment between what we are measuring (on which platforms) and what this means for different platforms. Even if accurate, eye tracking measures are not the best-in-class measure for attention. While EEG is a great measure, it is not scalable. Heart rate, on the other hand, can scale. Pilot project—massive in scale—50 programs, n>4000 using Stream Pulse platform; 1,576 in lab sessions and 2,437 in home sessions. Results demonstrate that HR via fitness trackers is viable with correlations as high as .81, but overall, this was only .32. Importantly, while promising, HR is a complex measure that necessitates more work to clean noise and harness full potential. Next steps: 1. Continue mining existing data to devise strategies for identifying and cleaning noise factors. 2. Large pilot “in-the wild” using natural fitness tracker data combined with ACR. Open questions: Where is attention being oriented to? Towards brand? If not, what’s the value in that? Then, thinking about what effect this has. Finally, not all measures lend themselves to AI. Trained against what reference data—is it measuring attention or something else, this depends on the reference data. A lot of the companies are using sales lift measures—you need attention to get sales lift, but you don’t measure this as attention. Finally, with what validation?

Key Takeaways

  • Inattention should be the focus.
  • Eye tracking is not the best measure; heart rate and EEG are better but only heart rate is scalable.
  • Heart rate is viable and scales so there is promise in its application.
  • Be skeptical (but hopeful) of AI measure of attention. Proper scientific validation is necessary before we adopt them.

Download Presentation

Member Only Access
Creators Drive Commerce

Creators Drive Commerce

Vita Molis – Head of Research & Insights, NA & Global Functions, TikTok

Abby Thorsgaard – Research & Insights Manager, TikTok

Vita Molis and Abby Thorsgaard, both from TikTok, discussed the reasons that TikTok’s creators are impactful, drive full funnel metrics for brands and are a good investment for advertisers. Consumers view TikTok creators as authentic and informative. Not only do people learn from creators in a fun and engaging format, they also trust and relate to them. Sixty percent of consumers find TikTok creators more influential than celebrities. Creator content on TikTok is entertaining and on-brand. The real and approachable persona of TikTok creators is transferable to brands. For example, one in two consumers accurately recalled the brand after exposure to a creator’s ad. Additionally, after watching creator content on TikTok, consumers said they felt like part of the creator’s community, as well as part of the brand’s community. Research methodology across four studies included quantitative surveys, in-context exposure, advanced analytics, eye tracking and qualitative research. A max diff exercise helped to define how creators drive commerce through the lens of the specific attributes of creators and their content. Research partners were Hotspex, Lumen, MarketCast and Material. This research enabled TikTok to prove that consumers rely on creators and their recommendations to learn about brands and to make purchase decisions. Importantly, this research also validated TikTok’s hypothesis that creators drive commerce. Sixty-five percent of TikTok users said they always rely on creator recommendations to decide what to buy online. The research also defined creative best practices for brands working with TikTok creators. Brands should leverage creator ads to inform and entertain. Consumers were motivated by ads showing the product, utilizing music and having a strong branding presence, including logos. Additionally, creator ads which featured a call to action saw a 205% uplift in purchase intent.

Key Takeaways

  • Creators on TikTok drive commerce and brands can leverage the influence of these creators to achieve full funnel metrics for brands.
  • Build relationships with brand-aligned creators to tap into their community credibility.
  • Collaborate with creators to create content that demonstrates value of the brand’s product or service.
  • Leverage an “always engaged” strategy, which includes ads, organic content, working with creators and leaning into the trends and communities on TikTok. This strategy enables brands to maximize their return.

Download Presentation

Member Only Access
AUDIENCExSCIENCE 2023

AUDIENCExSCIENCE 2023

The ARF hosted its annual flagship conference, AUDIENCExSCIENCE 2023, on April 25-26, 2023. The industry’s biggest names and brightest minds came together to share new insights on the impact of changing consumer behavior on brands, insights into TV consumption, campaign measurement and effectiveness, whether all impressions are equal, join-up solutions across multiple media, the validity, reliability and predictive power of Attention measures, targeting diverse audiences, privacy’s effect on advertising and the impact of advertising in new formats. Keynotes were presented by Tim Hwang, author of Subprime Attention Crisis, Robert L. Santos of the U.S. Census Bureau, Brian Wieser of Madison and Wall, LLC and Andrea Zapata of Warner Bros. Discovery.

Member Only Access
Expanding Spanish Language Audiences

Expanding Spanish Language Audiences

Sergey Fogelson – Head of Data Science, TelevisaUnivision

Edouardo Vitale – Data Scientist, TelevisaUnivision

Sergey Fogelson and Edouardo Vitale, both from TelevisaUnivision, outlined their motivations for developing a custom lookalike model (LAM) to expand Spanish-language audiences, which were under-represented:
  • Misidentification: 4 in 10 Hispanics are excluded from 3p datasets.
  • Waste: 70% of impressions targeted at Hispanics are wasted.
  • Scale: The true scale of the Hispanic population within a given brand’s 1p dataset is hard to identify without extensive validation.
In order to address this audience underrepresentation, data sources were leveraged to create a household graph incorporating 1P (first-party) viewership, 3P (third-party) viewership and demographics on age, gender, income and education from TelevisaUnivision’s partner. The combination of these sources created a robust household level representation of approximately 17MM Hispanic households in the U.S. Embeddings were used to create a latent space that allowed for the comparison of user similarities. Mathematically, very similar users have very similar embeddings. Similarities between individuals may be based on what content they viewed, where the content was viewed (zip code) or demographics of the viewer. Additional details of the embedding process as well as the autoencoder architecture steps and validation process were presented.

Key Takeaways

  • Developing a Lookalike Model (LAM) to expand Spanish-language audiences, corrected for the underrepresentation of this consumer target.
  • Expanding an audience with LAM identifies individuals who look and act just like a given target audience. These look-alike models are used to build larger audiences from smaller segments in order to create reach for marketers and advertisers and enable them to transact on an expanded audience.
  • Use of LAM can overcome the challenges of misidentification, waste and scale. LAM plus the household graph achieves significant increases in overall audience scale.

Download Presentation

Member Only Access
Context Matters

Context Matters

Heather Coghill – VP, Audience, Warner Bros. Discovery

Daniel Bulgrin – Director, Research Operations & Insights, MediaScience

Heather Coghill (WBD) and Daniel Bulgrin (MediaScience) shared methodologies and results from two in-lab studies that sought to understand how impactful category priming can be without brand mention and if viewers associate brands with adjacent unsuitable content. Their presentation focused on two types of contextual effects within program context—“excitation transfer” and “brand priming”. To see if these effects carried over to ad content through excitement or brand recognition in the content, the research team utilized distraction-free viewing stations that enabled neurometrics and facial coding followed by post-exposure surveys. Impact on brand perception was measured with lifts in brand attitude, attention and memory. Results showed brand priming did change how viewers experienced the ad by lifting brand recognition, with stronger effects in heavier ad loads. The research also concluded that although brands are not harmed by adjacency to perceived unsuitable content, context effects still need to be considered.

Key Takeaways

  • Even moderate category primes can push through effects, despite modest impact, in both linear and CTV. Category priming in streaming with limited ads impacted middle and lower funnel metrics, with 31% of viewers noticing a connection between the ad and the program.
  • Although viewers agreed that low intensity “unsuitable” content was most acceptable for advertisers, there were no adverse effects as intensity levels increased—all levels were deemed suitable for advertisers, with no significant differences in brand recall or purchase intent.
  • More research is required to understand what is unsuitable for brands. The current guidelines are based on what is thought to be unsuitable—not social science.

Member Only Access
A Peek Behind the Curtain – Preventing a Fumble: The Research POV on Super Bowl Advertising

A Peek Behind the Curtain – Preventing a Fumble: The Research POV on Super Bowl Advertising

  • By Jillian Rice, Ipsos (Creative Council Young Pros Officer)

The ARF Creative Council hosted its seventh Happy Hour event on Super Bowl advertising and what to watch for in the commercial breaks during this year’s big game. Council member Belle Frank of VMLY&R moderated a discussion with four creative experts – three of them from the Creative Council -- to discuss Super Bowl ad insights and trends, giving marketers important ideas to consider and advertising fans something to look for on Sunday night.

Member Only Access
<em>JAR</em> Calls for Papers: Immersive Technologies & Prosocial Advertising

JAR Calls for Papers: Immersive Technologies & Prosocial Advertising

  • JOURNAL OF ADVERTISING RESEARCH

Editors at the Journal of Advertising Research have launched two new calls for papers on timely themes — one on immersive technologies and the other on prosocial advertising messages. Interested in contributing your robust work in these areas? Read on to learn about goals for each call and the journal’s resources and support toward submitting a successful paper.

« Prev Next »