research quality

Face Verification as Real-Time Solution for Survey Quality

Face Verification as Real-Time Solution for Survey Quality

Mihkel Jäätma – CEO, Realeyes

Scott Jones – VP of Product, Realeyes

Nick Sutton – Chief Strategy Officer, Kantar Profiles

Thirty percent of surveys are fake, based on an analysis of over half a million surveys. Quality issues have been prevalent for a long time now, and there is a need to address this. Survey panel quality has gotten worse because the marketplace does not value quality, fraudsters move faster than researchers and quality issues are not addressed adequately. Previous tools to deal with fraud are insufficient specifically for programmatic sample that is almost 70% of all surveys now. Quality issues are not visible enough, and there is need to create a transparency dashboard that is publicly available. Solution: face verification, which can be a game changer in survey quality. It is real time, cross-supply and simpler than CAPTCHA. The tool is established on properly obtained training data, that is tackling algorithmic bias head on and works everywhere on any device. There are different types of “bad actors”: the disengaged panelist, the dishonest panelist, the fraudulent panelist—the single biggest bad actor is from out of country. Following this there are bots, ghost completes and inconsistent answers (dishonest). There is not one single solution but rather a basket of solutions to make sure the quality is maintained. Key takeaways:
  • Widespread quality issues and deteriorating survey panel quality is a significant problem.
  • Existing tools for detecting and preventing survey fraud are insufficient, particularly for programmatic sample surveys.
  • To combat these quality and integrity issues, face verification is suggested. It stands out for being real-time, applicable across different survey supplies and more user-friendly than traditional CAPTCHA methods.

Download Presentation

Member Only Access
AUDIENCEXSCIENCE 2024

AUDIENCEXSCIENCE 2024

The ARF’s annual AUDIENCExSCIENCE conference highlighted the most critical audience measurement issues. Through keynotes, panels, debates and rigorously peer-reviewed research presentations, attendees learned about a wide array of new and evergreen industry topics, endemic to our industry changes. World-class thinkers joined us in NYC to share their perspectives on the future of advertising research and measurement, and how tomorrow’s technologies and data trends will impact advertising and media.

Member Only Access
Human Experience: Why Attention AI Needs Human Input

Human Experience: Why Attention AI Needs Human Input

Dr. Matthias Rothensee – CSO & Partner, eye square

Stefan Schoenherr – VP Brand and Media & Partner, eye square

Speakers Matthias Rothensee and Stefan Schoenherr of eye square discussed the need for a human element and oversight of AI. Beginning their discussion on the state of attention and AI, Matthias acknowledge that race for attention is one of the defining challenges of our time for modern marketers. He quoted author Rex Briggs, who noted the "conundrum at the heart of AI: its greatest strength can also be its greatest weakness." Matthias indicated that AI is incredibly powerful in recognizing pattern from big data sets but at the same time there are some risks attached to it (e.g., finding spurious patterns, hallucinations, etc.). Stefan examined a case study using an advertisement for the candy M&Ms, which considered real humans using eye tracking technology and compared it to results using AI. The goal was to better understand where AI is good at predicting attention and where does it still have to optimize or get better. Results from a case study indicated areas for AI improvements in terms of gaze cueing, movement, contrast, complexity and nonhuman entities (e.g., a dog). The static nature of AI (e.g., AI prediction models are often built based on static attention databases) can become a challenge when comparing dynamic attention trends. Key takeaways:
  • Predictive AI is good at replicating human attention for basic face and eye images, high-contrast scenes (e.g., probability of looking at things that stand out) and slow-paced scene cuts where AI can detect details.
  • AI seems unaware of a common phenomenon called the "cueing effect" (e.g., humans not only pay attention to people's faces but also to where they're looking), which leads to an incorrect prediction.
  • AI has difficulties deciphering scenes with fast movements (e.g., AI shows inertia) in contrast to slow-paced scenes where AI excels in replicating human feedback. In this case human feedback is more accurate.
  • AI is more consumed with attention towards contrast (e.g., in an ad featuring a runner, AI gave attention to trees surrounding the runner), whereas humans can decipher the main aspect of an image.
  • AI decomposes human faces (e.g., AI is obsessed with human ears), whereas humans can detect the focal point of a human face. In addition, AI hallucinates, underestimating facial effects.
  • AI has difficulties interpreting more complex visual layouts (e.g., complex product pack shots are misinterpreted).
  • AI is human centric and does not focus well on nonhuman entities such as a dog (e.g., in scenes where a dog was present, AI disregarded the dog altogether).
  • AI tends to be more static in nature (e.g., AI prediction models are often built based on static attention databases), which can be a problem when comparing this to dynamic attention trends.

Download Presentation

Member Only Access

Too Much Attention?

  • Posted: February 29, 2024

One expert argues that attention alone does not bring ad success and that we should not forget the other important levels of the “ARF Model.”

Read more »

The ARF Handbook for Using AI in Advertising Research

The ARF Handbook for Using AI in Advertising Research

Significant developments in AI have occurred in the last two years, allowing it to be used in various places in the advertising industry. One area that has received little attention however is advertising research. Recognizing this, the ARF has conducted a significant number of its own research. The product of this effort is an AI handbook that offers practical advice in several key aspects of using AI for advertising research. Moreover, an interactive function allows experts to leave comments that, once verified, will be integrated into the report, making it a living, breathing document that continues to evolve as AI advances.

Member Only Access
Navigating the Evolving Media Landscape

Navigating the Evolving Media Landscape

  • OTT 2023

The media landscape continues to evolve, arguably at a faster rate than ever. Leading media and measurement experts presented research-based insights on how viewers use different forms of TV/video on various platforms. Attendees joined us at the Warner Bros. Discovery Studios in California and via livestream to understand the latest data and discussions of the data’s implications.

Member Only Access
The Other Thread: The Debate About Cross-Media Deduplication

The Other Thread: The Debate About Cross-Media Deduplication

On June 28, media research pioneer and journalist Bill Harvey sent excerpts of his latest Media Village article to several of the industry’s leading media researchers. That article was entitled: Cross-Media Duplication Must Be Rigorously and Empirically Determined. What followed was a robust intellectual debate that outlines where the lines are drawn in the industry as far as whether virtual IDs (VIDs) can overcome issues of privacy and still provide an accurate account of campaign cross-platform duplication. We now share this jaunty and insightful exchange with our members.

Member Only Access

ATTENTION 2023

  • Posted: July 11, 2023

On June 7, 2023, attention economy experts came together in NYC to share case studies and participate in engaging discussions on the attention measurement landscape. Plus, attendees heard a recap of the issues debated at AUDIENCExSCIENCE and an update on Phase I of the ARF Attention Validation Initiative, an empirically based evaluation of the rapidly developing market for attention measurement and prediction.

Inclusion by Design in Pharma Research and Marketing

  • Posted: July 11, 2023

This ARF Pharma Council event followed up on the Council’s podcast episode on “Inclusive Futures of Humancare,” focusing on the importance of inclusiveness in pharma research and marketing with respect to both demographic characteristics and health conditions.  Four speakers delivered brief presentations, followed by a discussion moderated by Pharma Council Co-Chair Marjorie Reedy of Merck.

 Next »