fbpx

research quality

Best Practices in Media and Market Research Studies

  • Dr. Horst Stipp, EVP, Research & Innovation, the ARF
  • KNOWLEDGE AT HAND; CMO BRIEF

Surveys are a necessary tool for exploring consumer behavior, attitudes and intentions. They provide valuable data to help make informed business decisions. However, quality matters and ignoring best practices impacts the validity and reliability of findings, which might make the data unusable.

Member Only Access
  • Article

NYCU: Research Quality in Surveys

This recent,  ARF L.A. Media Research Council event featured survey experts who discussed steps researchers should take to avoid the problems that affected political polls during the 2020 election.     Research quality has always been a key issue for the ARF. Last week, an event organized by the ARF’s L.A. Media Research Council addressed concerns about how shortcomings in political polling could impact trust in consumer and media research and explored what research suppliers are doing to implement best practices. The issues were discussed by a panel of experts in survey research: Risa Becker, SVP Research, MRI-Simmons; Jon Giegengack, Principal, Hub; Leanna Wood, senior vice president, Global Operations, National Research Group; and Cliff Young, president US Public Affairs, Ipsos. The panel was moderated by Sumi Barry, SVP Content and Consumer Insights, Entertainment Networks NBCU, and member of the L.A. Media Research Council. The panel agreed that political polls deal with some unique problems (i.e., sample bias because of political polarization), but that there are challenges that impact all survey research:

  • Obtaining representative samples
  • Assuring the validity of responses
  • Predicting behavior and attitude trends
In addition, the researchers described how they dealt with the impact of the pandemic, such as changing from in-person to online methodologies. Sampling - The panel explained several approaches they employ to achieve representative samples: probability sampling, robust recruitment practices and methods to engage potential respondents who do not have internet access, when conducting online research. Finally, if a panel is used, they recommended methods to catch “professional survey takers”. Validity of responses - Even if survey questions are clear and objectively worded, respondents’ poor recall, “social desirability” and “snap judgments” can lead to inaccurate and distorted data. Therefore, the researchers stressed the importance of thoughtful questionnaire design and suggested to employ “Data Quality Solutions” such as verification questions, to ensure respondents are providing consistent answers and custom questions and to catch and terminate individuals who are over-reporting. Predicting behavior and attitudes - Since decision making, from voting to purchase behavior, is impacted by many factors, researchers should not rely on just one question. To improve prediction accuracy, researchers should develop and test decision making models. Finally, we were reminded that even when best practices are employed, error remains. And such error does not only come from the often-discussed sampling error. We also need to pay attention to non-sampling error to develop a good understanding of the accuracy and validity of the data. Source: ARF. (2021, January 27). L.A. Council: Best Practices in Media Studies – How Researchers Can Learn from Recent Political Polling Challenges. L.A. Media Council: The ARF.  

Member Only Access

Take the 2nd Annual ARF Organizational Benchmark Survey

  • ARF

The Organizational Benchmark Survey was developed to examine the dramatic changes taking place in advertising and marketing research organizations in recent years. Developed by the ARF Analytics Council, last year’s survey produced a set of reports covering the advertiser, agency, research company, media & entertainment and consultancy sectors. The ARF also released access to the raw data for members to interrogate themselves via our interface. (Note, you must download the free Tableau reader before doing so. Find instructions on how to do so here). These reports and the database have been helpful to members in understanding how their organization compares to others in their sector, with respect to management of the insights and data analytics functions.

Member Only Access

The Organizational Benchmark Survey Database

  • THE ARF

Our first, annual benchmark survey looked into the dramatic changes that occurred in advertising and marketing research organizations from 2018-2019. We have delivered a series of reports: the advertiser, agency, research, media & entertainment and consultancy report. Now, the ARF has released the interactive access via a searchable database. Members can interrogate the data and findings themselves and query segmentations such as job title, category, structure, size and investment trends.

Member Only Access

The 1st Annual Organizational Benchmark Survey—The Consultancy Report

  • THE ARF

The Organizational Benchmark Survey investigates the changes in advertising and marketing research over the past year. This is the first of an annual series. The consultancy report covers a variety of subjects, including the names of their research departments and their structures, whether they are centralized or decentralized, their spending and KPIs, what skills researchers need and what tools they employ, and how satisfied they are with their department.

Member Only Access

Key Planning Tool Gets a Facelift for the Digital Age

  • JOURNAL OF ADVERTISING RESEARCH

A key planning tool that is widely used by advertising professionals for strategic and creative guidance, is seriously out of date, according to a forthcoming Journal of Advertising Research study available online on the JAR’s Digital First platform. Researchers in Japan and South Korea propose a refresh to the Foote, Cone & Belding (FCB) grid, created in 1980, to reflect current-day products and consumers’ online purchase decisions.

Member Only Access

The 1st Annual Organizational Benchmark Survey—Media & Entertainment Report

  • The ARF

The Organizational Benchmark Survey investigates the changes in advertising and marketing research over the past year. This is the first of an annual series. The media and entertainment report covers a variety of subjects, including the names of their research departments and their structures, whether they are centralized or decentralized, their spending and KPIs, what skills researchers need and what tools they employ, and also how satisfied they are with their department.

Member Only Access

Novel Approach Improves Marketing Survey Data Accuracy

  • WEDNESDAY WEBCAST

How can advertisers, media companies and market researchers acquire highly accurate consumer survey data, in order to make informed business decisions, without depleting their budget? Sure, everyone would like to utilize probability sampling, but the cost is out of reach for most. However, non-probability, opt-in, online samples are nowhere near as accurate. Editor’s Note: The full report is available to members only.

Member Only Access

Getting Better Creative from Data and Insights

  • CREATIVE COUNCIL WHITE PAPER

We have access to more data and research than ever before. But how well are they being used? It’s apparent to many in the advertising industry that there’s a gap between the research/data and creative/strategy communities, in terms of using data and insights effectively. To investigate this divide, the ARF Creative Council conducted a quantitative and qualitative research project, which has led to the publication of a new white paper titled: How to Get Better Creative from Better Insights.

Member Only Access