Research & Data Quality

• Research & Data Quality
• Article

LOLA: Revolutionizing Content Experiments with LLM-Assisted Online Learning

In the rapidly evolving digital content landscape, media firms and news publishers require automated and efficient methods to enhance user engagement. This study introduces the LLM-Assisted Online Learning Algorithm (LOLA), a novel framework that integrates Large Language Models (LLMs) with adaptive experimentation to optimize content delivery. Leveraging a large-scale dataset from Upworthy, which includes 17,681 headline A/B tests, the study investigates three pure-LLM approaches and finds that prompt-based methods perform poorly, while embedding-based classification models and fine-tuned open-source LLMs achieve higher accuracy.

LOLA combines the best pure-LLM approach with the Upper Confidence Bound (UCB) algorithm to allocate traffic and maximize clicks adaptively. Numerical experiments on data from the website Upworthy show that LOLA outperforms the standard A/B test method, pure bandit algorithms and pure-LLM approaches, particularly in scenarios with limited experimental traffic. This scalable approach is applicable to content experiments across various settings where firms seek to optimize user engagement, including digital advertising and social media recommendations.

• Research & Data Quality
• Article

Marketing Analytics Accelerator: 2024

The Marketing Analytics Accelerator – the only event focused exclusively on attribution, marketing mix models and the science of marketing performance measurement – returned for its ninth year on November 13. The industry’s boldest and brightest minds joined us in NYC to share their latest innovations and case studies that will improve your business outcomes.

• Research & Data Quality
• Article

Improve Marketing Mix Model (MMM) Accuracy by Identifying these Effects

• MSI

This study explores the identification of nonlinear and time-varying effects in marketing mix models (MMM). It highlights the challenges of conflation in model selection and proposes a framework for simulating and estimating these effects using Gaussian processes. The study emphasizes the importance of accurately identifying the underlying response to optimize marketing spending.

The research provides insights into the complexities of marketing effectiveness and offers practical solutions for improving model accuracy. By addressing the issue of conflation, the study aims to enhance the decision-making process in marketing strategies.

• Research & Data Quality
• Article

Overcoming the Limitations of Online A/B Testing

• MSI

This study explores how divergent delivery in A-B testing affects the accuracy of online advertising experiments. It highlights the role of algorithmic targeting and user heterogeneity in confounding test results, offering guidance for marketers to improve their experimental designs.

• Research & Data Quality
• Article

In the Age of the ‘Modeled’ Sample: Can We Still Trust Polling?

• Mark Blumenthal – Head of Election Polling, SurveyMonkey

The 2016 U.S. presidential election served a big blow to the polling community, as pre-election predictions proved to be dramatically wrong. But is there really a crisis in pre-election polling? Mark Blumenthal, Head of Election Polling at SurveyMonkey, summarizes the AAPOR’s post-election investigation that demonstrated the accuracy of national polls, but weaknesses in state-level polls. Part of the challenge going forward – correcting for sample bias in an age of lower survey response rates.

• Research & Data Quality
• Article

Why Re-Testing “Truths” Is Good Practice

• Marla B. Royne, Great Oaks Foundation professor of marketing and chair at the U. of Memphis Fogelman College of Business & Economics Dept of Market & Supply Chain Mgt.
• JOURNAL OF ADVERTISING RESEARCH

Should the marketing community support repeating a study, when so much value is placed on new research? The answer may be a resounding “Yes.” Learn why repeating a study matters, especially when marketers seek transparency and data-based support for recommendations — and how advertising research might need to change.

• Research & Data Quality
• Article

The ARF Supports ANA’s Raising 2020 Census Concerns

Troubling. Bias. Distortion. Why would the ANA use these words in a letter to the U.S. Government expressing concern about the impact of the 2020 Census on marketing quality? Learn about what led them to act, why the ARF supports them, and possibly join them in sharing your views before the feedback period ends on August 7, 2018.

• Research & Data Quality
• Article

Making Mobile Listening More Insightful

• Dave Vannette
• Qualtrics

As more consumers respond to surveys on smartphones, fewer are completing them. That trend puts the representativeness of your sample at risk. Qualtrics has developed six best practices to optimize mobile survey design and thereby minimizing the respondent burden and maximizing your data quality.