Analytics & Data Science

• Analytics & Data Science
• Article

People’s Performance Goals Shape Their Use of Predictive Algorithms

This study presents a framework for understanding people’s use of predictive algorithms, emphasizing their role as tools designed to support human decision-making. It argues that users’ performance expectations are a primary driver of their decisions to adopt these algorithms. By reviewing and reinterpreting the literature through the lens of laypeople’s performance expectations, the study aims to clarify why some algorithms are accepted and others are rejected. It concludes by suggesting avenues for designing algorithms that better meet users’ expectations, enhancing their usability and acceptance.

• Research & Data Quality
• Article

AI for Data Analysis in Marketing and Advertising

Attendees joined ARF in NYC for two workshops – a morning session tailored for professionals without a background in coding, and an afternoon session for data scientists, analysts, engineers, and other data science professionals – that highlighted how to utilize LLMs to analyze example datasets. Additionally, attendees had the chance to participate in real-time AI tool demonstrations.

• Past Event Highlights
• Article

The State of the Insights and Analytics Disciplines 2024

• By John Baro, Young Pros Officer
• Organizational Council

On October 30th, the ARF’s Organizational Council presented the results of the 2024 Organizational Benchmark Survey. This 3rd wave of the Benchmark Survey followed the 1st and the 2nd waves, which took place in 2019 and 2021, respectively. Council Chair Susan Pizzaro’s presentation touched on trends in research department structures, budgets and resources; changes in valued skills and tools used; and satisfaction with the value brought by insights and analytics teams. Afterward, Becky Bach of Pernod Ricard USA and Jim Spaeth of Sequent Partners joined Susan in a discussion moderated by ARF’s Chief Research Officer Paul Donato.

• Research & Data Quality
• Article

An Introduction to Robyn’s Open-Source Approach to Media Mix Modeling

• MSI

As privacy-centric changes reshape the digital advertising landscape, deterministic attribution and measurement of advertising-related user behavior are increasingly constrained. In response, there has been a resurgence in the use of traditional probabilistic measurement techniques, such as media and marketing mix modeling (m/MMM), particularly among digital-first advertisers. To address the gap for small and midsize businesses, marketing data scientists at Meta have developed the open-source computational package Robyn, designed to facilitate the adoption of m/MMM for digital advertising measurement.

Robyn is a widely adopted and actively maintained open-source tool that continually evolves. This article explores the computational components and design choices that underpin Robyn, emphasizing how it “packages up” m/MMM to promote organizational acceptance and mitigate common biases. The solutions described are not definitive but outline the pathways that the Robyn community has embarked on.

• Research & Data Quality
• Article

LOLA: Revolutionizing Content Experiments with LLM-Assisted Online Learning

In the rapidly evolving digital content landscape, media firms and news publishers require automated and efficient methods to enhance user engagement. This study introduces the LLM-Assisted Online Learning Algorithm (LOLA), a novel framework that integrates Large Language Models (LLMs) with adaptive experimentation to optimize content delivery. Leveraging a large-scale dataset from Upworthy, which includes 17,681 headline A/B tests, the study investigates three pure-LLM approaches and finds that prompt-based methods perform poorly, while embedding-based classification models and fine-tuned open-source LLMs achieve higher accuracy.

LOLA combines the best pure-LLM approach with the Upper Confidence Bound (UCB) algorithm to allocate traffic and maximize clicks adaptively. Numerical experiments on data from the website Upworthy show that LOLA outperforms the standard A/B test method, pure bandit algorithms and pure-LLM approaches, particularly in scenarios with limited experimental traffic. This scalable approach is applicable to content experiments across various settings where firms seek to optimize user engagement, including digital advertising and social media recommendations.

• Research & Data Quality
• Article

Marketing Analytics Accelerator: 2024

The Marketing Analytics Accelerator – the only event focused exclusively on attribution, marketing mix models and the science of marketing performance measurement – returned for its ninth year on November 13. The industry’s boldest and brightest minds joined us in NYC to share their latest innovations and case studies that will improve your business outcomes.

• Research & Data Quality
• Article

The Importance of Incrementality in Retail Media Measurement

• INSIGHTS STUDIOS

Despite massive growth driven by significant investments, retail media performance measurement still falls short in many areas. On October 15, OptiMine and Best Buy dove deep into the use of incrementality measurement for retail media, how it works and why it is so unique in the RMN space. Attendees explored why (and how) some of the world’s largest brands have embraced it for improved success.

• Research & Data Quality
• Article

MMM Best Practices and Success Stories

Attendees joined us on September 10 for an Insights Studio on the latest in Marketing Mix Modeling (MMM). Panelists shared proven strategies for implementing and optimizing MMM and revealed valuable insights into best practices and real-world case studies to help build, power, and calibrate MMMs.