marketing experiments

How Generative Search Changes Shopping Decisions

  • ARF; MSI

Generative search may help consumers move from broadly stated needs to more focused shopping decisions. In a field experiment involving more than 6.4 million users of Meituan—a Chinese ecommerce platform, consumers given AI-generated responses placed more orders and generated greater transaction value than those using conventional keyword search. They also spent less time browsing and concentrated their attention on fewer, more relevant categories and merchants.

Member Only Access

Analytics & Forecasting 2026

  • Analytics & Forecasting

AI is transforming how organizations generate insights and forecast outcomes, but questions remain about reliability and the need for human oversight. On September 14-15 in NYC, industry and academic leaders gathered to discuss integrating AI into analytics workflows and maintaining transparency.

Member Only Access

Making Sense of Open-Source Marketing Mix Modeling

  • ARF; MSI

As privacy changes make it harder for marketers to track individual customer journeys, media and marketing mix modeling (m/MMM) is experiencing renewed interest. Open-source tools from Meta, Google and the PyMC ecosystem promise to make these methods more accessible, particularly for organizations without the resources for customized proprietary models. In this MSI working paper, Julian Runge and Koen Pauwels evaluate the leading open-source approaches, examining their capabilities, differences and tradeoffs. Their analysis shows that while automation can democratize sophisticated marketing measurement, using these tools responsibly requires careful attention to model assumptions, diagnostics, calibration and validation.

Member Only Access

Beyond the Dashboard: Rethinking How Marketing Success Is Measured

  • ARF; The Advertising Club of New York

Marketers have access to more data than ever before, yet many still struggle to answer the most important question: Did the marketing effort truly make a difference? And if so, how? Lots of metrics exist. So, is collecting more metrics the best approach to evaluating true impact? In this guest article for The Advertising Club of New York, ARF Senior Director of Research & Insights Tracy Adams argues that the future of measurement isn't about collecting more metrics—it's about integrating multiple approaches to build a more complete, decision-ready understanding of marketing impact.

Member Only Access

Beyond Review: Why Human Judgment Remains Essential in AI-Assisted Research Design

  • Psychology of Gen AI
  • ARF; MSI

As generative AI becomes increasingly integrated into research workflows, what does "human in the loop" actually mean? This latest installment in the ARF and MSI's Psychology of Gen AI series explores how researchers and large language models can work together to develop rigorous research designs—and why human judgment remains indispensable throughout the process. By comparing multiple AI-generated experimental designs and refining them through expert evaluation, the study demonstrates that AI is most valuable as a collaborator that expands possibilities, not as a replacement for methodological expertise.

Member Only Access