ARF the Advertising Research Foundation
  • Membership
  • Research
  • Access Knowledge
  • Communities
  • Events
  • About the ARF

home

Membership

  • Committees & Boards
  • Events
  • Self Service Research
  • Member Bulletin Board

Research

  • Recent ARF Research
  • Programs & Resources
  • Research Focus Topics
  • ARF DASH: Annual TV Universe Study

Access Knowledge

  • Topics
    • Ad Effectiveness & ROI
    • Analytics & Data Science
    • Audience & Media Measurement
    • Creative & Branded Content
    • Future States & AI
    • Past Event Highlights
    • Research & Data Quality
    • Targets & Segments
  • Find a Solution
    • 2025 Top Member Questions
  • Publications
    • News You Can Use
    • Knowledge at Hand
    • CMO Briefs
    • September 2026: Journal of Advertising Research

Events

  • Upcoming ARF Events
    • View All Upcoming Events
    • Upcoming Conferences
    • Upcoming Young Pros
  • Insights Studios
    • Upcoming Insights Studios
    • Past Insights Studios
  • Town Halls
    • Upcoming Town Halls
    • Past Town Halls
  • Past Event Highlights
  • Submit

Communities

  • Young Pros
    • About Young Pros
    • Upcoming Events
    • YP Connect
    • Advisory Board
    • Young Pros Spotlight
    • Social Strategist Team
    • E-Newsletters
    • Past Events Highlights
  • Women in Analytics
    • About Women in Analytics
    • Board of Directors
    • Mentorship Program
    • Upcoming Events
    • Past Events Summaries
  • Councils
    • Learn About the Council Program
    • ANALYTICS COUNCIL
    • MSI/ARF B2B Exchange
    • CREATIVE COUNCIL
    • COGNITION COUNCIL
    • CROSS-PLATFORM COUNCIL
    • CULTURAL-EFFECTIVENESS COUNCIL
    • LA MEDIA RESEARCH COUNCIL
    • ORGANIZATIONAL COUNCIL
    • PHARMA COUNCIL
    • SHOPPER INSIGHTS COUNCIL
    • SOCIAL COUNCIL

About ARF

  • About the ARF
  • ARF Board of Directors & Trustees
  • ARF Leadership Team
  • ARF at 90
  • Marketing Science Institute (MSI)
  • Coalition for Innovative Media Measurement (CIMM)
  • Member Directory
  • ARF Awards
  • Press Kit
  • ARF in the Press
  • Contact Us
  • NYU/ARF Certificate in Marketing Optimization & Insights Certificate
  • ARF WIDE
  • ARF DASH: Annual TV Universe Study

My Account

  • Login
  • Create an Account

Invalid login. Please try again.

Password does not meet the required minimum length of 7.

Time to Ditch A/B Testing?

November 29, 2017

Simple A/B testing of digital search ads should be retired in favour of a multi-level option that is now enabled by ad rotation, according to a leading Google executive.

Matt Lawson, Director of Performance Ads Marketing, Google, explained that optimized rotation of ads allows marketers to have more ads present in an ad group and increase their chances of finding the right match across all of the variables that Google AdWords considers.

“Going from one ad to a minimum of three ads in an ad group can give that ad group up to a 5% to 15% lift on average in both impressions and clicks (with incremental uplift for each additional creative),” he stated.

He suggested that users could test one ad (A) against an experiment ad group with four ads (A, B, C and D) with rotation set to optimized. So instead of finding a “winner” and scrapping the “loser”, users “should plan on having options A, B, C and D all active at any given time” with an ad only being deleted when it generates minimal clicks at which point it can be replaced with a new one.

Lawson also cautioned against focusing on CTR (click-through rate) to assess an ad’s effectiveness. “CTR alone can be misleading because ads show on all sorts of queries in all sorts of contexts,” he pointed out. “There’s no way to control for all of the different devices, locations, situations and everything else that goes into one action.” Accordingly, he recommended using ad group-level metrics, including impressions, clicks, conversions, and CTR.

Sourced from Search Engine Land; Additional Content by WARC Staff. (2017, Nov. 27). Time to Ditch A/B Testing? WARC.

  • Topics
    • Ad Effectiveness & ROI
    • Analytics & Data Science
    • Audience & Media Measurement
    • Creative & Branded Content
    • Future States & AI
    • Past Event Highlights
    • Research & Data Quality
    • Targets & Segments
  • Find a Solution
    • 2025 Top Member Questions
  • Publications
    • News You Can Use
    • Knowledge at Hand
    • CMO Briefs
    • September 2026: Journal of Advertising Research

Trending

    • Future States & AI
    • Article

    When AI Becomes Part of the Consumer–Brand Relationship

    • Research & Data Quality
    • Article

    What Buyers Want from Retail Media Measurement

    • Creative & Branded Content
    • Article

    How Emotions Travel Between TV Programs and Advertising

    • Future States & AI
    • Article

    How AI Recommendations Are Changing the Rules of Brand Growth

    • Future States & AI
    • Article

    Beyond Visibility: What Makes Brands Discoverable Today?

Activate your Member Benefits

  • ABOUT
  • ARF DASH
  • ARF WIDE
  • ARF/NYU CERTIFICATE
  • CIMM
  • MSI
  • PRESS
  • SUBMIT
image description

To further, through research, the scientific
practice of advertising and marketing

© the Advertising Research Foundation. All rights reserved.

Privacy Policy | Request Permission to Use Content