Translating PC Survey Best Practices to Mobile: What Works Best?

Summary

As part of the ARF’s investment in on-line research quality, the Foundations of Quality projects 1 & 2 (FoQ1 &2) brought together 50 experts to analyze over 100,000 consumer surveys.  These experts have published best practices in on-line survey design, length, sample composition, and analysis in the process.

An unintended outcome of FoQ2 was a readable sample of mobile survey respondents.  That initial review demonstrated that consumers on mobile devices were more likely to be Millennial or Multicultural.  And, while their responses were comparable to people on PC’s, they were also more likely to drop out of the survey or take much longer to complete it.

To address this challenge, the group sought to answer these questions:  What PC best practices are relevant to Mobile? What changes to PC best practices might improve mobile response?

After a review of 40 academic and industry papers, two key changes to PC best practices were proposed:

1. The FoQ studies recommended limiting PC-based surveys to 20 minutes. For Mobile surveys, they suggested cutting that time in half, i.e., a mobile survey should be 10 minutes or less, after which the risk of respondent fatigue – and risky responses – sets in.
2. Mobile surveys need to be “thumb-friendly.” Just like any other mobile interface, mobile surveys should be clear, legible, load quickly, and like a PC survey, be engaging.  Too much scrolling or pinching wears consumers out.  Or worse, some response options may never get seen because the consumer doesn’t scroll left and right.