This article investigates to what degree a modelled approach to non-probability sample screening and weighting improves data accuracy.
Using the ARF’s Foundations of Quality survey data, the authors demonstrate that by applying optimization methods to a large, diverse data set, several key screening variables were identified. They minimized bias (for more than 100 questions covering 14 substantive areas) across 17 sample sources and two sample types.
The evidence suggests that some or all of the following nonattitudinal variables, if included in an adjustment model, can reduce bias by 23 percent beyond the age/gender/race baseline case:
The bias reduction can increase to 35 percent by adding to the model some or all of the following attitudinal variables:
The authors’ advice to survey practitioners: take a balanced approach. Begin by including variables such as age, gender, region, and race-ethnicity in the sampling scheme. Then include in the weighting scheme these same variables and some of the socio-demographic and attitudinal ones identified in this paper.