Election polling in the US has a problem. Or, depending on who you ask, more than one problem. Since the 2016 election, many big brains have tried to fix these issues with varying degrees of success.
I have problems too, but mine aren’t the same. Pre-COVID, my biggest problems were long fielding times, QC issues and inaccessible sampling points. I don’t have to deal with response rates in the low single digits.
For once this post isn’t about my problems. It is about what I have learned from other folks’ attempts to answer the question: what is the best way to obtain a random, proportionate sample when the tools we’ve always used (for North Americans, telephone surveys) aren’t working very well anymore?
Mixed Mode: Is it the Future?
The answer seems to be some form of “mixed mode” data collection. This means giving respondents several ways to participate in the survey. It could be by telephone, SMS, face-to-face, web, or even snail mail.
I’m not going to get in to the pros (there are some) and the cons (there are many) of each mode, particularly in low- and middle-income countries. Internet, smartphone and literacy distribution vary widely. Although the combination and proportionality are different for every country, mixed mode could mitigate some sampling and data collection problems that pre-date and/or have been exacerbated by COVID. Letting people respond on their terms, rather than mine, seems like an obvious step to take.
Meet Respondents Where They Are
While watching this very interesting University of Chicago panel, it occurred to me that even though I haven’t got the same worries as US election pollsters who can’t get people to answer the phone, I could make my surveys better by providing response options that are more convenient for respondents. These tools exist and COVID has forced us to start using them more!
Staying on top of things during the storm is hard
For example, if younger participants prefer online surveys, they should have that option. If it is better to reach older, technophobic or less literate respondents through face-to-face interviewing in their homes, then get out the Kish grid. With careful questionnaire design to mitigate design effects and framing that accounts for selection effects, it is possible to do both in the same survey. Creative use of incentives could also help.
Substituting one mode for another is rarely a solution. I’ve been resisting this approach through this whole COVID year. Running an online-only survey and expecting a representative sample in a country where many people lack smartphones, internet access or have difficulty reading is a big mistake. Younger, better-educated urbanites are already overrepresented in policy discussions. And don’t get me started about panels. Most aren’t, ahem, Pew-level quality.
Shorter Surveys Are Good
There’s more! Shifting away from long and complex face-to-face surveys will force clients to accept shorter questionnaires. As I’ve said before, making busy, less-educated, sometimes food insecure respondents slog through an hour-long survey on constitutional reform does not produce good data. Telephone and online surveys forcing simpler, shorter instruments is an unmitigated good.
COVID has forced us to rethink the ways we’ve always collected data. If one mode appeals more to one group and another group is easier to reach using different mode, why not figure out how to use them both? It’s worth putting in the extra thought. Respondents do us a big favor by answering our questions. It is incumbent on us to make it as easy as possible.
Contact QGS and we can talk about ways to make mixed mode work for your issue and country.
