Curious about how opinion polling can proceed when entire countries are closed down? I was, so I reached out to data collectors around the world to better understand what’s possible now and what kind of projects should wait.
Pollsters who use telephone interviews are in a good position to keep on working. Unfortunately, those who rely on interviewers who need to move through communities and interact with people in their homes are going to have a rougher time. In the short term, researchers will have to put those projects on hold or consider new methodologies.
Telephone Surveys Are Happening
From the early days of COVID-19, phone banks serving high-income countries of North America, Europe, and the Middle East started adapting. Obviously, big rooms filled with interviewers sitting in front of terminals are out of the question. In response, data collectors I’ve spoken to quickly set their interviewers up on secure systems that allow them to call from home. This is good news for pollsters and interviewers.
Given the unrelenting demand from election pollsters, this decision was a no brainer for US-based firms. Those with global calling capacity and multi-lingual interviewers can transfer also projects to offices in countries where workforces have been less affected by shutdowns. Response rates might also improve because people stuck at home could be more willing to respond to surveys. In the US at least, it would be hard for response rates to get worse.
For those who field in high-income countries, this is all good news. It’s important to keep a couple of things in mind, however. In the US, ordinary election year demands stretch collectors’ capacity up to and often beyond limits. Having limited remote interviewers will exacerbate the problem. Management capacity will also be stretched as quality control and training demands grow. Patience, oversight and regular communications with data collectors are critical.
If your project in high-income countries in North America, Europe or the Middle East needs to move forward, then experienced phone houses with strong management can get it done. If you can hold off, or wait for a lull in your collector’s schedule, the quality of your data will likely be higher. I analyze my data by day as it comes in with an eye for anomalies. It’s also smart to build in extra days for call-backs, or in case things go haywire and you need to replace interviews or do extra QC.
Face-To-Face Surveys Are Problematic
As a specialist in survey research in conflict and post-conflict environments and low/middle income countries, about 80% of my projects rely on face-to-face (F2F) interviews. Phone interviews are not an option in places where mobile penetration is low or disproportionately distributed. In normal times, in-person interviewers face obstacles such as transport difficulties, bad weather and conflict-related threats. Those problems are manageable. But add a highly contagious disease spread by personal contact and F2F data collection suddenly becomes untenable. It could be this way for while.
Whether F2F data collection is viable depends on a country’s infection rate and whether restrictions on movement and social contact have been put in place. This fast changing situation makes planning almost impossible. For example, I have a survey ready to field in India. Up until a week ago data were being collected normally in many states and slowly, and with extra precautions, in others. On 25 March, the entire country was locked down to stop the spread of the virus. Thankfully, we had opted to wait until the situation clarified and had not begun fieldwork.
The situation in many African countries is also ambiguous. Some countries have few infections and life continues normally, for the time being. Other countries are suffering badly from outbreaks. Ukraine has a serious outbreak and F2F data collection is on hold. Quant work in the Philippines is also on a slowdown, if not entirely stopped.
Experienced, ethical data collectors are in the best position to offer advice on the local situation. The final judgment always belongs to the researcher however. No one — interviewer or respondent — should be put in danger for the sake of survey research.
Data collectors in some F2F countries have begun experimenting with phone surveys and online panels. The possibility for error and biased samples remain serious concerns. Having a complete understanding of how these samples under- or over-represent populations is critical; some populations will simply not be reachable by these modes. Survey work in difficult environments often requires methodological tradeoffs. If you can live with increased and unpredictable error from non-probability samples, or if you can afford to be experimental, I say give it a try.
I Can Field a Survey. Should I?
Of course, pollsters need to consider ethical questions before fielding surveys during a global pandemic. Should interviewers be sent into the field, handhelds ready, to conduct interviews in people’s homes? Absolutely not. Should worried or scared respondents be pestered with questions about topics of less importance than life, death and economic survival? I hear the same argument against conducting surveys in conflict zones. Often, the concern is overstated. Many people — especially in low- and middle- income countries — are not asked their opinions asked about anything. They’re usually happy to oblige. Additionally, respondents can be more forthcoming in times of insecurity and unpredictability. Data collected during this period will be a real “snapshot of a strange time” and will be fascinating to track over time. As I do in conflict zones or closed spaces, I let respondents tell me if they are uncomfortable or unwilling to participate. I look at response rates, drop-offs and interviewer comments before I make judgements about respondent willingness.
The need to understand public opinion during and after this crisis is not going to go away. Economic, political and social disruption could shape perceptions in unpredictable ways in low/middle income countries and upper-income countries alike. Views that once seemed hardened and unlikely to shift may change radically, or not at all. Whether you’re working in a high income country of North America, Europe or MENA, or looking at surveys in lower/middle income countries, Quirk Global Strategies can help you sort through your options. Contact us through the link on this site.
