With traditional focus groups off the table during physical distancing, qualitative researchers face options that may require too many tradeoffs to make the effort worth the costs. The challenges for qualitative research could be even greater than those facing quant, particularly for social or opinion research projects.
If you’re interested in challenges facing quantitative researchers, check out my last two posts.
What Are My Options?
If your qualitative needs can be met with chatroom-style online focus groups, you’re in good shape. It continues to be a solid technique for ad and message testing with larger audiences and for mixing qual and quant methodologies. Recruiters report that homebound respondents have the time and inclination to participate in a variety of formats. This is good news.
No FGDs for now, whether in a room or under a tree
Focus group discussions present a greater challenge however. The most valuable part of a focus group is the personal interaction between participants that an engaged moderator can elicit through thoughtful probing and skillful management of group dynamics. If the success of your project depends on getting beyond top-of mind responses, generating new ideas, and exploring feelings about abstract concepts, remote in-person focus groups on Zoom or other proprietary platform fall short. I worry that so much illuminating data will be distorted or lost that it’s worth asking “what’s the point?”
Do You Enjoy Zoom Meetings? I Don’t
Think of your last Zoom(s) with eight friends or colleagues. Where you satisfied with the quality of interaction with others on the call? Did you enjoy it? How engaged were you with the topic? After more than two months of physical distancing, Zoom interactions are proving to be, at best, unsatisfying and at worst, exhausting. There are good reasons for this.
Axios technology editor Scott Rosenberg articulated my misgivings about video-conferencing as a qualitative tool. Here’s what he writes:
Videoconferencing imposes cognitive and psychological frictions and aggravates social anxieties. As experts in human-computer interaction point out, using Zoom means putting on a show for others without being able to rely on the cues we primates depend on in physical encounters.
- There’s usually a slight audio lag, as well as mute-button mistakes and “your internet connection is unstable”-style dropouts.
- We’re also often opening a chunk of our homes for others to view, and that can trigger social worries.
- By showing us our own image as well as others’, Zoom ensures that we will critique ourselves in real time.
- On top of standard-grade performance anxiety, the “big face” image that Zoom uses by default in its “speaker view” can trigger a “fight-or-flight” surge of adrenaline, writes Jeremy Bailenson, founding director of Stanford’s Human Computer Interaction Lab.
- If you switch to the “Hollywood Squares”-style “gallery view,” you’re confronted with a sea of separated faces, which is not how evolution has adapted us to group interactions.
- As M. Sacacas observes,you can’t really achieve true eye contact with anyone: If you look right into someone else’s eyes, you will appear to them as if you aren’t looking right at them — to achieve that, you have to look right at the camera.
- Nonetheless, the whole experience of a videoconference feels to us like an extended bout of direct staring at other people staring back at us. That’s draining, which is why it’s not what actually happens when we meet in person, where we only occasionally look right at one another.
How will participants react when deprived the visual and social cues that help them interpret the reactions of others? Will feeling uncertain, frustrated, insecure, or tired influence their opinions, particularly on sensitive or political topics? How will these dynamics shape strategy built on the data? Additionally, Rosenberg’s analysis is helpful for contextualizing the reactions of citizens of high-income countries who are comfortable interacting with technology. How will citizens of low- or medium-income countries, or those with different cultural expectations for social interaction, respond? I don’t know if we know the answers to these questions.
Traditional focus groups have many biases, all of which are well-known and factored into qualitative analysis. We know people respond to questions differently in a group setting than they would alone. Online video-conferencing multiplies these biases plus adds others that we’ve barely begun to understand. Not only are there different biases to adapt to, the medium itself diminishes the greatest advantage of a focus group: group/moderator interaction.
So What Can We Do?
Because qualitative research can’t grind to a halt because of physical distancing, we have to come up with ways to mitigate these problems. Here are some suggestions.
- Assess whether videoconferencing is the right tool for the job. Does the topic demand strong rapport among moderator and participants to explore controversial social topics ? If so, your findings may lack depth, mislead or entirely miss important findings.
- Write a shorter, simpler moderator’s guide. This medium is not appropriate for a two-hour long discussion on constitutional reforms.
- Recruit fewer participants. Aim for quality of responses, rather than quantity.
- Use a moderator well-trained and experienced in managing the complicated dynamics of this type of discussion. A high energy, animated moderator might be better able to engage remote participants than one with a more low-key personality.
- Manage client expectations. Video-conference groups and in-person groups are not interchangeable. They will provide different kinds of data. Clients who expect traditional focus group data are likely to be disappointed,
- To avoid the impulse to compare data collected in a traditional focus group and that from a video-conference, start fresh with a new project.
- Be highly cognizant of data privacy protocols. Compliance with privacy laws apply more stringently to shared and stored video.
It’s going to be hard to convince me that video-conference focus groups, despite being absolutely possible, are advisable for all projects. This is particularly true for social and opinion research projects. All researchers have to adapt to our changing environment. The first question before launching a project should always be “what is my research goal and what tools can I use to reach it?” Then you can decide if having some data is better than having no data. Contact Quirk Global Strategies and we can help you decide.
