Two excellent examples in the news (more or less) this week about conducting polls in unstable countries. (Tomorrow, the latest poll from Georgia).
The first was a snippet in an interesting Slate article by Alex Halperin called What’s Going on In Kenya?
Unfortunately, one bit of data has not surfaced. The International Republican Institute, a democracy-fostering nonprofit funded by the U.S. government—and despite the name, officially nonpartisan—commissioned an Election Day exit poll but has declined to release the results. Two people familiar with the results told me that they showed Odinga with a substantial lead over President Kibaki—one reported eight points, the other nine points. One has only to remember the United States’ 2004 elections to know how fallible exit polls are, but a U.S.-sponsored survey would have weight here and could have given the ECK pause before it called the election so disastrously.
Ken Flottman, an official in the IRI’s Nairobi office, said the data would serve additional purposes, such as studying voter demographics. The organization issued a statement criticizing the vote counting but does not mention its data. It missed an opportunity to advance its mission of promoting democracy and fair elections.
There’s been limited news coverage or reaction in the blogosphere so far to this except from knee-jerk reactions from people who know little about IRI’s mission or the purpose of exit polls. Furthermore, drawing conclusions about the fallibility of an exit poll in Kenya based on the 2004 election in the US or any other country, as Halperin does, is specious.
Halperin asks a fair question, though: Where’re the data?
I am going to preface my comments by saying I consult for IRI in a different region and have been to Kenya, but that is the extent of my knowledge of both the country and IRI’s operation there.
Because different kinds of fraud are found in different countries, exit polling is appropriate in some places and not in others. While they are designed to account for election day fraud like ballot box stuffing and miscounts, exit polls cannot measure fraud that takes place in the pre-election period (voter intimidation, bad lists or vote buying). They can also be marred by the old FSU standby of carousel voting (people who vote multiple times and will probably respond to an exit poll multiple times).The decision to move ahead with an exit poll should follow an assessment of what types of fraud are expected.
Like all polls, exit polls are tools. Knowing nothing about Kenya’s pre-election political environment, I can’t assess whether IRI’s exit poll was the right tool or not. But since my impression of Kenya is that most fraud happened in the counting process, it sounds like an exit poll could play a constructive role. (And please, correct me if I am wrong. That there’s so much surprise that things could have gone so badly so quickly in “democratic” and “economically-robust” Kenya suggests that a lot of people just started paying attention the week before the election. I predict the same thing will happen in Georgia, a democratic paradise after 2003 that everyone promptly forgot about, allowing it to slide backwards. Whoops!)
Unlike the US, where exit polls exist largely to pacify campaigns and political junkies (that may change!), countries that rely on them to expose election fraud are, by definition, sketchy and corrupt, making data collection a challenge. For example, it can be hard to find reliable firms that can withstand political or economic pressure. Additionally, despite layers of checks and balances, data can be manipulated at different points in the process (although if enough systems are in place, this is very, very difficult to do). Because of this, any pollster who is concerned about his or her own credibility reserves the right to not release the data he or she believes it may have been compromised during collection, coding, processing or reporting. In this case, releasing corrupted data would be far worse than releasing no data at all.
Keep in mind, however, that in some political environments, failing to release the data from a highly publicized exit poll would be a disaster. In countries where there is little or no trust in institutions like the media, election commissions or even private business (Georgia being a perfect example), suppressing the results will almost certainly be perceived as a provocation and as evidence of the very fraud the exit poll was supposed to expose. In places like these, fact and rumor exist on equal footing, allowing conspiracy theories to grow like mushrooms after a hard rain. The exit poll becomes a tool to justify all kinds of bad behavior.
In Georgia, for example, the opposition is looking for any excuse to take to the streets after the election; a suppressed exit poll (or one that shows that they, uh, lost) would provide the excuse they need to protest. In a country where post-election violence is a real possibility, the decision to proceed with an exit poll is a serious one. My impression of Kenya is that its post-election violence has absolutely nothing to do with IRI’s failure to release its poll.
Do I discount monkey business with a USG-funded exit poll? I worked for an NGO long enough and have had enough exit poll experience in the former Soviet Union to never completely discount theories like digital emunction’s. But when I hear hoofbeats, I tend to think horses first, not zebras. Even in Kenya.
