That depends on the meaning of “all”

February 10, 2012 at 3:33 pm

| Gabriel |

Public Religion Research Institute has a poll showing denominational breakdown of opinions on the question “All employers should be required to provide their employees with health care plans that cover contraception or birth control at no cost.” They made a very nice bar graph of this that has been widely picked up (eg, Dave Weigel at Slate). The graphic shows that only among white evangelicals do a majority of respondents disagree with the prompt. Most notably the poll puts Catholics against the official position of their church. This is very interesting but it also illustrates a few problems with interpreting poll results.

First, the underlying question is a 5-point Likert scale and the summary graphic lumps together “mostly agree” and “completely agree.” It’s a fool’s errand to try to translate Likert scale responses into equivalent legislation, but it’s plausible to imagine someone putting “mostly agree” whose more elaborated thought would be “in general I support such coverage, but I think there should be a conscience exemption.”

Moreover, we can even imagine someone who would think that if primed but it doesn’t occur to them and so they say “completely agree.” In fact, the poll did prime people in a follow-up question by asking specifically about health plans offered by “churches” and “religiously affiliated colleges and hospitals” and this wording gives you  about a ten point swing.

Now in a way this doesn’t make sense. To a mathematician the set “all employers” includes the set “religiously affiliated colleges and hospitals” but that’s not how people think. Rather people basically read “all” as “typical” or “unmarked” and so they may revise their theoretically absolutist position when you present them with a compelling special case.

You can see this in GSS “violent experiences” module (much of which is actually about opinions), which has long asked when we would approve of ordinary people and cops hitting other people. The GSS first asks “are there any circumstances” (HITOK, POLHITOK) and then asks about several specific scenarios. In the first two waves, NORC treated HITOK and POLHITOK as skip codes. That is, if you said no to “are there any situations” where you’d approve of violence they took you at your word. This was a mistake as revealed in later waves when they started asking the follow-up questions anyway, regardless of responses to the generic questions. It turns out that quite a few people who categorically disapprove of violence in the abstract will approve of it once you prime them with a compelling justification for violence. For instance, about 70% of theoretical pacifists approve of hitting a man who is beating a woman. Actually, some people don’t even need a compelling justification, about 3% of theoretical pacifists approve of hitting a protest marcher and about 8% approve of a man hitting a stumbling drunk who accidentally  bumped into his wife.

Moral of the story, you can’t treat poll responses as things that you can plug into syllogisms. Rather you should ignore words like “all” or “never” and realize that in practice they mean “usually.” For the more compelling case you have to adjust up, for the less compelling case you adjust down, and if at all possible you use a wording that describes the particular scenario rather than inferring from generic conditions. In this respect it’s a bit baffling that PRRI emphasized the generic wording rather than the more specific wording that was pertinent to the bishops’ revolt against the administration. On the other hand, the generic wording is suggestive that an effort to eliminate the mandate entirely is likely to fail badly.

Entry filed under: Uncategorized. Tags: .

How is a Textbook Like an MRI? [updated] Sympathy for the IP Industry


The Culture Geeks


%d bloggers like this: