[I'm trying to get this one out the door quickly so I'll be using an outline format because it's faster.]
1. Unlike most of the polls we've seen over the past week, this survey actually serves a useful purpose in terms of history, political science, sociology, etc. It is important to capture immediate public reactions to big events. Researchers in the future will want to know the answer to questions like what percent of the population agreed with this verdict? It will take time for voters to digest these new developments and decide what impact it does or does not have on their choices. That's a question best left for later. How you felt when you heard the news is a question that has to be asked now.
2. That said, the analysis from YouGov is based on things that aren't in the data. Though this is presented as a side-by-side, apples to apples series of comparisons, you cannot dip your toe twice in the same river, as the saying goes. The April and the June numbers are based on fundamentally different questions, making it basically impossible to get an accurate read on shifts in opinion.
2.a. In order for a question to be the same, it has to be the same implicitly and explicitly, and presented in the same context. This is why the first thing they teach you when you take a class on survey design is that order of questions matters. If you look at the fine print on this widely circulated question from the survey, you'll see they didn't even try to duplicate the original conditions. This isn't a criticism. There really was no way casually work up to "OK... and what if it were the president?"
2.a.i. There is, however, one point we should criticize the April survey on. The way these questions are formatted ("Do you think a felon should be allowed to be a schoolteacher?" "Do you think a felon should be allowed to be a bank teller?" Etc.) is likely to produce a kind of ratcheting action. Once you've said yes to a far less responsible post, it would feel inconsistent to say that it's okay for president. This makes comparisons for this question especially problematic.
2.b. In the April survey, these questions were hypothetical. People think about these what-ifs differently. Among other things, it can be easier to justify contradictions in your position. In the June survey, these contradictions were impossible to ignore. It is extremely difficult to say that you are planning on voting for Donald Trump and that you don't think a convicted felon should be allowed to become president while maintaining any sense of consistency. It would be surprising if a substantial portion of Republicans had not changed their answer on this question.
3. None of this is meant to suggest that opinions have not shifted or that Republicans haven't become more open to the idea of a felon being allowed to serve as president. We would expect this to happen. The problem is we have no way of knowing what part of the shift in the data was caused by an actual change in opinion and what part was caused by the inevitable mismatch of the questions from April to June. Add to this high probability that things are still in flux, and you have a very good argument for not putting too much weight on analyses of the political implications of this and similar surveys for the next few days.
4. Finally, a note of psychic criticism for the analysts of the near future. If these numbers remain fairly constant, and continue to line up with other data we've seen from surveys and exit polls of Republicans, do not fall in the trap of believing that Trump holding onto 80 or 85% of the Republican vote is an encouraging sign for the candidate. I'm not saying that this survey indicates trouble for Trump. I'm not putting any weight in any surveys taken within two weeks of the verdict, nor am I going to get all that worked up about any polls at least until the conventions. What I am saying is that when a pundit tells you that having more than 20% of the members of your party say the de facto nominee shouldn't be allowed to be president is good news, it's time to get another pundit.
[Garbled sentences in 3. have been fixed.]
I am mostly bothered by the fact that if you ask me to visualize a "convicted felon," I would not have come up with a business mogul who cheated on his taxes by expensing hush money; that's not the average convicted felon. So on your point 2b, after the Trump conviction, and pertaining to this particular poll, it would seem appropriate to think of "convicted felon" as the business mogul who cheated on his taxes by expensing hush money, thus, I think you are right, the two sets of responses can't be compared.
ReplyDelete