Model Decay, Causal Mechanisms and Townhallophobia

[Now with title.]

I know I've been going to the well a bit too often recently in terms of old posts. I'll admit there's an element of laziness here, but I also think it's useful and even important to review old arguments in terms of new developments.

This also gives us a chance to dig a little deeper into pertinent related topics. For instance, despite living in a data obsessed age when every journalists feels the need to pepper reporting with graphs and statistics and models and algorithms (and frequently models mislabeled as algorithms), there are still a number of fundamental analytic concepts that generally evade even the more sophisticated data journalists.

A notable example is model decay. With the exception of the physical sciences, even the best models tend to lose explanatory and predictive power over time (though in the life sciences, the changes can be very slow). This is particularly true in the social sciences where people quickly adapt to conditions, sometimes specifically in response to awareness of the very models in question.

Model decay can happen slowly or suddenly and there's no simple procedure for seeing how well things are standing, but there are red flags you can look out for.

Though I may get some pushback on this, I think most good modelers and researchers at least implicitly have a set of plausible mechanisms in mind when they propose a causal relationship. (If a researcher can't think of any plausible mechanisms that might cause the relationship he or she is seeing, that's probably cause for concern.) For example, let's say a study found that people who purchased a gym membership and did not make serious use of it saw their body fat tend to increase. The researcher might hypothesize that the subjects were rationalizing bad dietary choices based on the belief that they would work off the pounds.

Though these plausible mechanisms are just good guesses, it is a good idea to keep them in mind when considering model decay. If we have reason to believe that the mechanisms for a particular model no longer function in the same way, we also have reason to believe that the model may no longer be viable. Going back to our previous example, imagine that a certain unnamed financial institution buys a health club chain and starts signing up customers without their knowledge. We definitely have reason to believe that the relationship between unused gym memberships and body fat will break down.

With the rise of data-based political reporting, we've also seen an uptick in simplistic A-then-B models. Writers will often explicitly acknowledge that conditions are completely different then turn around and insist that the same relationships that held before hold now.

Consider the incumbency advantage. There's obviously a relationship here, but what are the plausible mechanisms behind it? We could always reach out to political scientists or dig into relevant studies and polling data, but we can also make some pretty good common sense guesses. It's entirely reasonable to suppose that the advantage is at least partially dependent on incumbents being responsive to their constituents' wants and needs and maintaining a strong connection with the people back home through regular contact via multiple channels.

When politicians take stands that are unpopular with their constituents and then abruptly and publicly cut off these channels of communication, there is reason to believe that assumptions about the incumbency advantage may no longer hold.

Here's what we were saying about this ten months ago.

Though, to be perfectly fair, Tennessee has always been a hotbed of leftist radicals

We have all heard the statistics about how difficult it is for a Congressional representative to lose his or her job. This is partially because of things like gerrymandering and spigots of campaign cash, but it also reflects a process that does a pretty good job allowing a reasonably competent and dedicated legislator to keep the constituents fairly happy in his or her district. A big part of that process is the maintaining of good relationships and lines of communication with voters and communities. Many political career has ended when voters felt someone had "lost touch with the people back home."

In this context, stories like the following from Talking Points Memo's Allegra Kirkland take on a special significance.

Constituents requesting that Rep. Jimmy Duncan Jr. (R-TN) hold a town hall on repealing the Affordable Care Act aren't being met with a polite brushoff from staffers anymore. Instead, Duncan's office has started sending out a form letter telling them point-blank that he has no intention to hold any town hall meetings.

“I am not going to hold town hall meetings in this atmosphere, because they would very quickly turn into shouting opportunities for extremists, kooks and radicals,” the letter read, according to a copy obtained by the Maryville Daily Times. “Also, I do not intend to give more publicity to those on the far left who have so much hatred, anger and frustration in them.”

In the first weeks of the 115th Congress, elected officials dropping by their home districts were surprised to find town halls packed to the rafters with concerned constituents. Caught off guard and on camera, lawmakers were asked to defend President Donald Trump’s immigration policies and provide a timeline on repealing and replacing the Affordable Care Act.

Now, many of them are skipping out on these events entirely. Some have said large meetings are an ineffective format for addressing individual concerns. Many others have, like the President himself, dismissed those questioning their agenda as “paid protesters” or radical activists who could pose a physical threat.

Voters turning out to town halls are pushing back hard on this characterization, arguing that they represent varied ideological backgrounds and have diverse issues to raise. Constituents unable to meet with their elected officials over the weekend told TPM that they’re not attending town hall events to make trouble. Instead, they say they want accountability from the people they pay to represent them.

Kim Mattoch, a mother of three and event planner, told TPM that she tried to go to a Saturday town hall in Roseville, California with GOP Rep. Tom McClintock but couldn’t make it in. The 200-seat theater hosting the event was quickly filled to capacity, leaving hundreds waiting outside.

“I’m a constituent of McClintock and a registered Republican in a very Republican district—though I don’t really align very well these days with the Republican Party,” Mattoch said in a Monday phone call. “So I wanted to go to the town hall because I legitimately had questions for the congressman.”

Mattoch said the protesters waiting outside had a wide range of “legitimate concerns.” She personally hoped to ask her representative about how the GOP was progressing on repealing and replacing the ACA and why House Republicans last week voted to kill a ruling aimed at preventing coal mining debris from ending up in waterways.

Yet McClintock told the Los Angeles Times that he thought an “anarchist element” was present in the crowd outside his event, and said he was escorted to his car by police because he’d been told the atmosphere was “deteriorating.”

Ramon Fliek, who attended the McClintock event with his wife, told TPM on Monday that police “were kind enough to block the whole road” to make space for the overflow crowd, and that he overheard protesters thanking law enforcement for “doing their jobs.”

“If you look at the videos from the event, you can’t get any notion that it was aggressive,” he said. “There was an older woman with a poodle that ran after him and it’s like, okay, the older lady with the poodle is not going to threaten you. I understand that he might want to give that impression, but it was very pleasant.”

Admittedly, it is a long time until midterms, but possibly not long enough to repair this kind of damage.