How does one evaluate evidence in complex circumstances

So this is Joseph, the rarely appearing co-blogger and I wanted to comment on an evidence question.

There has been a lot of discussion about a recent DC court of appeals case (Halbig).  The essence of the argument is that there is a sentence in the law that could be read as only allowing state exchanges to offer federal subsidies.  I think it is pretty clear everyone agree that this isn't the pinnacle of clear and artful writing.  Either it failed to make a major piece of the law obvious (people buying policies on Federal exchanges would want to know if this was intended, as would states like Oregon considering switching from a state to a federal exchange) or it was a sloppy sentence.  If that was the end of it then it probably would not be bloggable.

What was interesting is what people are holding as the standard of evidence.  Scott Lemieux argues that, due to previous federal law giving deference to how federal agencies interpret law, that the standard is:

The challengers don’t just have to show that their interpretation is plausible; they have to show that it’s the only possible reasonable interpretation.

and

We still have essentially everyone responsible for drafting, voting on and/or implementing the legislation at the federal and state levels (not to mention the Sebelius conservative joint opinion) assuming that the federally established exchanges were intended to work, and a consultant making a bare assertion with no explanation that the federally established exchanges weren’t intended to work. It’s pretty obvious which is more significant, particularly since the next decent explanation for why Congress would bother to create a federal backstop that couldn’t actually function would be the first.

Megan McArdle, whom I think it is well know is not a fan of the affordable care act, proposes a different standard.  She argues:

The thrust of this outrage is that obviously no one ever intended to restrict subsidies only to state exchanges, because that is now endangering the whole program, and obviously, the law’s architects did not mean to endanger the whole program.

But unless this is some sort of elaborate hoax, I think this definitively puts to rest the notion that none of the bill’s architects could possibly have thought or intended that the law would have this effect. Gruber thought the law would have this effect -- and if anyone would know, he would. 

Now I am not a lawyer, but I like talking about standards of evidence.  The first is a standard that you need to show that the law cannot be interpreted in the proposed flexible manner that an agency wants.  The second is to focus really tightly on the exact text of the bill and come up with the most literal interpretation.

Now these are very different ways to thinking about evidence.  McArdle's approach is very frequentist.  Here we create a specific standard of evidence based on one piece of the whole family of possible errors (e.g. random variation for the statistician, text only for the lawyer) and use that as the main decision making element (with some soft text about possible bias).  Lemieix's approach seems more Bayesian -- the analogy to priors is looking at the totality of the evidence, including but not limited to the law's text.

Ironically, I am like Noah Smith in science and kind of like frequentist approaches.  I certainly use them in 95% of my scientific papers.  But this is really highlighting, in an area (law) where the frequentist approach isn't like a second skin for me, that these differences can matter a lot.  I'm also not going to say which side I think is correct here, merely that I am seeing appeals to two very different kinds of evidence.  In real life I have a paper where I called an estimate with a p-value of 0.0504 non-significant (yes, those of us who take the rules seriously see these things happen).  But in the context of a family of associations where everything else was significant and known bias cutting against the association, I wonder if I could have been more sophisticated.  Or would that be less principled.

So now I think I have something to ponder.  But it does highlight that this type of debate about standards of evidence, like all entrenched debates, is complicated.