Using the outcome as a conditioning variable

This is Joseph.

Felix Salmon has a great comment on Facebook and Female Coders:

To spell it out: The more your code is rejected, at Facebook, the less likely you are to rise up the ranks. So the fact that women suffer from significantly higher levels of code rejection is a big problem. The evidence for this being a problem is precisely the fact that Facebook’s female engineers are disproportionately found at lower levels rather than higher levels. 

And yet somehow, Facebook has contrived to use that fact in support of its claim that there isn’t a difference in how male and female engineers at the company are treated. Instead of treating the prevalence of men in the upper engineering ranks as prima facie evidence that there’s something amiss, they use it to exonerate themselves of sexism.

As Thomas Lumley notes, conditioning on the outcome rather ensures that the model is unhelpful (I found this article reading his most excellent site).

What would be interesting to see would be what would happen if you randomized some gender swapping on the code authorship.  Or were able to blind reviewers to the gender of the code author; that would likely have some pretty positive consequences.  Because another consequence of this finding is that either some sub-par code gets accepted or some good code is being rejected.  You would assume that a private company would want to correct this issue, either way.