Relative risk is a ratio of the probability of an event occurring in the exposed group versus the probability of the event occurring in the non-exposed group.
The three relative risks proposed are: 1.25,1.66E+ 52, and 5.65E+ 05. No, the second two are not jokes. A relative risk of 1.25 is plausible, if hard to prove with a tricky outcome like cancer. But something went quite wrong with the other two. It might have been a hint when this was in the acknowledgements:
We wish to acknowledge with grateful thanks the work of Professor Mark Stevenson in modifying and enlarging the capacity of “epiR” to handle the enormous integers encountered in this study. His prompt and timely assistance is greatly appreciated indeed.
The other issue is with E-values. An E-value is defined as:
The E-value is defined as the minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment-outcome association, conditional on the measured covariates. A large E-value implies that considerable unmeasured confounding would be needed to explain away an effect estimate. A small E-value implies little unmeasured confounding would be needed to explain away an effect estimate.
But the authors slide this into other forms of bias in a rather interesting way:
Furthermore this was also an ecological study. It is therefore potentially susceptible to the short-comings typical of ecological studies including the ecological fallacy and selection and information biases. Within the present paper we began to address these issues with the use of E-values in all Tables.
Now, E-values are quite useful for thinking about confounding but I am quite interested to see how they apply to things like selection bias, except insofar as a larger estimate requires more substantive bias to be entirely artifactual due to selection bias. The other stuff, about biological plausibility, is fine and that is the reason to have conducted the study. But the E-values say nothing about bias other than how strong a set of confounders would need to be to induce a spurious association.
Also this is not quite true:
Causal inference was addressed in two ways. Firstly inverse probability weighting (IPW) was conducted on all mixed effects, robust and panel models which had the effect of equilibrating exposure across all observed groups. IPW were computed from the R-package “ipw”. Inverse probability weighting transforms an observational dataset into a pseudo-randomized dataset so that it becomes appropriate to draw inferences as to truly causal relationships.
This is only true under a set of assumptions,
One, there is no unmeasured confounding present. Two, that the marginal structural model is correctly specified (both the marginal structural model and the model for exposure). Three, that each participant’s counterfactual outcome under their observed exposure is the same as their observed outcome (consistency). Finally, we need to assume positivity — that there are exposed and unexposed participants at all levels of the confounders. If these assumptions are not met, then the marginal structural model may give misleading estimate
IPW can be a powerful and useful technique, but it is hardly magic and cannot replace randomized data in causal inference. The use of E-values immediately casts doubt on the first assumption being fully realistic.
Now there may be some important information in this ecological study, but I find these epidemiological methods issues really detract from the potentially good science elsewhere in the paper.
