Sub-groups in clinical trials

 This is Joseph

Astra-Zeneca/Oxford have the third (or fourth if you include Sputnik V) successful covid-19 vaccine. This is exceedingly good news for the chances of ending the pandemic with a vaccine, especially as this is the second successful mechanism (adenovirus-based delivery system) for a vaccine to have (Pfizer and Moderna are mRNA). 

But the sub-group of interest was present by chance:

The British drugmaker said on Monday that the vaccine could be around 90% effective, when administered as a half dose followed by a full dose a month later, citing data from late-stage trials in Britain and Brazil.

“The reason we had the half-dose is serendipity,” Mene Pangalos, the head of AstraZeneca’s non-oncology research and development, told Reuters.

A larger group who had received two full doses - as planned - resulted in an efficacy read-out of 62%, leading to an overall efficacy of 70% across both dosing patterns.

I don't want to undersell these results but the "serendipity" was in a sub-group of only 2700-odd participants, and the randomization was 2-1 vaccine to placebo. That is much smaller than the 41,000 in the final readout of the Pfizer trial. I also tend to omit Sputnik V because of the small sample size:

The data is based on 20 cases of Covid-19 from 16,000 volunteers given the Sputnik V vaccine or a dummy injection.

But some math suggests that the Oxford vaccine sub-group isn't based on much more data, and perhaps less (we don't have actual numbers quite yet).

All of this said, what would the authors have done if the error in dosing had the low efficacy and the correct dosing schedule had the high efficacy? Would they be highlighting the sub-group at all, or would they hedge with terms like "when properly used". Furthermore, since this was an accident it is, by definition, a post-hoc sub-group (intention to treat analysis would pool them with the higher initial dose group). That suggests that the statistical penalty is even higher (now we have at least 2 statistical tests, and more if you think of the hypothetical sub-groups). 

So perhaps caution is in order when considering these results. Certainly, if you think that this evidence is sufficient then you are in the 4th successful vaccine camp, as Sputnik V definitely had more total evidence than this vaccine does. 

 Still, the Oxford vaccine is likely to be inexpensive, fast to manufacture, and less susceptible to cold chain problems. These are not inconsiderable advantages and it might still have a valuable role to play as the numbers continue to come out.