Cheerios vs the FDA

Via Tyler Cowen, there is this new article.

So, because health research is being cited, Cheerios are a drug? So what is the rationale for regulation of this product? Patient safety? I fear that denying patients food because clinical research has not been done would seem to be an unwise idea.

I do think that poor regulation of products can lead to harm but there does seem to be a difference of kind here. Nor do I think that merely citing research should be an issue: there should be more careful research into diet and not less.

What should your final model be?

In the decision to present the best possible model based on the weight of evidence, should you present the results of Bayesian Model Averaging or of a single selected model based on variables that had strong posterior probabilities in the BMA analysis?

I think you have to present the posterior probabilities. They actually contain MORE information than the selected model. More importantly, the p-values in a selected model don't seem to give a good sense of model uncertainty. This feature is something that I dislike about all automated model selection approaches in that they don't represent competing models that were close to the final model in likelihood but just happened not to be selected.

But it looks like almost no substantive papers use BMA. Why is this?

Measuring changes

Is there anything trickier than measuring changes? Probably, yes, but this has got to be competitive for the difficulty in translating clear concepts into analysis. We can all describe, using ordinary language, the idea that a person may become more physically active with time.

But how do we operationalize this using the tools available in the standard cohort study. Do we look at change? What about change conditional on baseline? But the second quantity is going to have issues with measurement error.

What about categorizing things? Sadly, this creates the annoying problem of regression to the mean and ignores the bigger question of how do we define a meaningful threshold. Let alone the issue of reducing the information content of a continuous variable by breaking it into two categories. This reduction might be defensible if these categories align with clear clinical guidelines and so make results easier to operationalize. But many variables don't have this lucky property.

The more I look at this, the trickier the whole thing becomes.

Mission Creep

Professor in Training has a post on the amazing amount of time that it takes to certify a lab and/or a researcher. I find it very interesting how there has been a slow increase in the amount of overhead to do research. Each step may be both reasonable and justified. But the total effect can be quite counter-productive.

And, I must admit, that I am not altogether clear that it is a better world when the key skill for a junior professor is negotiating bureaucracy and/or finding ways around regulation. I know why making these rules apply to everyone is useful (as people avoid these things if they have any chance at all -- or at least I do). But maybe we should learn to balance total impact on research with protection of lab-workers and research subjects?

Sick Days part 2

A nice follow-up to my post yesterday about the public health issues of poor sick leave policy. This area is a place where there is a bad tendency to spout what good practice is and then put incentives in place that make following such practices highly problematic at best.

Sick Days

Do we get enough here in the United States? I wonder what the contribution to public health would look like if we made it unacceptable to show up to work or school ill. In the beginning this approach would increase absenteeism but, in the long run, it might reduce the spread of diseases across the population very effectively.

The issue is more that we have no mechanisms for verifying that people are actually ill (and herding them into an MDs office isn't an ideal solution as it pools all of the ill people together to infect older and sicker patients there for other health concerns). Add in a cultural dislike of reasonable accommodation and it is not surprising that people comply with the incentives to show up to work or school ill.

Don;t you wish that there was a better way?

Academic Reform

Any attack on an entrenched institution seems to draw strong feelings as to whether the critique is justified. However, this one seems to be flawed even by the standards of people with serious concerns about the current model for scientific research.

In general, I think questioning a model is good and necessarily to ensure strong future research. But there are several things that I think need to be kept in mind.

1) If it is obvious, has universal agreement and yet is not being done then there might be a reason for that. The idea that science and liberal arts degrees should better prepare students for work in the real world is an old idea; that it isn't being done is both because it is hard and because it diverts students from research productivity.

Incentives matters and so does viability.

2) Reforming tenure is an old canard. Let me make one point. At best speed, to become a tenured professor, one needs to spend 4 years as an undergraduate, 6 years as a graduate student and possibly as many as 5 years as a post-doctoral fellow (where salaries are typically well below what could be obtained elsewhere). After this 15 year preparation, professors then have 5 years to win grants and prove themselves worthy of tenure. This is a 20 year preparatory program, much of it at very low wages.

It is true that the reward system is loaded at the back end but reforming tenure to make being a senior professor less rewarding should ideally also consider the poor working conditions, long hours and low pay of the multi-decade training program!

Just a thought.


I really liked MsPhD's concern about what happens in a world of pure "soft money" positions. I am starting to suspect that the downsides of this paradigm (at least as currently implemented) outweigh the upsides.

Bayesian Priors

I know that it is difficult to actually create a prior that substantively influences a point estimate (except by design). But I always find it hard to justify a default prior. Is it merely that Ihave not spent enough time thinking about my data? Or am I missing some sort of trick to this process?

Aetiology

This epidemiology site has an excellent coverage of the current swine flu outbreak. It is nice to see some smart commenting on the progress of this disease and the Battlestar Galatica quotes are an added bonus.

Swine Flu

Well, it looks like there is a case in Northern California now. Joy!

It is unclear, when the numbers are so low, what will be the size of the epidemic and what will be the mortality rate. The rate of 173 death out of 1,995 infections would be alarming if we really knew that 1,995 was the correct denominator. However, there is likely to be considerable under-ascertainment in a country like Mexico among the survivors (especially those who did not seek medical care). This factor is likely to be especially important if reports of patients being turned away from medical care turn out to be widely true.

On the flip side, with only 40 cases in the US, many of which may be at the beginning of the course of the disease, we have no idea what the mortality rate is likely to be. It is still possible for it to be 0%; at which point we'll look like we over-reacted.

I think that this is a good example of the challenges of problems with lagged information (incubation times) and geometric growth rates. The point at which intervention can be effective seems to also be the point at which there is not enough data to distinguish a serious public health issue from a decidedly mild outbreak.

OT: Truly bad ideas

Mark has a nice question about a suggestion from the Ohio militia to have 1,000,000 armed people march on Washington as a demonstration.

What could go wrong?

Somehow the mind boggles . . .

Pay if forward

MsPhD brings up an interesting point: what is the duty of people in the field to ensure the success of the next generation? Modern science has reached an interesting place where the talented and engaged scientist is able to continue to contribute for an extremely long period of time. In many cases, the ability to have people with this refined level of judgment around is really important.

But there is also a tension with the newer generation. In biomedicine we have seen less of this tension lately because there was recently a big expansion of funding. But now that we are getting back to a more stable funding paradigm, it is worth asking what is the balance between mentoring new people and continuing work.

It'd be more obvious if the barriers to entry were lower -- mentoring a new investigator is hard work; can or should we find creative ways to make it more rewarding?

You can never go home

Today's post by an post-doc [chall] reminded me of my own immigration experiences. It was a very depressing and demoralizing time period; one that I hope to never have to repeat.

But it has another, unexpected, consequence. It is really hard to go home for work. This idea of working for a while back home has come up, but leaving the US for an extended period of time for employment really appears to place the Green Card at risk. I could just wait it out, become a citizen and then consider such things. But that seems like a bad approach at a number of levels.

So there really is a "burning of the boats" feeling with my time in the US. It's probably a good thing that I like the environment here a lot. :-)

Only in Canada

I encountered his thought provoking article on a science blog:

Gordon, R. & B.J. Poulin (2009). Cost of the NSERC science grant peer
review system exceeds the cost of giving every qualified researcher a
baseline grant. Accountability in Research: Policies and Quality
Assurance 16(1), 1-28.

Basically, the authors point out that (given the denominator and the success rate of NSERC grants) that we might all be better off with just awarding a fixed award to all applicants. It seems to make sense only in the Canadian context where high success rates, low indirects and a lack of soft-money positions might actually make this approach viable.

There are definitely some well founded concerns about this modest proposal, but the mere fact that it has such a strong argument underpinning it focuses us to reflect on the peer review system. Peer review is a massive burden and can lead to a "herd mentality". Some peer review is essential as you can't evaluate top experts in any other reasonable way (at least in the context of awarding grants).

Is there too much redundancy in the system?

OT: Pension Plans

A recent post has me wondering about how aware people are of how things are linked. A pension plan based on future government promise is no different than a pension plan that invests in government bonds at the most fundamental level. It doesn't matter if the promise is via a government program or via a government promise to pay out yields on bonds. Both require the future health of the government and a decision to honor obligations.

The history of bonds has many examples of where government policy worked to ensure that bonds (i.e. the basis of the National debt) were not paid. While countries like the United States and Canada are not Spain or Russia, even Britain has had some questionable episodes with national debt.

So why do people seem to assume that "defined contribution plans" based on government debt are safer than "defined benefit plans" like social security or the Canadian Pension Plan?

It is true that you could invest "defined contributions" into the stock market or corporate bonds instead. I venture to say that recent events have shown non-zero risk with corporate bonds. Stock markets have a bad tendency to have large shifts. So low risk investing is looking a lot like believing in government's ability to meet future debt obligations.

I think that once we accept this fact than the complexity of the situation is a lot more apparent.