Pay if forward

MsPhD brings up an interesting point: what is the duty of people in the field to ensure the success of the next generation? Modern science has reached an interesting place where the talented and engaged scientist is able to continue to contribute for an extremely long period of time. In many cases, the ability to have people with this refined level of judgment around is really important.

But there is also a tension with the newer generation. In biomedicine we have seen less of this tension lately because there was recently a big expansion of funding. But now that we are getting back to a more stable funding paradigm, it is worth asking what is the balance between mentoring new people and continuing work.

It'd be more obvious if the barriers to entry were lower -- mentoring a new investigator is hard work; can or should we find creative ways to make it more rewarding?

You can never go home

Today's post by an post-doc [chall] reminded me of my own immigration experiences. It was a very depressing and demoralizing time period; one that I hope to never have to repeat.

But it has another, unexpected, consequence. It is really hard to go home for work. This idea of working for a while back home has come up, but leaving the US for an extended period of time for employment really appears to place the Green Card at risk. I could just wait it out, become a citizen and then consider such things. But that seems like a bad approach at a number of levels.

So there really is a "burning of the boats" feeling with my time in the US. It's probably a good thing that I like the environment here a lot. :-)

Only in Canada

I encountered his thought provoking article on a science blog:

Gordon, R. & B.J. Poulin (2009). Cost of the NSERC science grant peer
review system exceeds the cost of giving every qualified researcher a
baseline grant. Accountability in Research: Policies and Quality
Assurance 16(1), 1-28.

Basically, the authors point out that (given the denominator and the success rate of NSERC grants) that we might all be better off with just awarding a fixed award to all applicants. It seems to make sense only in the Canadian context where high success rates, low indirects and a lack of soft-money positions might actually make this approach viable.

There are definitely some well founded concerns about this modest proposal, but the mere fact that it has such a strong argument underpinning it focuses us to reflect on the peer review system. Peer review is a massive burden and can lead to a "herd mentality". Some peer review is essential as you can't evaluate top experts in any other reasonable way (at least in the context of awarding grants).

Is there too much redundancy in the system?

OT: Pension Plans

A recent post has me wondering about how aware people are of how things are linked. A pension plan based on future government promise is no different than a pension plan that invests in government bonds at the most fundamental level. It doesn't matter if the promise is via a government program or via a government promise to pay out yields on bonds. Both require the future health of the government and a decision to honor obligations.

The history of bonds has many examples of where government policy worked to ensure that bonds (i.e. the basis of the National debt) were not paid. While countries like the United States and Canada are not Spain or Russia, even Britain has had some questionable episodes with national debt.

So why do people seem to assume that "defined contribution plans" based on government debt are safer than "defined benefit plans" like social security or the Canadian Pension Plan?

It is true that you could invest "defined contributions" into the stock market or corporate bonds instead. I venture to say that recent events have shown non-zero risk with corporate bonds. Stock markets have a bad tendency to have large shifts. So low risk investing is looking a lot like believing in government's ability to meet future debt obligations.

I think that once we accept this fact than the complexity of the situation is a lot more apparent.

How long should a post-doctoral fellowship last?

I think that we have a privileged view of this question in Epidemiology. None of the professors who were being hired when I started my PhD had ever done a post-doctoral fellowship (as other than a one year stint to get another university on their CV; only about 50% of them had done even that).

Now I am entering the second year of a post-doctoral fellowship and you can see the change where it is becoming common to do a multi-year fellowship. Although, I still see cases of people being hired into faculty directly out of the PhD or, more commonly, trying to be hired as faculty directly out of their PhD program.

So this makes me sympathetic to this post by MsPhd. It being widely panned elsewhere, but I wonder if the main point is being missed. Post-doctoral training is an ideal phase for "creep" in expectations. It's pretty clear that standards are rising in biomedicine for what it takes to succeed. Some of this is good -- a solid arms race will produce better outcomes. But the dark side of this process is people being kept back for very long periods of time. If this process leads to happy outcomes (or decent outcomes) for all involved than this is perfectly fine.

But what if it doesn't?

What is the ideal length of a fellowship training period? My field's traditional answer of "none" seems too short but more than five years seems too long. Or am I missing something?

Tenure Track?

Over in blue lab coats, a comment was made about the downside of only accepting students that looked promising for tenure track positions. Namely, this criteria would bias against students who are unlikely to be accepted into tenure track positions.

Now, this bias can take one of two forms. There can be irrational discrimination (such as age or gender-based). I think that we can all agree that this type of discrimination is a bad thing and should be avoided.

But at a more pragmatic level, the scheme where one student in ten succeeds (and these appear to be optimistic odds) at the primary career path of a 5+ year high intensity training program seems to need revision as well. If the goal of the academy is to produce the next generation of academic researchers then we need to fess up to the unfair nature of over-production.

And don't get me started on the whole "non-tenure track" movement in medical research. Vile!

Why clear methods are important

A really nice post was done today by DrugMonkey. I think that he issues that DrugMonkey raises are even more important in epidemiology than in bench science. In bench science you have the possibility of replication in a strict sense (and it is, in fact, a governing principle of bench science). In Epidemiology, the population changes between studies and it is difficult to compare between populations. As Heraclitus said "You can never step into the same river twice"; in Epidemiology you never have the same study population twice.

Perversely, this fact actually makes the methods more important as failure to replicate could also indicate unique features of a population. This makes it critical to be able to separate methodological issues from population issues, insofar as this is possible.

Given how badly medical papers seem to document decisions (part of it being a style issue for the field), there is a lot we could do to improve on matters.

Who will be successful?

Post from Professor in Training and Physio-Prof have gotten me thinking about academic evaluation procedures. The discussion about who got hired and subsequent success makes me consider one of the difficult issues in academic hiring. You want to hire people who will be successful. But, most of the time, you don't know what success will look like. So you use proxy measures; some of which are blindingly unfair. For example, the institution that a person attended could be due to brilliance or it could be due to location, connections or a number of other factors.

The same thing is true of obtaining funding. This issue seems to be the major hurdle for success as a junior academic. I seem to have no trouble with publishing papers but I have had a series of miserable failures when applying for fellowships. I was never sure why success in one domain translated into abject failure in another. Or maybe I just never understood the CIHR criteria to fund students?

But if the measure of success of a fellowship is productivity then it is odd that I never obtained one as many of my peers who were easily offered several choices had far less success at producing research. In this sense, I find the academy more difficult to succeed in than my old career as as statistician. Back then, poor prognostic signs could be overcome with hard work, smart ideas and a lot of success. People stopped caring what your alma mater was once you become highly success.

In academics, failing to get a fellowship is a reason not to promote somebody further. So once you have one thing go wrong it is much harder to get back into the pipeline. These days I have half given up on a career path and mostly stick around doing cool research. I like what I do and that is rare enough that I have kind of stopped caring about the whole "career management" issues.

But it strikes me as a sub-optimal system in a lot of ways . . .

OT: Knights of the Old Republic

While I know I am late to the table in singing the praises of this game, the storyline is remarkable. It is a very nice implementation of an interactive story and seems to strike a good balance between the level of restriction required to make voice acting viable and freedom to do interesting things. It is a good plot with a rather well foreshadowed plot twist.

But why, for what possible reason, would they insert a "difficult to beat" (for my bad reflexes at least) first person shooter with a long sequence of videos between the last save point and the shooter. I think I have seen the same 5 minutes of video so many times that I want to scream!

Otherwise a fine game!

Bayesian Model Averaging

I know that it is the best choice for selecting variables for predictive models. I know that it is massively superior to stepwise model selection. Chris Volinsky's page is a great resource for doing these models in R.

But I don't speak R and I have been resisting for a while. It's the decade of SAS training -- I am so fast in SAS that it feels like I am walking through water when I switch to R or STATA. I can do it but the experience is unpleasant, to say the least.

On the other hand, it's the right approach for a project that I am looking at right now. So maybe it is time to bite the bullet and use a new software interface?

When non-linear relationships attack

In a lot of cases, in epidemiology, it is conventional to assume that relationships are linear. In a lot of cases this assumption seems to be pretty reasonable. Occasionally you get a relationship where very large values have biological plausibility (like income or albumin excretion rate) and a base 2 logarithmic transformation is a more logical way to proceed.

So far, so good.

Now look at the rate of rate of change of bone mineral density by age for women as reported in this article from the Canadian Medical Association Journal. Between ages 25 and 40 the rate of change of bone mineral density is positive. Then, between ages 40 and 60 the rate of change is negative. Suddenly, at age 60+, the rate of change becomes positive again. Yes, the first derivative of the relationship between age and bone mineral density is quadratic! That would rather imply that the actual relationship is cubic!!

Without seeing this sort of high quality descriptive data, I would have screamed "overfit" if I saw a cubic age term in a statistical model. As it is, I am having pessimistic thoughts about just have big the sample size needs to be in order to estimate the polynomial association between age and bone mineral density.

Now imagine that you are trying to remove confounding by age from another association; just how are you going to be sure that you don't have residual confounding?

Wow, juSt wow!

Stepwise Regression

Why is it so hard to get consensus on a good variable selection technique for exploratory models? The classic one, implemented everywhere and understood by everyone, is really sub-optimal (Stepwise regression -- I am looking at you). It seems to just love to include noise parameters which make any and all models difficult to explain.

Sure, you can build a model based on theory but what do you do when you want to know what factors might be associated with an outcome? And, of course, exploratory datasets tend to be the smallest possible data sets!!

Grrrr . . . .

Reporting results

Writing an effective scientific paper is an art form. It requires presenting a complex idea with about 3000 words. However, it is always the case that there are features of the study that are hard to describe cleanly with a small number of words. So what does one do?

If you go into detail then you inevitably confuse the reader. On the other hand, you want to produce the most complete report possible. Online supplements can help but only so far.

This happened to me in this paper:

Delaney JAC, Daskalopoulou SS, Suissa S. Traditional versus marginal structural models to estimate the effectiveness of β-blocker use on mortality after myocardial infarction. Pharmacoepidemiol Drug Saf 2009; 18(1):1-6.

There were two effects going on in the same paper. One, we were accounting for time dependent confounding. Two, we were switching from a conditional to a marginal estimate. Both of these changes contribute to the differences in estimates.

But does separating them increase or decrease confusion? If the goal is to find an analytic approach that is equivalent to a randomized controlled trial then the reasons why are less important.

My question is whether I'd aid understanding by pointing these subtle points out or if I would enhance confusion by bringing in tangential points. To this day I question which approach was correct!

Budget Cuts

One of the hardest things to handle in the modern environment is that the academy is not designed for dynamism. It is based on a system where hard work and apprenticeship eventually lead to a slow succession of improvements. However, when circumstances can change rapidly this puts the whole system into flux. Positions and outcomes that people have labored for decades to achieve can now be at risk.

I think that this situation is less true for the very senior people and more for those "too far in to back out" but who have not "really made it yet". Basically, PhD Students, post-doctoral fellows and assistant professors are the class at risk and are often placed into situation that are Kafkaesque.

Now it is hard to argue for who deserves resources; "deserve" being such a complicated word. But it is pretty clear to me that the budget cuts to higher education (at least here in the state of Washington) look to be pretty deep. I suspect few companies would cut so deeply unless they were in crisis.

This leads me to ask: is higher education failing so badly that it should count as being in crisis?

But, either way, there really is not a lot of fat, per se, left to be cut. Schools seems to be in (generally) slightly poor repair. Equipment seems to old and a lot of work-arounds exist. Teaching resources are hardly in surplus. It could be that we are trying to do too much with too little. But that suggests rethinking of priorities -- not the brutal selection of massive reductions in budgets.

Or at least that is the way that I see it.

What to do in hard times

In this time of funding cuts and stress, what is the right reaction? I think that Professor in Training nails it: keep working! Stressing about what might happen is useful only insofar as it informs contingency planning. But, as a post-doctoral fellow, I have limited options that don't involve trying to develop a record of accomplishment.

One day I might get around to telling the long story of my attempts to move forward into the academy. But, for the moment, let me say that I like this attitude. Way better than assuming that disaster is coming and seeing that as an excuse for there to be at least some excitement in life. I really hope that I never get that cynical.

I like research but I admit that I also really liked working in the private sector. It had it's downsides, but I liked the dynamism and the idea that accomplishment was to be prized. I think that is why I have liked my current unit so much -- they have the same sort of culture!