Post from Professor in Training and Physio-Prof have gotten me thinking about academic evaluation procedures. The discussion about who got hired and subsequent success makes me consider one of the difficult issues in academic hiring. You want to hire people who will be successful. But, most of the time, you don't know what success will look like. So you use proxy measures; some of which are blindingly unfair. For example, the institution that a person attended could be due to brilliance or it could be due to location, connections or a number of other factors.
The same thing is true of obtaining funding. This issue seems to be the major hurdle for success as a junior academic. I seem to have no trouble with publishing papers but I have had a series of miserable failures when applying for fellowships. I was never sure why success in one domain translated into abject failure in another. Or maybe I just never understood the CIHR criteria to fund students?
But if the measure of success of a fellowship is productivity then it is odd that I never obtained one as many of my peers who were easily offered several choices had far less success at producing research. In this sense, I find the academy more difficult to succeed in than my old career as as statistician. Back then, poor prognostic signs could be overcome with hard work, smart ideas and a lot of success. People stopped caring what your alma mater was once you become highly success.
In academics, failing to get a fellowship is a reason not to promote somebody further. So once you have one thing go wrong it is much harder to get back into the pipeline. These days I have half given up on a career path and mostly stick around doing cool research. I like what I do and that is rare enough that I have kind of stopped caring about the whole "career management" issues.
But it strikes me as a sub-optimal system in a lot of ways . . .