Bad collaborators

I have said it before and I will say it again: good collaborators are the best gift an early career scientist can have. That makes cases like this one all the more stark!

It's pretty clear to me that nobody will come out of this particular mess happy. The main researcher will have a hard time publishing their paper. The collaborator has lost an important paper. Confusion has been created in the the scientific literature. Nobody comes out ahead.

So far I've been very lucky on this front. Here is hoping that this continues.

Bad teachers, thought experiments and anecdotal data

Statisticians often have to come up with a first draft of metrics, filters, winnowing processes, etc. without having a sample of the data they'll be using. One approach to the problem is to take some anecdotal cases and ask ourselves how the system we've proposed would handle them. Would it have trouble classifying, leaving them in some 'other' box, or worse yet, would it mis-classify them, putting something that's clearly bad into the good or even excellent category?

Here's a thought experiment. Many years ago, when teaching at a medium-sized suburban school, I had a classroom across the hall from a football coach who taught history. For the record, some of the best teachers and administrators I have ever dealt with came from coaching. They were gifted motivators who brought to the classroom the same belief in excellence and "giving 110%" that they brought to the field or the court.

This was not one of those coaches.

Not only did he make no effort to motivate his students; I'm not sure he interacted with them in any way. His desk was set up at the back of the room, not a bad arrangement for a study hall but it effectively precluded addressing the class or answering questions or leading a discussion. As far as I could tell, the issue never came up. Students spent their hour filling out worksheets that he had Xeroxed out of a workbook. He spent the hour grading them.

I have never seen a more mind-numbing, soul-crushing approach to education but that didn't stop the principal from holding up this teacher as a role model for the rest of us. His classes were quiet, he never sent a student to the principal's office, and though the student's grasp of the material seldom extended beyond the rote level, that was sufficient for pretty good standardized-test scores (at least for knowledge-based rather than process-based courses).

This was almost two decades ago. Significant chunks of the current reform movement were already in place but No Child Left Behind was still years away. The teacher in question retired the year before I entered graduate school, but assuming he was still around, how well would he do under the proposed teacher evaluation system?

Presumably, most teacher evaluation metrics will largely be based on some combination of three factors: student test scores; classroom management; and supervisor evaluations. Our worksheet-dispensing educator would normally do well on the first and would max out the other two. I said 'normally' because (as mentioned before) these metrics are easy to game and the principal could easily arrange things to bump the test scores for his favorite teacher while screwing over a trouble-making teacher he would like to get rid of (someone like me, for instance).

Even if we assume that the principal didn't play favorites (and that's not an assumption I would have made with this administrator), this teacher would unquestionably be looking at generous bonuses. The question is, is this how we want to define excellence in education?

Chart of the day

Without getting into tax policy (and way out of my area of expertise), I think this chart (from Ezra Klein) does a good job putting the debate into prospective.

FemaleScienceProfessor on Tenure

This article is well worth the read. My favorite part:

Would a system of renewable contracts really allow professors to break out of the "publish or perish" mania? Methinks it might have even the opposite effect. If there were no tenure, the rat race would never end. And, since academia is apparently equivalent to a customer service industry, consider what renewable contracts for advisers would do to their graduate students and postdocs, not to mention the research infrastructure that we build in part from grants and in part from our institutions, and use to train our advisees.

The more one thinks about the whole tenure issue, the clearer it becomes that things are not as simple as "removing tenure would improve the academy". I had not even considered the issue that professors would be training their competitors in a rotating contract system, which would definitely make the sort of long term investment strategy that we currently have hard to incent.

FSP has a more sympathetic view of Cathy Trower's piece; I'll grant that the ideas at the end of the Trower piece are an improvement over the part I like to quote. I'm not anti-reform but I would prefer that reform not consist entirely of massive changes to employment contracts introduced from above.

But it's a very well thought out piece and definitely worth a read.

Epidemiology Data

In principle, I am highly supportive of the free release of data. But the issue is very tricky in epidemiology for two reasons.

One, data can make years (or even decades to collect). Making a study with many objectives instantly publicly available would make it hard for the orginal research team to be properly credited for the work. There are solutions that might work, but so long as the primary method of credit for grant and data collection is via the papers produced this will be tricky.

Two, epidemiological data often has a lot of tricky analysis issues. It's not implausible that taking short-cuts with the data analysis and taking an overly simple approach might not result in a publication being ready more quickly. It's good for neither the main team (which now has to rush) or the reading public (which has a higher risk of scientific errors).

So the principle is good but the implementation is much harder than it looks. It really is an area waiting for a good idea.

Mentioned in passing

I've been overusing the "Save and Quit" option on Firefox too much of late, holding onto things that seemed to merit blog posts I didn't have time for. So in the interest of good desktop hygiene, here's a quick summary of some items I'd like to say more about later:

If you can manage it, there's a good reason to sleep in tomorrow.

Here's a list of the worst paying college degrees. Guess who lands at #2?

The LA Times blog has an interesting post on the relationship of toy sales and film-making.

I just learned that the LA Times lost one of its best writers. It's still a better paper than that other Times but the lead is shrinking.

America's finest fake news source remains our best source of real news analysis:

The Daily Show With Jon Stewart	Mon - Thurs 11p / 10c
Deductible Me
www.thedailyshow.com
Daily Show Full Episodes Political Humor Tea Party

Educational Reform and resources

This was an interesting suggestion on educational reform proposed by Dana Goldstein (h/t Matt Yglesias). He comments:

Rather, I'm imagining something like what the best public, private, and charter schools are already doing: a mix of additional instructional time and mealtimes with small group break-out activities like reading clubs, sports, board games, supervised computer time, library browsing time, and art and music lessons.

As a practical matter, to make this happen schools need extra labor: more hours from teachers, as well as specialized, perhaps part-time instructors in the arts and athletics.

Now I don't want to guarantee that this is a good idea. However, in a world of two working parents, a longer school day could be welfare enhancing and it's not impossible that it would have the effects on childhood obesity that are suggested by Ms Goldstein.

But there is one feature of this plan that I think it really makes sense to consider -- Ms Goldstein is discussing increasing the resources directed at schools (via meal subsidies, extra staff and additional funding) in order to improve outcomes. Could it fail? No questions. But it differs from a lot of education discussions by not trying to link a reduction of resources (e.g. removing tenure as a form of compensation) to improved outcomes. Instead, it argues that putting more resources into schools could result in a net public good.

That is a much better starting point for discussion (i.e. is this the best use of scare public resources) than the argument that removing resources will improve outcomes (so we can pay less and have a better school system).

In terms of the obesity argument, I suspect that much of this will hinge on the ability of the school to control eating and activity patterns. If students have snacks and/or school meals are not healthy then this seems less likely to work. In the same sense, putting together an activity program that succeeds in getting students to become active is not necessarily trivial.

But it certainly is worth an open discussion.

Replication

I have the opposite problem that Candid Engineer has with "fishy results". I have long advocated the central role of replication in science. This is especially important in Epidemiology where experiments are (by their nature) rare and so one needs to do most of their inference from observational research.

But how do you make a paper that has a near perfect replication seem interesting?

I mean it's good for science but it rather deadens the discussion section to have not that all much new to add except "that association is also observed in different populations".

Sigh!

Notes from a once and future reformer

This will shock those who know me personally, but when it comes to education I have always been a bit of a malcontent. As a student, I generally found grades 7 through 12 boring and, with the exception of a few really good classes, largely a waste of time. My senior year I took advantage of a program where I could go to college half the day (a much better option than AP if you can manage it). At that point high school became a brief obligation I had to take care of every morning before getting on with my life.

When I started taking education classes, I still had an attitude problem. I questioned the value of much of what was presented. I chafed at the endless buzzwords. I was suspicious of the research. I wondered (sometime out loud) about the competence of education professors who had spent little time in a non-college classroom. Once I actually made it to the classroom, things were better but there were still plenty of bad administrators, questionable standards, mind-numbing staff-development seminars and wasted potential to keep me bitching.

You might think that, given decades of accumulated dissatisfaction, I would be all for reform.

You'd be right.

The trouble is that almost none of the people using the term 'reform' are actually suggesting any reforms. Most of the proposals that have been put forward are simply continuations or extensions of the same failed policies and questionable theories that have been coming out of schools of education for years, if not decades.

Here's a small but telling example. When I was taking education classes back in the late Eighties, the professors spent endless hours discussing the proper way to write a lesson objective. Much was made of the importance of explicitly stating things in terms of student behaviors (objectives where student behavior was implicit were verboten, no matter how obvious that behavior might be). It was taken as an article of faith that subtle changes in wording could determine the outcome of a class.

Now take a look at these paragraphs from a recent reform puff piece from the Wall Street Journal editorial page:

Earlier this year, TFA released "Teaching as Leadership: The Highly Effective Teacher's Guide to Closing the Achievement Gap," which shares the practices of teachers who have made significant gains with students. One chart explains why teachers should choose an objective like "The student will be able to order fractions with different denominators," rather than "The teacher will present a lesson on ordering fractions with different denominations."

Objectives, says the guide, should be "student-achievement based, measurable and rigorous." Seem obvious? Well, as Ms. Kopp says, successful teaching is "nothing magic. It's nothing elusive. It's about talent and leadership and accountability."

Imagine you're in a bad pulp novel, captured by a cult that celebrates every full moon by sacrificing a half dozen virgins to a giant cabbage. You escape, free a group of dissidents and lead them to safety, only to find out they want to substitute a rutabaga for the cabbage and go for a full dozen virgins.

For me, following the reform debate has been one long concatenation of rutabaga moments.

Distributions

John Cook has another insightful post today on data distributions. It is an area that I know that I could stand to develop a fuller intuition about.

One point that I think comes out of his post is the idea that no real data ever fully fits a theoretical distribution. Ever. After all, all real data has noise in it. While this seems to be a obvious point, it is possible to get reasonable patterns of residuals using different distributions to fit data. Which could mean it is either distribution, or possibly neither.

Even worse, real data is sometimes a mixture of distributions based on a latent (or non-latent variables). Andrew Gelman has an example with height -- male and female adult heights are both approximately normally distributed but the combination of the two is not (there is a nice picture on page 14 of "Data Analysis using regression and multilevel/hierarchical models").

Even worse, there may be factor (e.e. genetic) that result in different mean adult heights. So you can get "fat tails". This is not a small problem as it means that your models will wildly underestimate the probability of an extreme result. I was not a huge fan of the Black Swan, but this point was correct (and, to be fair, it was the central theme of the book).

All of which is to say that I am definitely going to have to think more about this issue and , hopefully, see cases where I am not making the correct assumptions.

Complex processes are hard to simplify

One thing that I have noticed is that simple narratives about employment often gloss over important details. Mark has an example with teacher employment. Yves Smith has another one, where she points out that employers who are having trouble finding employees often manage this only by offering very poor terms of employment:

This does not add up. If the company can afford to spend ten weeks training people (and the additional cost of setting up a course), that suggests it could have offered more than $13 an hour, particularly given the opportunity cost of the orders it could have filled if it had had people on board sooner. The article later notes that Mechanical Devices “hire[s] through staffing agencies to help control health-care costs and maintain flexibility.” Um, that means they fire people as soon as orders fall.

I think that this issue is an old idea that we often forget. The jobs that are difficult to fill (in a functional economy) either require complex skills/credentials or have a known downside. I would worry that Yves Smith understates the case with her example: the worst of jobs are the ones where you never actually become unemployed but for which you don’t have a predictable income. Economists back to Adam Smith have noted that a lack of job security tends to raise wages.

Are we surprised that experiments with unpredictable income and low wages don't always work out? After all, there is a minimum income threshold below which basics like food, shelter and hygiene may start to become difficult to maintain.

In the same sense, why are we worried that it is hard to break into education? Many professions are hard to break into without, for example, credentials. Neither lawyers nor medical doctors can practice without the specific credential? Why is the lack of a graduate degree making it hard to become a teacher in competive markets an issue?

I suppose we could re-imagine the economy from the ground up. But top down approaches to comprehensive social and economic reform don't always result in wild successes and may generate adverse effects in the process.

This is not to say that there are not real issues to be resolved that can occur due to technological change. I just worry when the proposed solutions to a complex problem are extremely simple. That suggests either an agenda or a failure to appreciate all of the moving parts involved.

Now, before I get to be labeled an arch-conservative, I tend to think that slow and incremental reform is the best choice in most cases. There can be specific cases where the slow reform approach has failed completely and where the options are limited, but I'd rather be cautious rather than aggressive in finding them.

A better-late-than-never edition of reasons not to trust Naomi Schaefer Riley

As we used to say back in the Ozarks, I've been busier than a one-legged man at in an ass-kicking contest, so I still don't have time to go through the logical and factual problems with this piece in the Wall Street Journal Op-Ed section, but I have to take a moment to point out that Riley defines 'impossible' in much the same way that Newman defines 'rarely.'

In the spring of 1989 Wendy Kopp was a senior at Princeton University who had her sights set on being a New York City school teacher. But without a graduate degree in education or a traditional teacher certification, it was nearly impossible to break into the system. So she applied for a job at Morgan Stanley instead.

"Nearly impossible' in this case means 'mildly inconvenient.' It was no big deal. Lots of people did it. I know this because at around the same time Ms. Kopp was cranking out spreadsheets in Lotus 1-2-3, I was teaching high school despite the fact that I had neither an education degree nor traditional teacher certification.

I'm not saying Ms. Riley is woefully ignorant in her area of supposed expertise. I'm not saying that she's a liar. I'm just saying we've got the possibilities narrowed down.

Social Security Thought

I am not an economist and I don’t necessary have a strong understanding of personal finance. Mark brought up the issue of wealth transfer as a part of the whole social security debate. But I will admit to one additional element of puzzlement when it comes to privatizing social security.

Namely, precisely what would one invest in? If one is investing in government bonds (the safest instruments) then isn't the government just writing an IOU to the bondholders? The only difference I can see between a personal account investing in US treasury bonds and social security is that social security can be amended, like in 1983, without a default on United States bonds.

Or we could invest in stocks, but them we have risk due to issues like decades with a net negative return. I'd always naively assumed that the real role of programs like social security was to smooth out variability in returns (as the government can afford to plan for the real long run) and that'd seem to be defeated by personal accounts.

Hiltzik puts SS in plain English

Michael Hiltzik is one of the reasons I still feel that, despite cruel budget cuts and bad management, the LA Times is a better paper than the NYT. The latter gives us stories like this. Hiltzik gives us this:

What trips up many people about the trust fund is the notion that redeeming the bonds in the fund to produce cash for Social Security is the equivalent of "the government" paying money to "the government." Superficially, this resembles transferring a dollar from your brown pants to your gray pants — you're no more or less flush than you were before changing pants.

But that assumes every one of us contributes equally to "the government," and by equal methods — you, me and the chairman of Goldman Sachs.

The truth is that there are two separate tax programs at work here — the payroll tax and the income tax — and they affect Americans in different ways. The first pays for Social Security and the second for the rest of the federal budget.

Most Americans pay more payroll tax than income tax. Not until you pull in $200,000 or more, which puts you among roughly the top 5% of income-earners, are you likely to pay more in income tax than payroll tax. One reason is that the income taxed for Social Security is capped — this year, at $106,800. (My payroll and income tax figures come from the Brookings Institution, and the income distribution statistics come from the U.S. Census Bureau.)

Since 1983, the money from all payroll taxpayers has been building up the Social Security surplus, swelling the trust fund. What's happened to the money? It's been borrowed by the federal government and spent on federal programs — housing, stimulus, war and a big income tax cut for the richest Americans, enacted under President George W. Bush in 2001.

In other words, money from the taxpayers at the lower end of the income scale has been spent to help out those at the higher end. That transfer — that loan, to characterize it accurately — is represented by the Treasury bonds held by the trust fund.

The interest on those bonds, and the eventual redemption of the principal, should have to be paid for by income taxpayers, who reaped the direct benefits from borrowing the money.

So all the whining you hear about how redeeming the trust fund will require a tax hike we can't afford is simply the sound of wealthy taxpayers trying to skip out on a bill about to come due. The next time someone tells you the trust fund is full of worthless IOUs, try to guess what tax bracket he's in.

Quote of the day

In Unqualified Offerings, Thoreau made the following quote about reform in higher education:

In any other line of business, a person who says “How hard can it be to do this efficiently?” is usually a clueless idiot. Ditto for many aspects of higher education.

It gives one pause.