Evaluating evidence

I want to steal a quote from Paul Krugman to illustrate a point:

And in the end, Ryan’s answer is that we need strong economic growth, the kind that we get by cutting taxes on the rich. Because that’s why the Clinton years were an economic disaster and the Bush years so prosperous.

Is this strong evidence?

First of all, we need to consider a number of causal hypotheses:

1) Tax rates on the rich are unrelated to economic growth
2) Higher tax rates on the rich increase economic growth
3) Economic growth makes it easier to tax the rich
4) Higher tax rates on the rich decrease economic growth

Then we need to consider lags between tax policy and changes in economic growth. I am suspicious of anyone who says that this is an easy problem. After all, what we really want (the counterfactual of what would happen if Bush/Clinton not changed tax policy) is completely unavailable.

So what value is this evidence?

It does rule out one very clear talking point in the debate. It suggests that moderate changes in tax policy (Bush Tax cuts) do not have a stronger effect on economic growth than the economic fundamentals do. We may even take this as weak evidence of hypothesis #4 above (with all of the caveats about not being able to make a strong inference).

So the ideas that tax cuts [focused on high income earners] are a good response to short term problems with weak economic growth seems to be contrary to the best evidence available. Nor does looking at period like the 1950' (with very high marginal rates and rapid growth) seem to provide a lot of support for Hypothesis #2.

But if it is case that Hypothesis #2 is true, we know that it is unlikely to overcome other economic issues (or it would have made the Bush years a time of prosperity). Or, in other words, that the overall effect size of this tax policy change is small relative to other factors (if it works in the direction predicted by Hypothesis #2). Now one can reframe this as a moral question, and some do.

But it is worth considering that, in the absence of controlled experiments, how do we update our expectations when a strategy that sounds reasonable doesn't seem to give expected results.

Why aren't you reading the incidental economist?

Because if you care about health care, they are one of the most informative blogs around for those of us in the medical research community.

Consider this statistic:

By 2010, more than 60% of people lived in areas where insurance premiums cost at least 20% of their income. And that’s just premiums; it doesn’t include deductibles, it doesn’t include co-pays, and it doesn’t include co-insurance.

This is likely unsustainable. The growth rate of insurance is far above that of wages, meaning that health care costs are going to consume a higher and higher percent of people’s incomes in the future. Moreover, this is a problem of the non-elderly. Because of Medicare, few elderly have premiums which consume this level of income.

This statistic very nicely frames the entire underlying issue with the explosion in medical costs. Placed in such stark terms, the question shifts from "can we reduce medical costs" to "how are we going to reduce medical costs".

Hard work

Noah Smith has a couple of interesting posts up, but the one that I really found interesting was "Why conservatives can't get people to work hard". It had several insightful comments including the classic:

One basic idea is that hard work should be rewarded. Obvious, right? I mean, we're supposed to be economists here! People respond to incentives, and they are risk averse. A winner-take-all society is not very conducive to hard work; I'm not going to bust my butt for 30 years for a 1% shot at getting into The 1%. But I am going to bust my butt for 30 years if I think this gives me a 90% chance of having a decent house, a family, some security, a reasonably pleasant job, a dog, and a couple of cars in my garage. An ideal middle-class society is one in which everyone, not just anyone, can get ahead via hard work.

Even more interesting, he points out the underlying ambivalence among conservatives as to whether hard work has a causal link to productivity:

Conservatives, meanwhile, are all too often divided on whether they actually believe that hard work works. Plenty of conservatives have undermined Cowen's hard-work-and-discipline bloc by saying that success in life is all due to natural differences in ability. These "I.Q. conservatives" see inequality as the natural order of things. They have focused on getting people to accept their place in society and learn to live with what they have, rather than strive to move up in the world. This is a very Old British sort of conservatism, a nobility-and-peasants ethos dressed up in the faux modernism of psychometric testing.

Conservatives need to look in the mirror and ask themselves: "Do we really want people to work hard and be disciplined? Or do we just say that in order to keep the peasants from getting restless, when deep down we believe that it's all about good genes?" Because if it's the former, conservatives should do some hard thinking about what actually gets people to work hard. And they should think about how to respond to those among their colleagues for whom it is simply the latter.

I think that the "I.Q. conservatives" (as Noah calls them) are actually a fairly concerning movement. We all know that social structures based on accepting one's lot in life (think feudalism) have shockingly low levels of productivity. A social creed that suggests that this lack of productivity is due to innate personal differences is also one that cannot address any social dysfunction that may be present. After all, if the reason person A is successful is that they are the "right sort of person" then we don't have to handle questions like "why is person B unsuccessful".

A broad adoption of this ethos would be an unfortunate outcome for any society because it then concentrates decision making ability into a more and more restricted class. Democracy and capitalism succeed by making the information base broad. It's not that they always succeed in creating good outcomes. But the track record of a narrow elite making decisions is . . . poor.

Laboratory animals in biomedicine -- a recycled reply

In response to Joseph's recent post on the over-reliance on mice in medical research (which was prompted by this thought-provoking piece in Slate), I thought I'd dredge up something I wrote on the subject last year:

Landscapes and Lab Rats

In this post I discussed gradient searches and the two great curses of the gradient searcher, small local optima and long, circuitous paths. I also mentioned that by making small changes to the landscape being searched (in other words, perturbing it) we could sometimes (with luck) improve our search metrics without significantly changing the size and location of our optima.

The idea that you can use a search on one landscape to find the optima of a similar landscape is the assumption behind more than just perturbing. It is also the basis of all animal testing of treatments for humans. This brings genotype into the landscape discussion, but not in the way it's normally used.

In evolutionary terms, we look at an animal's genotype as a set of coordinates for a vast genetic landscape where 'height' (the fitness function) represents that animal's fitness. Every species is found on that landscape, each clustering around its own local maximum.

Genotype figures in our research landscape, but instead of being the landscape itself, it becomes part of the fitness function. Here's an overly simplified example that might clear things up:

Consider a combination of two drugs. If we use the dosage of each drug as an axis, this gives us something that looks a lot like our first example with drug A being north/south, drug B being east/west and the effect we're measuring being height. In other words, our fitness function has a domain of all points on our AB plane and a range corresponding to the effectiveness of that dosage. Since we expect genetics to affect the subjects react to the drugs, genotype has to be part of that fitness function. If we ran the test on lab rats we would expect a different result than if we tested it on humans but we would hope that the landscapes would be similar (or else there would be no point in using lab rats).

Scientists who use animal testing are acutely aware of the problems of going from one landscape to another. For each system studied, they have spent a great deal of time and effort looking for the test species that functions most like humans. The idea is that if you could find an animal with, say, a liver that functions almost exactly like a human liver, you could do most of your controlled studies of liver disease on that animal and only use humans for the final stages.

As sound and appealing as that idea is, there is another way of looking at this.

On a sufficiently high level with some important caveats, all research can be looked at as a set of gradient searches over a vast multidimensional landscape. With each study, researchers pick a point on the landscape, gather data in the region then use their findings to pick their findings and those of other researchers to pick their next point.

In this context, important similarities between landscapes fall into two distinct categories: those involving the positions and magnitudes of the optima; and those involving the search properties of the landscape. Every point on the landscape corresponds to four search values: a max; the number of steps it will take to reach that max; a min; and the number of steps it will take to reach that min. Since we usually want to go in one direction (let's say maximizing), we can generally reduce that to two values for each point, optima of interest and time to converge.

All of this leads us to an interesting and somewhat counterintuitive conclusion. When searching on one landscape to find the corresponding optimum of another, we are vitally interested in seeing a high degree of correlation between the size and location of the optima but given that similarity between optima, similarity in search statistics is at best unimportant and at worst a serious problem.

The whole point of repeated perturbing then searching of a landscape is to produce a wide range of search statistics. Since we're only keeping the best one, the more variability the better. (Best here would generally be the one where the global optimum is associated with the largest region though time to converge can also be important.)

In animal testing, changing your population of test subjects perturbs the research landscape. So what? How does thinking of research using different test animals change the way that we might approach research? I'll suggest a few possibilities in my next post on the subject.

The limitations of laboratory animals in biomedicine

There is a very interesting take in Slate on the reliance of Mouse Models in biomedical science. You can see the temptation in a fast growing animal that elicits limited sympathy from the general public. But the concern with missing key drugs is real -- especially when the diseases in mice operate differently than in humans.

But the real kicker was the discovery that control mice are overfed, under-stimulated and obese. This puts an entirely new spin on studies that restrict caloric intake. They may be saying that eating a normal diet is life enhancing and not caloric restriction. These sorts of blind alleys can have enormous consequences.

Go and read -- it is worth it.

How not to convince people

The approach taken by Occupy Seattle here seems to be deeply counter-productive. I think that new ideas and paths forward require us to spend less time in "gumming up process" and more time deciding how to address real problems.

Not a good start for a dynamic movement seeking change.

Tone poem -- how Duncan Black and company missed the big picture

I'd thought about blogging this yesterday, then I decided that too much had been written about Penn State, at least too much by people like me who knew nothing about the subject. I reconsidered when I followed this link from Brad DeLong. DeLong, Black and co. see David Brooks' statements as standard issue liberal bashing, but I think they missed a more significant part of story.

Take a look at this excerpt from All Things Considered:

RAZ: In the just short time we have left, E.J., I want to ask you about Penn State - both of you. Obviously, the university president and the head coach, Joe Paterno, were fired by the board of trustees.

Do you think they should've considered shutting down the program for a year?

DIONNE: Yeah. Well, you look at that indictment, I mean, what happened was hideous. What was done to kids, 10-year-old boy and others that young, was just awful. And you had an institution that seemed more interested in self-protection than anything else. And we've seen that before.

And I understand Joe Paterno is a much-loved figure in sports terms. He was one of the better college coaches. His kids graduated. But this entire episode is so ugly and it, again, you hate institutional protection over the interests of little kids.

RAZ: David.

BROOKS: Yeah, I guess think - it's I have a bigger view, which is that when we have a society where we don't know how to handle the concept of evil when we see it, we don't know how to deal with it, we're not really aware of it and people hid away. I do not think they should shut down the program, however. I think a lot of very honest football players have committed themselves to that program. I don't think they should be punished.

If you've followed any of the major debates going on now, you have to be discouraged by the lack of progress. You will see some good substantive arguments but more often the pundits will simply focus on maintaining the appropriate tone and making the right associations.

David Brooks is a master of this kind of punditry. That skill has allowed him to maintain his reasonable conservative persona under very difficult circumstances (look at what happened to David Frum). See how he hits both notes here, first chastising us for moral relativism then warning against excessive reaction. In terms of tone, this combination of conservative values and measured responses is almost perfect. As an argument, though, it's gibberish.

There is no way to reconcile the two sides here. You can't condemn society for lacking the strength and clarity of vision to deal with evil then recommend that we not punish an institution that tolerated and enabled genuine atrocities. Brooks, who is a very intelligent man, undoubtedly knows what he said makes no sense, but he also knows that, as long as maintains that proper tone, most people won't care.

Cross sectional reasoning

I want to follow up on a post by Bad Astronomy that has been discussed by both Mark and Noah Smith. The post comments on the results of the 2010 census on employment rates:

I highlighted one in particular: Astronomy and Astrophysics. Note that it has a 0% unemployment rate; in other words, last year everyone who majored in these fields got a job! Now, I find myself being a tad skeptical about this, but if there’s some weird thing going on with this survey, I can at least make the broad assumption that the relative job numbers are probably OK. Majoring in astronomy is still a good idea, and will strengthen your chances of getting a job after college.

I want to take this in a different direction. What this metric shows is that, if you were lucky enough to have majored in Astronomy in 2005 then you were very likely to be employed in 2010. It says nothing about what will happen to somebody who enters the program in 2011 and whether they will be employed in 2015.

See, I was actually a physics major in the mid 1990's, in a school with a large astrophysics group. I knew a lot of these students and even took classes with them. Do you know what they mostly ended up as: High School Teachers. Plus a few academics. At the time there was a terrible job placement rate in physics and we were all depressed by the poor employment outcomes. Using the tool, I see a 4.5% unemployment rate for physics, which does make me wonder how many astrophysicists are counted in this group.

But, in general, past performance is no guarantee of future employment. A depressed job market could easily have led to full employment years later, long after only the most dedicated students remained. I've seen this phenomenon in a lot of fields -- people go where the markets signal but, in education, the signals are lagging indicators.

So maybe we are seeing the unemployment ghettos of the future?

Another "It's too late tonight to do it justice..."

Noah Smith has an excellent post on the employment picture for science major with important implications for our ongoing mathematics education debate:

No, a science major does not guarantee you a job...

I'm not knowledgeable enough to sufficiently mock this* #

DNA Testing Could Help Choose Your Kid's Sport





* Yeah, it's a split infinitive. What are you, the grammar police?

# Now with no words omitted from the title.

Libertarians in Space

Herman Cain gave a speech to a group of Young Republicans and the subject of the space shuttle came up:

In his speech, Cain praised President John F. Kennedy as a "great leader" for inspiring a national effort to put a man on the moon, a goal achieved when astronaut Neil Armstong stepped onto the moon's surface in 1969.

"He didn't say, 'We might.' He didn't say, 'Let's take a poll,'" Cain said. "He said, 'We will.' And we did. Only for this president to move us back by canceling a major part of our space program."

Cain also criticized Obama for using Russian technology to ferry astronauts and cargo to the International Space Station.

"I can tell you that as president of the United States, we are not going to bum a ride to outer space with Russia," Cain said to loud applause. "We're going to regain our rightful place in terms of technology, space technology."

I don't know what the reaction of the crowd was (the reporting wasn't that detailed) but I'd imagine it was friendly. You can usually get a warm response from a Republican crowd by coming out in favor of manned space exploration which is, when you think about, strange as hell.

If you set out to genetically engineer a program that libertarians ought to object to, you'd probably come up with something like the manned space program. A massive government initiative, tremendously expensive, with no real role for individual initiative. Compared to infrastructure projects the benefits to business are limited. You could even argue that the government's presence in the field crowds out private development.

(Much has been made of the rise of private space firms, but barring a really big and unexpected technological advance, their role is going to be limited to either unmanned missions or human flights in low earth orbit for the foreseeable future.)

There have been efforts in libertarian-leaning organs (The Wall Street Journal, Reason, John Tierney's NYT columns) trying to argue that interplanetary exploration can be done on the cheap. These usually rely heavily on the blatant low-balling of Robert Zubrin* (Tierney, a science writer who has no grasp of science, made a particularly ripe mark), but even if we were to accept these numbers, it's still difficult to reconcile this kind of government program with libertarian values.


* On a related note, check out this other example of Zubrin's estimation skills.

Tex Avery animates David Frum's sudden move to the center

At about ninety seconds in.

Pre-blogging Jack Shafer

I've got a couple of posts coming up on related subjects so I thought I'd get this out of the way in advance.

Here's Jack Shafer explaining how his intense antipathy toward plagiarism is all about the readers (via Salmon):

The plagiarist defrauds readers by leading them to believe that he has come by the facts of his story first-hand–that he vouches for the accuracy of the facts and interpretations under his byline. But this is not the case. Generally, the plagiarist doesn’t know whether the copy he’s lifted has gotten the story right because he hasn’t really investigated the topic. (If he had, he could write the story himself.) In such cases he must attribute the material he borrows so that at the very least the reader can hold somebody accountable for the facts in a story.

Putting aside the fact that Shafer has never gotten that worked up about colleagues' inaccuracy (you'll notice he didn't jump on this story about his friend Gregg Easterbrook), there's an interesting game here of rhetorical Three-card Monte. The cards in this game are the three types of plagiarism: theft of ideas and interpretations; theft of facts and data; and theft of wording.

In the world of research, the first two are considered the most serious. Stealing the hypothesis of another paper or presenting someone else's data as your own is about the worst thing you can do. Lifting a passage of someone else's writing is frowned upon but prose style does not drive impact factors.

For journalists, the situation is exactly reversed. Reusing another writer's phrases is clearly considered the worst kind of plagiarism, perhaps the worst journalistic crime period. Stealing facts (such as using other people's reporting to cover an event) is seldom even mentioned except in the most flagrant of cases. As for appropriating ideas, the practice is so common as to almost be standard. Even those most modern of journalistic concepts, memes, are almost always based largely on the plagiarism of hypotheses and arguments.

Now, here's the part where the cards really start to move. Shafer's criticism only applies to the first two types of plagiarism, the two types he doesn't object to. (aren't the first two nested in the third? Sometimes, but Shafer apparently doesn't have a problem with borrowing and paraphrasing, it must be the not-paraphrasing part that bothers him). If Shafer really wants to convince us that borrowing without paraphrasing is more than journalist-on-journalist crime, he'll have to do better than that.

Health Care Costs

We have known for a while that the United States spends a lot on health care. What is interesting is that the direction seems to be moving in the wrong way both in terms of life expectancy and cost:

You can see that not only is the United States the outlier when it comes to spending, but we are moving in the wrong direction: we are becoming more of a spending outlier, and we are drifting down from the average life expectancy into the lower group (currently surpassing only Turkey, Hungary, Mexico, Poland, and Czech Republic).

and

The other thing you see is that our life expectancy gain was the absolute lowest of the whole group (and we weren’t starting from a particularly high level, as you can see in the previous chart).

Ordinarily, you would think there should be convergence across countries. Since other countries spend less and live longer, you would think that we would learn from them—global competition, you know. But instead we’re moving the wrong way on both dimensions.

The article and neat charts are worth looking at in their entirety.

Now, it is true that there can be a lot of reasons for low life expectancy and high medical costs. It could be that the environment in the United States makes us much more accident prone, for example, requiring both higher spending and more fatalities.

But, in general, it is uncomfortable when the most important metric of health care outcomes (all cause mortality) is so uncorrelated with cost. This suggests the possibility of productivity improvements. I read a lot of the Incidental Economist, who try to explain these issues. But I admit that I tend to come away confused.

The major comparison is often Canada. It is a bad reference on a lot of levels (as they have their own issues). But they have similar culture, ethnic diversity, large geography, heavy use of cars, high levels of obesity and yet they are improving on both metrics (from a lower level of cost and higher life expectancy at baseline).

Why is health care the one area that we aren't willing to look at how other countries have been successful and try to steal ideas?

Old story, different hemisphere

More good work from American Public Media's Marketplace:

Rob Schmitz: If you walk down the street in Shanghai, it's hard to miss the advertisements for college placement agencies. In big bold letters, they promise -- some even guarantee -- your child's admission to an American university. The price: $5,000, $6,000, sometimes $7,000.

Jiang Xueqin is a well-known education reformer in China. He also heads the International division at Peking University High School -- one of China's best high schools. He has followed college placement agencies for years. He says this is how many of them deliver on their admission promises: they falsify students application materials.

Jiang Xueqin: There's a lot of pressure on the agencies to write the application essays, to fake transcripts, to fake recommendation letters. This is just general business practice in China to falsify a lot of documentation.

A report by consulting firm Zinch China seems to confirm this. Zinch advises American colleges and universities on recruiting Chinese students. The firm interviewed agents and admissions consultants, as well as more than 200 Beijing students headed to U.S. schools. Zinch estimates 90 percent of these students submitted false recommendation letters; 70 percent had other people write their personal essays, and half of them submitted forged high school transcripts.