Laboratory animals in biomedicine -- a recycled reply

In response to Joseph's recent post on the over-reliance on mice in medical research (which was prompted by this thought-provoking piece in Slate), I thought I'd dredge up something I wrote on the subject last year:

Landscapes and Lab Rats

In this post I discussed gradient searches and the two great curses of the gradient searcher, small local optima and long, circuitous paths. I also mentioned that by making small changes to the landscape being searched (in other words, perturbing it) we could sometimes (with luck) improve our search metrics without significantly changing the size and location of our optima.

The idea that you can use a search on one landscape to find the optima of a similar landscape is the assumption behind more than just perturbing. It is also the basis of all animal testing of treatments for humans. This brings genotype into the landscape discussion, but not in the way it's normally used.

In evolutionary terms, we look at an animal's genotype as a set of coordinates for a vast genetic landscape where 'height' (the fitness function) represents that animal's fitness. Every species is found on that landscape, each clustering around its own local maximum.

Genotype figures in our research landscape, but instead of being the landscape itself, it becomes part of the fitness function. Here's an overly simplified example that might clear things up:

Consider a combination of two drugs. If we use the dosage of each drug as an axis, this gives us something that looks a lot like our first example with drug A being north/south, drug B being east/west and the effect we're measuring being height. In other words, our fitness function has a domain of all points on our AB plane and a range corresponding to the effectiveness of that dosage. Since we expect genetics to affect the subjects react to the drugs, genotype has to be part of that fitness function. If we ran the test on lab rats we would expect a different result than if we tested it on humans but we would hope that the landscapes would be similar (or else there would be no point in using lab rats).

Scientists who use animal testing are acutely aware of the problems of going from one landscape to another. For each system studied, they have spent a great deal of time and effort looking for the test species that functions most like humans. The idea is that if you could find an animal with, say, a liver that functions almost exactly like a human liver, you could do most of your controlled studies of liver disease on that animal and only use humans for the final stages.

As sound and appealing as that idea is, there is another way of looking at this.

On a sufficiently high level with some important caveats, all research can be looked at as a set of gradient searches over a vast multidimensional landscape. With each study, researchers pick a point on the landscape, gather data in the region then use their findings to pick their findings and those of other researchers to pick their next point.

In this context, important similarities between landscapes fall into two distinct categories: those involving the positions and magnitudes of the optima; and those involving the search properties of the landscape. Every point on the landscape corresponds to four search values: a max; the number of steps it will take to reach that max; a min; and the number of steps it will take to reach that min. Since we usually want to go in one direction (let's say maximizing), we can generally reduce that to two values for each point, optima of interest and time to converge.

All of this leads us to an interesting and somewhat counterintuitive conclusion. When searching on one landscape to find the corresponding optimum of another, we are vitally interested in seeing a high degree of correlation between the size and location of the optima but given that similarity between optima, similarity in search statistics is at best unimportant and at worst a serious problem.

The whole point of repeated perturbing then searching of a landscape is to produce a wide range of search statistics. Since we're only keeping the best one, the more variability the better. (Best here would generally be the one where the global optimum is associated with the largest region though time to converge can also be important.)

In animal testing, changing your population of test subjects perturbs the research landscape. So what? How does thinking of research using different test animals change the way that we might approach research? I'll suggest a few possibilities in my next post on the subject.

The limitations of laboratory animals in biomedicine

There is a very interesting take in Slate on the reliance of Mouse Models in biomedical science. You can see the temptation in a fast growing animal that elicits limited sympathy from the general public. But the concern with missing key drugs is real -- especially when the diseases in mice operate differently than in humans.

But the real kicker was the discovery that control mice are overfed, under-stimulated and obese. This puts an entirely new spin on studies that restrict caloric intake. They may be saying that eating a normal diet is life enhancing and not caloric restriction. These sorts of blind alleys can have enormous consequences.

Go and read -- it is worth it.

How not to convince people

The approach taken by Occupy Seattle here seems to be deeply counter-productive. I think that new ideas and paths forward require us to spend less time in "gumming up process" and more time deciding how to address real problems.

Not a good start for a dynamic movement seeking change.

Tone poem -- how Duncan Black and company missed the big picture

I'd thought about blogging this yesterday, then I decided that too much had been written about Penn State, at least too much by people like me who knew nothing about the subject. I reconsidered when I followed this link from Brad DeLong. DeLong, Black and co. see David Brooks' statements as standard issue liberal bashing, but I think they missed a more significant part of story.

Take a look at this excerpt from All Things Considered:

RAZ: In the just short time we have left, E.J., I want to ask you about Penn State - both of you. Obviously, the university president and the head coach, Joe Paterno, were fired by the board of trustees.

Do you think they should've considered shutting down the program for a year?

DIONNE: Yeah. Well, you look at that indictment, I mean, what happened was hideous. What was done to kids, 10-year-old boy and others that young, was just awful. And you had an institution that seemed more interested in self-protection than anything else. And we've seen that before.

And I understand Joe Paterno is a much-loved figure in sports terms. He was one of the better college coaches. His kids graduated. But this entire episode is so ugly and it, again, you hate institutional protection over the interests of little kids.

RAZ: David.

BROOKS: Yeah, I guess think - it's I have a bigger view, which is that when we have a society where we don't know how to handle the concept of evil when we see it, we don't know how to deal with it, we're not really aware of it and people hid away. I do not think they should shut down the program, however. I think a lot of very honest football players have committed themselves to that program. I don't think they should be punished.

If you've followed any of the major debates going on now, you have to be discouraged by the lack of progress. You will see some good substantive arguments but more often the pundits will simply focus on maintaining the appropriate tone and making the right associations.

David Brooks is a master of this kind of punditry. That skill has allowed him to maintain his reasonable conservative persona under very difficult circumstances (look at what happened to David Frum). See how he hits both notes here, first chastising us for moral relativism then warning against excessive reaction. In terms of tone, this combination of conservative values and measured responses is almost perfect. As an argument, though, it's gibberish.

There is no way to reconcile the two sides here. You can't condemn society for lacking the strength and clarity of vision to deal with evil then recommend that we not punish an institution that tolerated and enabled genuine atrocities. Brooks, who is a very intelligent man, undoubtedly knows what he said makes no sense, but he also knows that, as long as maintains that proper tone, most people won't care.

Cross sectional reasoning

I want to follow up on a post by Bad Astronomy that has been discussed by both Mark and Noah Smith. The post comments on the results of the 2010 census on employment rates:

I highlighted one in particular: Astronomy and Astrophysics. Note that it has a 0% unemployment rate; in other words, last year everyone who majored in these fields got a job! Now, I find myself being a tad skeptical about this, but if there’s some weird thing going on with this survey, I can at least make the broad assumption that the relative job numbers are probably OK. Majoring in astronomy is still a good idea, and will strengthen your chances of getting a job after college.

I want to take this in a different direction. What this metric shows is that, if you were lucky enough to have majored in Astronomy in 2005 then you were very likely to be employed in 2010. It says nothing about what will happen to somebody who enters the program in 2011 and whether they will be employed in 2015.

See, I was actually a physics major in the mid 1990's, in a school with a large astrophysics group. I knew a lot of these students and even took classes with them. Do you know what they mostly ended up as: High School Teachers. Plus a few academics. At the time there was a terrible job placement rate in physics and we were all depressed by the poor employment outcomes. Using the tool, I see a 4.5% unemployment rate for physics, which does make me wonder how many astrophysicists are counted in this group.

But, in general, past performance is no guarantee of future employment. A depressed job market could easily have led to full employment years later, long after only the most dedicated students remained. I've seen this phenomenon in a lot of fields -- people go where the markets signal but, in education, the signals are lagging indicators.

So maybe we are seeing the unemployment ghettos of the future?

Another "It's too late tonight to do it justice..."

Noah Smith has an excellent post on the employment picture for science major with important implications for our ongoing mathematics education debate:

No, a science major does not guarantee you a job...

I'm not knowledgeable enough to sufficiently mock this* #

DNA Testing Could Help Choose Your Kid's Sport





* Yeah, it's a split infinitive. What are you, the grammar police?

# Now with no words omitted from the title.

Libertarians in Space

Herman Cain gave a speech to a group of Young Republicans and the subject of the space shuttle came up:

In his speech, Cain praised President John F. Kennedy as a "great leader" for inspiring a national effort to put a man on the moon, a goal achieved when astronaut Neil Armstong stepped onto the moon's surface in 1969.

"He didn't say, 'We might.' He didn't say, 'Let's take a poll,'" Cain said. "He said, 'We will.' And we did. Only for this president to move us back by canceling a major part of our space program."

Cain also criticized Obama for using Russian technology to ferry astronauts and cargo to the International Space Station.

"I can tell you that as president of the United States, we are not going to bum a ride to outer space with Russia," Cain said to loud applause. "We're going to regain our rightful place in terms of technology, space technology."

I don't know what the reaction of the crowd was (the reporting wasn't that detailed) but I'd imagine it was friendly. You can usually get a warm response from a Republican crowd by coming out in favor of manned space exploration which is, when you think about, strange as hell.

If you set out to genetically engineer a program that libertarians ought to object to, you'd probably come up with something like the manned space program. A massive government initiative, tremendously expensive, with no real role for individual initiative. Compared to infrastructure projects the benefits to business are limited. You could even argue that the government's presence in the field crowds out private development.

(Much has been made of the rise of private space firms, but barring a really big and unexpected technological advance, their role is going to be limited to either unmanned missions or human flights in low earth orbit for the foreseeable future.)

There have been efforts in libertarian-leaning organs (The Wall Street Journal, Reason, John Tierney's NYT columns) trying to argue that interplanetary exploration can be done on the cheap. These usually rely heavily on the blatant low-balling of Robert Zubrin* (Tierney, a science writer who has no grasp of science, made a particularly ripe mark), but even if we were to accept these numbers, it's still difficult to reconcile this kind of government program with libertarian values.


* On a related note, check out this other example of Zubrin's estimation skills.

Tex Avery animates David Frum's sudden move to the center

At about ninety seconds in.

Pre-blogging Jack Shafer

I've got a couple of posts coming up on related subjects so I thought I'd get this out of the way in advance.

Here's Jack Shafer explaining how his intense antipathy toward plagiarism is all about the readers (via Salmon):

The plagiarist defrauds readers by leading them to believe that he has come by the facts of his story first-hand–that he vouches for the accuracy of the facts and interpretations under his byline. But this is not the case. Generally, the plagiarist doesn’t know whether the copy he’s lifted has gotten the story right because he hasn’t really investigated the topic. (If he had, he could write the story himself.) In such cases he must attribute the material he borrows so that at the very least the reader can hold somebody accountable for the facts in a story.

Putting aside the fact that Shafer has never gotten that worked up about colleagues' inaccuracy (you'll notice he didn't jump on this story about his friend Gregg Easterbrook), there's an interesting game here of rhetorical Three-card Monte. The cards in this game are the three types of plagiarism: theft of ideas and interpretations; theft of facts and data; and theft of wording.

In the world of research, the first two are considered the most serious. Stealing the hypothesis of another paper or presenting someone else's data as your own is about the worst thing you can do. Lifting a passage of someone else's writing is frowned upon but prose style does not drive impact factors.

For journalists, the situation is exactly reversed. Reusing another writer's phrases is clearly considered the worst kind of plagiarism, perhaps the worst journalistic crime period. Stealing facts (such as using other people's reporting to cover an event) is seldom even mentioned except in the most flagrant of cases. As for appropriating ideas, the practice is so common as to almost be standard. Even those most modern of journalistic concepts, memes, are almost always based largely on the plagiarism of hypotheses and arguments.

Now, here's the part where the cards really start to move. Shafer's criticism only applies to the first two types of plagiarism, the two types he doesn't object to. (aren't the first two nested in the third? Sometimes, but Shafer apparently doesn't have a problem with borrowing and paraphrasing, it must be the not-paraphrasing part that bothers him). If Shafer really wants to convince us that borrowing without paraphrasing is more than journalist-on-journalist crime, he'll have to do better than that.

Health Care Costs

We have known for a while that the United States spends a lot on health care. What is interesting is that the direction seems to be moving in the wrong way both in terms of life expectancy and cost:

You can see that not only is the United States the outlier when it comes to spending, but we are moving in the wrong direction: we are becoming more of a spending outlier, and we are drifting down from the average life expectancy into the lower group (currently surpassing only Turkey, Hungary, Mexico, Poland, and Czech Republic).

and

The other thing you see is that our life expectancy gain was the absolute lowest of the whole group (and we weren’t starting from a particularly high level, as you can see in the previous chart).

Ordinarily, you would think there should be convergence across countries. Since other countries spend less and live longer, you would think that we would learn from them—global competition, you know. But instead we’re moving the wrong way on both dimensions.

The article and neat charts are worth looking at in their entirety.

Now, it is true that there can be a lot of reasons for low life expectancy and high medical costs. It could be that the environment in the United States makes us much more accident prone, for example, requiring both higher spending and more fatalities.

But, in general, it is uncomfortable when the most important metric of health care outcomes (all cause mortality) is so uncorrelated with cost. This suggests the possibility of productivity improvements. I read a lot of the Incidental Economist, who try to explain these issues. But I admit that I tend to come away confused.

The major comparison is often Canada. It is a bad reference on a lot of levels (as they have their own issues). But they have similar culture, ethnic diversity, large geography, heavy use of cars, high levels of obesity and yet they are improving on both metrics (from a lower level of cost and higher life expectancy at baseline).

Why is health care the one area that we aren't willing to look at how other countries have been successful and try to steal ideas?

Old story, different hemisphere

More good work from American Public Media's Marketplace:

Rob Schmitz: If you walk down the street in Shanghai, it's hard to miss the advertisements for college placement agencies. In big bold letters, they promise -- some even guarantee -- your child's admission to an American university. The price: $5,000, $6,000, sometimes $7,000.

Jiang Xueqin is a well-known education reformer in China. He also heads the International division at Peking University High School -- one of China's best high schools. He has followed college placement agencies for years. He says this is how many of them deliver on their admission promises: they falsify students application materials.

Jiang Xueqin: There's a lot of pressure on the agencies to write the application essays, to fake transcripts, to fake recommendation letters. This is just general business practice in China to falsify a lot of documentation.

A report by consulting firm Zinch China seems to confirm this. Zinch advises American colleges and universities on recruiting Chinese students. The firm interviewed agents and admissions consultants, as well as more than 200 Beijing students headed to U.S. schools. Zinch estimates 90 percent of these students submitted false recommendation letters; 70 percent had other people write their personal essays, and half of them submitted forged high school transcripts.

Werner Herzog, LA, and structural employment in hot jazz

The director had an interesting interview on KCRW's none-too-imaginatively named show on the business of Hollywood. The part that particularly caught ear occurred about nineteen minutes in:

"We [Herzog and his wife] have to move into the city with the most substance in the United States, cultural substance."
"And that wasn't New York..."
"New York is more on the receiving end, it's more on the consuming end. ... Los Angeles, this is a place where things get done."

This jibes with something I've noticed since moving to LA and falling in (through no fault of my own) with the hot jazz scene in town. There are some extraordinary musicians around here, the kind who develop international followings and get glowing notices in the New York Times.

LA has a wealth of musical talent, but if those musicians want any wealth for themselves they have to head east. Because, though you might argue that the West Coast has the best musicians, the East Coast unquestionably has the best fans, at least when it comes to ponying up. In NYC these musician actually make good money; in LA you can often find them working for kind words and Pez.

If you're looking for creative energy, I'm with Herzog -- you can't do better than LA. If you're looking for a market for all that creativity, you might want to try the other coast.

But some of my best friends are administrators

As I've said before, most principals and superintendents I've dealt with have been dedicated and highly professional educators, so why do I keep picking on them? While it's possible that I do have a small problem with authority, my motives here are mostly pure. I'm trying to correct a cherished but dangerous piece of misinformation embraced by movement reformers.

For all their admirable traits, we have to acknowledge the world administrators live in: they are generally at least one step removed from the classroom and regular contact with students; their jobs are highly paid and highly political with badly aligned incentives and considerable temptations to abuse the system. Movement reformers like Jonathan Chait have convinced themselves that a major key (perhaps the key) to improving education is removing checks on administrators' power and upping rewards for manipulating the system. I'm just pointing out that some administrators need less power and fewer temptations.

I haven't taken a cheap shot at an administrator for a while

And superintendent Eric Ely certainly has one coming. He's certainly not the main villain in this amazing story from This American Life, but he clearly did his share to allow this psycho* to terrorize a number of people and endanger students, and while that character ended up in prison (probably for the rest of his life), Ely landed in another cushy superintendent's job.






* think this is hyperbole? Listen to the episode.