Health Care

We haven't talked as much about Health Care lately, because I have been slammed by my first year of teaching. But I did want to highlight this post by Aaron Carroll:

You ready for this? They didn’t ship the medication. Why? It’s on back-order.

So explain this to me, awesome insurance system. You will only allow me to get my medication from one pharmacy, which is not local, and which you own and run. And you won’t send the medication early, which would provide a safe buffer. And then you run out of the medication. And then you don’t tell me until the day I run out. And then you don’t really tell me, you leave me a cryptic message on my voice mail that I would usually not be able to get until I arrive home and you are closed.

Will you allow me to go to another pharmacy? Sure. But you won’t cover it. Will you have the medication soon? Yeah, you hope to have it tomorrow.

I don’t believe you.

Do you have any concern – whatsoever – that I am without my medication for a period of time? That it’s entirely your fault?

I’m a model patient. I pay all my bills. I go to the doctor. I get the labs done. I refill the meds on time with a weeks’ advance. I follow the rules. And you screw me.

Are we really sure that the health care system in the United States meets the requirements for market efficiency*? Really?


* And, if you have not read the classic post by Mark Thoma then now is as good a time as any. Well worth the ten minutes to breeze through it and it is a very good way of evaluating whether a particular market is likely to be efficient.

More thoughts on Charter Schools

Mark quotes Diane Ravitch about the new film Waiting for Superman.

I think that these paragraphs really deserve some more discussion:

Geoffrey Canada is justly celebrated for the creation of the Harlem Children’s Zone, which not only runs two charter schools but surrounds children and their families with a broad array of social and medical services. Canada has a board of wealthy philanthropists and a very successful fund-raising apparatus. With assets of more than $200 million, his organization has no shortage of funds. Canada himself is currently paid $400,000 annually. For Guggenheim to praise Canada while also claiming that public schools don’t need any more money is bizarre. Canada’s charter schools get better results than nearby public schools serving impoverished students. If all inner-city schools had the same resources as his, they might get the same good results.

But contrary to the myth that Guggenheim propounds about “amazing results,” even Geoffrey Canada’s schools have many students who are not proficient. On the 2010 state tests, 60 percent of the fourth-grade students in one of his charter schools were not proficient in reading, nor were 50 percent in the other. It should be noted—and Guggenheim didn’t note it—that Canada kicked out his entire first class of middle school students when they didn’t get good enough test scores to satisfy his board of trustees

I note that the Harlem's Children's Zone seems to get only positive press in the blogosphere. Yet two facts here really seem to stand out. One, making the compensation structure that of a corporation is going to mean that we get advertising and not honest accounting of performance. A very high salary ($400,000 per year) and a corporation that lives or dies on profit does not incent actors to give a balanced view of the shortcomings of the corporation.

The second, and more damning, issue is that kicking out an entire class of students when their tests scores are low is guaranteed to ensure that Charter schools look better -- after all, the weak students have been ruthlessly removed. Anybody can look competent under this system.

This cuts to the heart of my discomfort with the charter school system. One of the most important issues facing American education is the drop-out of vulnerable students. Charter schools that are willing to remove poorly preforming students for reasons of "optics" cannot, in aggregate, improve this situation.

[Mark has already extensively documented other concerns with interpreting charter school performance]

Nervous Nate -- why timidity at FiveThirtyEight is a sign of professionalism

I've been working on an article on educational statistics. One of the major themes is the way statisticians should handle research when they don't have much faith in the results. This is not quite the same as the way Bayesians assign weights to beliefs but it's not all that different. Us poor, simple frequentist-folk approach the problem in an informal way and try to present our results with sufficient caveats that the listeners will understand that we may have a small p-value but that doesn't mean we'd bet the farm.

As I was working on that part of the article, I was struck by how Nate Silver was doing exactly what I was describing, pointing out the factors that could throw his forecasts off, reminding people that in this election, more than most, anything could happen.

That's one of the reason's Silver is the gold standard* in the field.





* Sorry, I just couldn't help myself.

An excellent Maxine Udall post

Really good stuff (via guess who):

In 1940, Howard Florey and Ernst Chain, at Oxford University, produced enough stable chemical from a mold, penicillium, to test on mice infected with streptococcus. Of eight infected, 4 were treated and recovered rapidly. "Eureka!," you say. "The rest is history, yes?"

Well, no. Not quite.

Florey and a colleague packed up their penicillin and their results and took them to major pharmaceutical companies in the UK, the US, and Canada. None wanted to invest the capital in scaling it for production, mainly because there were signs that bacteria were becoming resistant to the sulfa drugs they were already manufacturing. Even if penicillin were effective for a while, it would eventually become ineffective and demand would dwindle. Where's the profit in that, they rightly asked?

So how did we the people get penicillin (and the many subsequent antibiotics that repeatedly haul us back from the brink of death, bronchitis, and septicemia)? The US government stepped in. The US military pushed a bit on this. After all, who better to sense on the eve of war the potential gains from a new antibiotic? Roughly half of all deaths in at least one previous war had been from infection.

It was a group of government scientists at the US Department of Agriculture who scaled production and increased the drug's efficacy four-fold. And in record time. Imagine. Guvmint scientists produced something quickly, efficiently, and made it (and us) better.

I know what you're thinking. You're thinking that surely now, after the US government and the US taxpayers had volunteered the start up costs, the drugs manufacturers must have agreed to produce it, yes? By the time it had been scaled, it was more apparent than ever that the US might be drawn into a major world war. The military wanted it. Surely an appeal to patriotic duty would move them?

No. I'm afraid not. Not until George Merck, CEO of Merck & Company, agreed to do it and persuaded the heads of several other manufacturers to join him. "Medicine is for people, not for profits," George said.

"How much math do we really need?"

I disagree with a fair chunk of this, but it's worth reading:

Twenty-seven years have passed since the publication of the report "A Nation at Risk," which warned of dire consequences if we did not reform our educational system. This report, not unlike the Sputnik scare of the 1950s, offered tremendous opportunities to universities and colleges to create and sell mathematics education programs.

Unfortunately, the marketing of math has become similar to the marketing of creams to whiten teeth, gels to grow hair and regimens to build a beautiful body.

There are three steps to this kind of aggressive marketing. The first is to convince people that white teeth, a full head of hair and a sculpted physique are essential to a good life. The second is to embarrass those who do not possess them. The third is to make people think that, since a good life is their right, they must buy these products.

So it is with math education. A lot of effort and money has been spent to make mathematics seem essential to everybody's daily life. There are even calculus textbooks showing how to calculate -- I am not making this up and in fact I taught from such a book -- the rate at which the fluid level in a martini glass will go down, assuming, of course, that one sips differentiably. Elementary math books have to be stuffed with such contrived applications; otherwise they won't be published.

You can see attempts at embarrassing the public in popular books written by mathematicians bemoaning the innumeracy of common folk and how it is supposed to be costing billions; books about how mathematicians have a more clever way of reading the newspaper than the masses; and studies purportedly showing how much dumber our kids are than those in Europe and Asia.

As for the third, even people who used to proudly proclaim their mathematical innocence do not wish to abridge the rights of their children to a good life. They now participate in family math and send the kids to math camps, convinced that the path to good citizenship is through math.

We need to ask two questions. First, how effective are these educational creams and gels? With generous government grants over the past 25 years, countless courses and conferences have been invented and books written on how to teach teachers to teach. But where is the evidence that these efforts have helped students? A 2008 review by the Education Department found that the nation is at "greater risk now" than it was in 1983, and the National Assessment of Educational Progress math scores for 17-year-olds have remained stagnant since the 1980s.

The second question is more fundamental: How much math do you really need in everyday life? Ask yourself that -- and also the next 10 people you meet, say, your plumber, your lawyer, your grocer, your mechanic, your physician or even a math teacher.

Unlike literature, history, politics and music, math has little relevance to everyday life. That courses such as "Quantitative Reasoning" improve critical thinking is an unsubstantiated myth. All the mathematics one needs in real life can be learned in early years without much fuss. Most adults have no contact with math at work, nor do they curl up with an algebra book for relaxation.

Diane Ravitch vs. "Superman"

There are few places where the divide in the educational debate between popular perception reality is as depressing as the reaction to Waiting for Superman. Here Diane Ravitch (whose resume includes the first Bush administration and the Hoover Institute) lists some of the distortions, omissions and factual errors.

More on this later.

"The Freakonomics Fallacy"

From an excellent post by Barbara Kiviat (pay close attention to the last sentence):

There’s been some interesting discussion in response to my earlier post about why we expect too much from economists, although a lot of the comments miss my larger point. What I was trying to say is that economics might not entirely be up to the task of explaining what we generally consider to be economic phenomena because we are overconfident about what the discipline has the ability to account for. You might call this the Freakonomics Fallacy. Whatever it is in the world we are trying to explain—crime, climate change, test scores—economics has the answer.

If this isn’t in fact true, then why would we think it? Again, to underscore a part of my earlier argument that I probably didn’t make forcefully enough: we think economics has all the answers because economics has become our major mode of understanding the social world around us. Charities are social businesses. Policy makers are cost-benefit analyzers. Education is a market.

This has not always been the case. In earlier eras, we were often more likely to understand human behavior and social dynamics through other prisms, such as political science, sociology, psychology, anthropology or biology. Indeed, for better or for worse, many of these fields took a turn being the dominant social science. I’m not saying that one frame gives a more accurate or useful picture of the world than another. Just that each leads to a different way of understanding why things happen the way they do because each comes with its own set of assumptions and simplifications.

You can see my thoughts on the subject here.

Children and Placebos

From:

Greater Response to Placebo in Children Than in Adults: A Systematic Review and Meta-Analysis in Drug-Resistant Partial Epilepsy

(published in PLoS Medicine)

The researchers searched the literature for reports of RCTs on the effects of antiepileptic drugs in the add-on treatment of drug-resistant partial epilepsy in children and in adults—that is, trials that compared the effects of giving an additional antiepileptic drug with those of giving a placebo by asking what fraction of patients given each treatment had a 50% reduction in seizure frequency during the treatment period compared to a baseline period (the “50% responder rate”). This “systematic review” yielded 32 RCTs, including five pediatric RCTs. The researchers then compared the treatment effect (the ratio of the 50% responder rate in the treatment arm to the placebo arm) in the two age groups using a statistical approach called “meta-analysis” to pool the results of these studies. The treatment effect, they report, was significantly lower in children than in adults. Further analysis indicated that this difference was because more children than adults responded to the placebo. Nearly 1 in 5 children had a 50% reduction in seizure rate when given a placebo compared to only 1 in 10 adults. About a third of both children and adults had a 50% reduction in seizure rate when given antiepileptic drugs.

...

These findings, although limited by the small number of pediatric trials done so far, suggest that children with drug-resistant partial epilepsy respond more strongly in RCTs to placebo than adults. Although additional studies need to be done to find an explanation for this observation and to discover whether anything similar occurs in other conditions, this difference between children and adults should be taken into account in the design of future pediatric trials on the effects of antiepileptic drugs, and possibly drugs for other conditions. Specifically, to reduce the risk of false-negative results, this finding suggests that it might be necessary to increase the size of future pediatric trials to ensure that the trials have enough power to discover effects of the drugs tested, if they exist.

Of course, with open-label experiments, this should lead to false positives.

And a final word on the subject from America's most successful philosophy major

But I quit that! I've quit ALL drugs. Well... let me say one thing: I twisted my ankle this morning, and I was in quite a bit of pain... so I went to the doctor, and I asked him to give me some pain pills. And he didn't want to do it, but I talked him into it. So he gave me some pills -- and I shouldn't have done this, but I took some about an hour before the show tonight, and right now... I am high... as a KITE! I mean, it is unbelievable! And I would NEVER say this to you people, but, in this case: if you EVER get a chance, to take these drugs... DO IT! They're called... [ he glances from side-to-side cautiously ] Placebos! I mean, I'm thinking that right now I have NO idea where I am at all! It is WILD! [in a hushed voice] PLUH-SEE-BO!

Steve Martin on SNL

Placebos

This is relevant to Mark's posts on how comparisons between students who win and lose charter school lotteries are similar to open label drug trials. Andrew Gelman quotes Kaiser Fung stating that (in drug trials) the standard comparison is:

Effect on treatment group = Effect of the drug + effect of belief in being treated
Effect on placebo group = Effect of belief in being treated

I think that the mathematical formulation of Mark's concern about using charter school lotteries as follows:

Effect in charter school group = Effect of the charter school intervention + effect of belief in being treated + baseline

Effect on students in standard schools = baseline - (any discouragement effect from losing the lottery)

So if the effect in the charter school students is greater than that in students who lose the lottery and are placed in standard schools, that skill doesn't separate the two sources of variation (placebo versus innate effect of the intervention).

The real test would be to introduce some sort of placebo intervention (a charter school that used standard educational approaches??). But that is not an easy thing to accomplish, for obvious reasons. I suspect that this is why psychologists end up having to use deception as part of their studies, despite the ethical quagmires this produces.

UPDATE: missing link fixed

Worth a quick look

Very worthwhile read on the nuances of economics. In particular:

"You see, in a capitalist economy, wealth and well-being are supposed to redistribute to everyone. As capital is allocated and risk managed more efficiently, more opportunities are created for all. Wealth and well-being are supposed to become less concentrated in the hands of a few and more dispersed to the hands of the (often more and increasingly) productive many. Much of Wealth of Nations is devoted to describing the instances and the conditions under which this seemed to be occurring as the economy of Great Britain transitioned from feudalism to one of commercial exchange, industrial production, and small business owners."

H/t: Mark Thoma

Diffferences in Professional Standards

I find it fascinating how the rules change between disciplines. In Epidemiology, the idea of a supervisor being an author on papers resulting from a doctoral discipline is completely non-controversial (and standard practice).

I think it is the difference in resources. The question is asked:

If, as an undergraduate, you handed in a paper that had partially been written by someone else, you would be sent down to the Dean's office, read the riot act with regard to academic integrity, and (probably) given an automatic F in the course.

Why doesn't the same thing happen with respect to a PhD thesis?

In Epidemiology the answer is that there would be no PhD theses under this rubric. Data takes too long to collect and analyze. Furthermore, granting agencies are simply unwilling to give massive amounts of grant funding to a PhD student. All of our research is deeply collaborative and single authored work can only happen in areas quite abstracted from reality or for very senior people who have access to data in a way that others do not.

A war with Iran would look nothing like WW II

When someone says something outrageous and stupid, there's a tendency (particularly on the blogosphere) to lock in on the outrageous part and give the stupidity a pass. Today's case in point comes from David Broder (via a suitably appalled Mark Thoma):

What else might affect the economy? The answer is obvious, but its implications are frightening. War and peace influence the economy.

Look back at FDR and the Great Depression. What finally resolved that economic crisis? World War II.

Here is where Obama is likely to prevail. With strong Republican support in Congress for challenging Iran's ambition to become a nuclear power, he can spend much of 2011 and 2012 orchestrating a showdown with the mullahs. This will help him politically because the opposition party will be urging him on. And as tensions rise and we accelerate preparations for war, the economy will improve.

I am not suggesting, of course, that the president incite a war to get reelected. But the nation will rally around Obama because Iran is the greatest threat to the world in the young century. If he can confront this threat and contain Iran's nuclear ambitions, he will have made the world safer and may be regarded as one of the most successful presidents in history.

Broder is, of course, suggesting just that, but while everyone is reacting to the indecency or, like Dean Baker, pointing out that infrastructure spending has the same effect, no one seems to be focusing on the stupidity of the analogy. So here I go in handy list form:

1. We don't have a draft. This war would mainly be fought with the forces at hand with minimal impact on unemployment;

2. WW II required a massive build-up in our air force and navy. How many more carriers are we going to need if we go war with Iran? In other words, this war would do little to absorb excess capacity;

3. We had a tightly controlled economy during the war that created a build-up in consumer demand. How likely is it that a GOP Congress would go along with that?

4. Immediately after the war, we extended a huge and unprecedented social safety net for the returning forces. Any chance of that happening?

If you're going to suggest fighting a war for economic gain, at least try to pick one that would actually do. Otherwise, you're being evil and stupid.

And now for something completely different...

I'll be taking a break from p-values and peer effects tomorrow to watch the latest instalment of Sherlock now airing on Masterpiece Mystery. Mark Gatiss and Steven Moffat provide the scripts. That alone should be enough to get you to tune in.

Now if we can just get them to air Jekyll.

The Volunteer Effect

Before I jump back into the weeds here, let's step away for a paragraph and remind ourselves that the question of the moment is "Do charter schools provide a good, workable model for the overall education system?" This is a high bar, but it's one that the reformers themselves set. There are things that many charter schools do which tend to improve student performance and attitude, but which would lose their effectiveness if applied to the general population.

Which brings us to the volunteer effect.

In Influence, Robert Cialdini observes that "we accept inner responsibility for a behavior when we think we have chosen to perform it in the absence of strong outside pressures." (I couldn't find my copy but it's on page 97 of the library's.)* It's part of the chapter on commitment which is filled facts and examples that are relevant to the current education debate.

Indoctrination plays a large part in the culture of schools like KIPP and HCZ. This isn't a bad practice -- all schools do it to some extent -- but it is particularly important when a school tries to greatly increase student work load. If you are assigning hours of homework every night and study sessions every Saturday and you expect reasonable compliance, you are going to have to make the students believe that hard work is the right thing to do and that it will lead to large rewards.

That kind of belief modification is easiest when subjects see themselves as volunteers who not only chose to engage in the accompanying behaviors but who actually made an effort to do so. This works well in the current incarnation of charter schools, but everyone can't be a volunteer. When educators show the ability to inspire these beliefs in the kids who don't want to be there, then we'll have something.







* As part of this discussion, Cialdini also mentions that small rewards tend to create more long-lasting behavior changes than large rewards. This sets up an interesting topic for a future blog post on economics and compensation.