Bad public policy

I want to put together this post by John D Cook:

We can’t use common sense because it doesn’t fit on a form.

We can’t treat people like people because that doesn’t scale well.

We can’t use a simple approach to solve the problem in front of us unless the same approach would also work on a problem 100x larger that we may never have.

If the smart thing to do doesn’t scale, maybe we shouldn’t scale.

with this post by Radley Balko about a note sent home because a child was too heavy:

•Back when I wrote about the obesity debate for Cato, I remember when public schools telling parents their kids are too fat was the sort of thing people on my side of the debate warned about, and people on the other side of the debate said was ridiculous hyperbole. Also, the kid’s BMI is 19.4, and the school is sending home fat warnings? Why not just go ahead and build a vomitorium next to the girls bathroom?

Now, it is true that the person in question was in the 85th percentile for her group. But let us be clear here: if every single person in America had a healthy weight then somebody would still be in the 85th percentile. By definition.

We don't talk epidemiology enough on this blog (given the title) so this is a good tiem to talk about the need for careful developed public health policies. All that badly designed policies like this will do is to make it harder to implement effective and useful interventions.

To link this back to the quote at the beginning, blanket policies like this need to include nuance or else they are at risk of doing more harm than good. If we can't design a policy that can avoid these kinds of absurd outcomes then we should really be thinking about whether the area is a good one to implement policies in. But bad policies are worse than no policies . . .

This is exactly right

Candid Engineer has a very nice post about focusing on the things in your control. Luck is a major part of success but luck definitely favors the prepared. In the long run, it pays to focus on what you can control and let things onfold as they may.

New blog alert

David Warsh highly recommends this new blog by journalist Bill Bulkeley. I don't agree with his take on these companion pieces from the New York Times and the Wall Street Journal about Teach for America (time permitting I'll elaborate later), but he makes some good points and asks right question.

Definitely a site we'll want to keep an eye on.

The yahoos at Yahoo

A few days ago, the Yahoo homepage featured a Bankrate.com article that consisted of an amazing assortment of bad advice on homes as investments. I posted a rebuttal, but I could have simply waited for a few days and linked to this article from U.S. News which was also featured on the Yahoo homepage and which explains why everything the earlier piece said about home ownership was wrong.

I realize that Yahoo's homepage is not exactly a publication, but it does have tremendous reach and it would be nice if they hired someone to actually read what they post.

Divergent and Convergent Learning

As much as I complained about them at the time, the education classes I had to take to get certified did have some highly useful concepts. One of those was the distinction between convergent learning (where you want all students to reach the same final answer) and divergent learning (where you want each student to come up with a unique answer). Before you made a lesson plan or write a test, you were supposed to ask yourself where you want to see convergence and where you want to see divergence.

[There is an obvious connection between this divergent learning and the creativity discussion here, here, here and here]

There is an common but fatally naive misconception that convergent learning goes with math and science while divergent learning goes with arts and humanities. Almost all subjects start with a large convergent learning component including the arts (try picking up an instrument and see how much divergence your teacher tolerates in the first few lessons). More importantly, ALL subjects are fundamentally divergent at a high enough level. Writing a novel, composing a symphony, proving a conjecture or designing an aircraft are all creative exercises in constrained problem solving. We demand that certain conditions be met but we expect that each solution (or at least the method behind it) will be unique.

Which begs the disturbing question: will the proposed educational reforms produce more or fewer of the divergent thinkers we actually need?



p.s. Liam Hudson came up with a uses of objects test to measure divergent thinking. It ignores the complex interaction between possibility and constraint and is therefore, in my ever-humble opinion, complete crap, but take a look anyway.

Bias versus precision

In epidemiology, we are typically trying to estimate an unbiased measure of associations between an exposure and an outcome. Generally, we punt on the causal question of "does exposure X cause outcome Y?", but it is inevitably in the background. After all, if we say that poor exercise habits are associated with early mortality it is generally taken as an advisory to consider improving one's exercise habits rather than as an interesting coincidence.

But not all models are confounding models and the instincts that serve us so well for confounding models can be misleading for predictive models. Nate Silver has a very well explained example of how inaccurate (or, to be more formal, imprecise) predictive models can be worse than biased models. It's a very interesting confusion between bias and precision but it makes me wonder if we don't focus too much on unbiased and too little on efficiency for some of our models.

Thoughts?

More on creativity and slippery metrics

John D Cook has a interesting take on the whole creativity question. I'm not sure that I accept the premise that schools have as much influence on these factors as people think. But it does bring up an interesting point of the tail wagging the dog.

On of the classic criticism of scientific management is that they took a good idea (try and find measureable metrics of business success and failure) and turned it into a myopic focus on what is easy to measure in a spreadsheet. As a result, they focused on easy to measure metrics of success and, in the process, tended to neglect things that are hard to measure.

It's obviously true that creativity is a slippery and hard to measure concept. It is hard to, for example, design a standardized test around it. I wonder if part of the issues of education reform come from a focus on what can be measured and not what is important?

The best way to get rich is to sell books to gullible people

I sometimes wonder how much the current crisis was acerbated by bad advice from financial gurus on TV, advice that has changed amazingly little. Here's a recent example from David Bach (author of Start Late, Finish Rich and endless variations):

The best type of debt is debt that builds wealth over the long run, and the
No. 1 example of that is mortgage debt.

"Home values have increased an average of 6.5 percent a year over the past 30 years," says Bach. "So when you borrow to buy a home, chances are that's good debt. You'll build value."

Bach heavily promotes the idea of homeownership, saying that everyone needs to own where they live. "About 40 percent of Americans are renters," says Bach, "and the fastest way to wealth in America is buying where you live."

Bach cites some shocking numbers to back this up. "The average renter has a median net worth of $4,000, and the average homeowner has a median net worth of about $150,000."

...

One of the reasons so many Americans seem mired in bad debt (Bach reports that the average American carries approximately $8,400 in credit card debt) is that financial education is pratically (sic) nonexistent. "This type of commonsense stuff isn't taught in school," says Bach, "and most Americans don't realize how bad high-rate credit cards are hurting them."

Bach has been doling out this same advice complete with the same net worth statistic (apparently included under the assumption that schools didn't teach statistics either) for about a decade*. It is slightly more sound now than it was five years ago, but it is still not what you'd call good.

This ground has been covered before (notably by Felix Salmon) but just to recap:

1. Home ownership is not a particularly good investment either in terms of returns. It's bad in terms of liquidity and horrible in terms of diversification;

2. Analyses purporting to show the amount saved by owning vs. renting almost always overlook the direct and indirect costs associated with suburban commuting.

3. Home ownership makes it difficult for workers to go where the jobs are. This has economic consequences on the national level, but it's on the individual level that this can turn into a real horror show. Housing prices, liquidity and employment are all correlated by region.




*In Start Late, Finish Rich, Bach says "you cannot get rich being a renter."

Creativity

Felix Salmon has a post on creativity that got me thinking about comparisons. After all, in 1966 (when the test that he discusses was developed), it was a lot harder to get access to first rate creative materials. In that sense, "do it yourself" made a lot of sense and might have encouraged creativity.

You can see this as technology progresses. Consider live music and plays over time. In 1890, if you wanted to hear music or see a story acted out in your home town then you needed to get a local musician to play or go to a local community theater. In 2010 you downloaded the music to your iPod or a top film to your television set.

You see this phenomenon with lots of things. When my dad did weight lifting in the 1960's, the best you could do was a book on the topic. People competed to invent new techniques (and likely had a lot of injuries as a result). Today, I can find massive amounts of information on the topic sitting at home. In the 1970's, Gary Gygax and company could create a new type of game (role-playing games) as an extension of war gaming. Today the ability to create a new game is much harder given the development costs.

So is the lack of creativity a symptom of a richer environment where finding rhea optimal solution is better than creating a new (and likely sub-optimal one)?

I am not saying that this is the case and Felix Salmon brings up a lot of other good points. But it's another reason to wonder about what the implications of these claims would be, even if they were to prove to be correct.

Net impact

The ever interesting Professor in Training has a post on the issues of extraneous demands on a professor's time. I think that this is another example of things where each individual element is likely to be justified but the whole is not.

Examples of these sorts of things include regulations, old laws (regarding horse thievery, for example) assistant professor committee work, software patches and so forth. For example, the last university I was at required a one hour course on the hazards of asbestos every year. While I am sure that asbestos is deadly substance, it seems like overkill to have a class on such a narrow topic every year.

What is hard to determine is how to identify these situations ahead of time and determine the appropriate responses. It's not easy -- as a junior professor I have said no to a few requests already that were pretty reasonable (and sounded like they'd add value) just to make sure that I had enough time.

I would like to think of a way to handle this issue that did not involve "over-all impact committees".

Another reason not to aggregate elementary and post-elementary school data

This is from a National Center for Education Statistics summary of TIMSS data from the Nineties* when the reform movement was pretty much settling into its present form. They were already sounding the alarm about math and science scores and proposing extensive reforms for grades K through 12, despite the fact that the performance issues were almost entirely limited to junior high and high school.

In mathematics, fourth-grade students in 7 countries outperform our fourth graders (Singapore, Korea, Japan, Hong Kong, the Netherlands, the Czech Republic, and Austria). Students in 6 countries are not significantly different from ours (Slovenia, Ireland, Hungary, Australia, Canada, and Israel). U.S. fourth graders outperform their counterparts in 12 nations (Latvia, Scotland, England, Cyprus, Norway, New Zealand, Greece, Thailand, Portugal, Iceland, Islamic Republic of Iran, and Kuwait).

In science, students in only one country--Korea--outperform U.S. fourth graders. Students in 5 countries are not significantly different than ours (Japan, Austria, Australia, the Netherlands, and the Czech Republic), and U.S. fourth graders outperform their counterparts in 19 nations (England, Canada, Singapore, Slovenia, Ireland, Scotland, Hong Kong, Hungary, New Zealand, Norway, Latvia, Israel, Iceland, Greece, Portugal, Cyprus, Thailand, Islamic Republic of Iran, and Kuwait).

*I'll try to dig up some more recent results just to see how things are trending.

If all students were 15 years old...

Joseph has an interesting link to a comparison of test scores of fifteen-year-olds in different countries. I recommend you go by and take a look. It has some interesting stuff, but what caught my eye was what wasn't there. I couldn't find a complementary break-down for any other age group.

After bad metrics, I think the worst problem in educational research may be the inappropriate aggregation of primary and secondary education. If there was ever a case where two populations should be treated separately it's here. The two systems face different problems of different severity that respond to different solutions.

I'll try to follow up with some specifics later, but in the meantime, when you hear someone making a blanket proclamation about our schools, remember that about the only meaningful statement you can make that's true about primary and secondary schools is that the teachers in both are about to get screwed.

Statistical programming languages

This post by John D Cook got me thinking about whether it is possible to do a similar simplification of programming languages for Epidemiologists doing analysis.

These days I see the following languages in heavy use: STATA, R, SPSS, SAS and some S-plus. Furthermore, one is requires to do data management in some combination of SQL/Oracle, SAS, Excel and Access. That doesn't even touch on the people who still use C++ and/or FORTRAN for specialized programming applications.

My question is which ones does it make sense to support? In my department, I think we'll do a combination of R (freeware, flexible, powerful) and SAS (FDA standard) as languages that we officially support. But not supporting STATA is a very painful choice! What have others done in similar circumstances?

Speaking for the unhinged

Jonathan Chait dismisses critics of proposed education reform as 'unhinged.' Speaking as one of the whackjobs, here's the sort of thing that makes us loonies so nervous.

From a press release (6/8/09) from the Department of Education:

Emphasizing the need for additional effective education entrepreneurs to join the work of reforming America's lowest performing public schools, U.S. Secretary of Education Arne Duncan told reporters during a conference call this afternoon that states must be open to charter schools. Too much is at stake for states financially and for students academically to restrict choice and innovation.

"States that do not have public charter laws or put artificial caps on the growth of charter schools will jeopardize their applications under the Race to the Top Fund," Secretary Duncan said. "To be clear, this administration is not looking to open unregulated and unaccountable schools. We want real autonomy for charters combined with a rigorous authorization process and high performance standards."

From a May 1st story in the New York Times:

But for all their support and cultural cachet, the majority of the 5,000 or so charter schools nationwide appear to be no better, and in many cases worse, than local public schools when measured by achievement on standardized tests, according to experts citing years of research. Last year one of the most comprehensive studies, by researchers from Stanford University, found that fewer than one-fifth of charter schools nationally offered a better education than comparable local schools, almost half offered an equivalent education and more than a third, 37 percent, were “significantly worse.”

Us nutjobs would like to see the administration reconsider its position based on the data.

"Fewer Low-Income Students Going to College"

From the Wall Street Journal blog Real Time Economics (via Thoma):

Fewer low- and moderate-income high school graduates are attending college in America, and fewer are graduating.

Enrollment in four-year colleges was 40% in 2004 for low-income students, down from 54% in 1992, and 53% in 2004 for moderate-income students, down from 59% over the same period, according to a report recently submitted to Congress by the Advisory Committee on Student Financial Assistance.

If that trend has continued, low- and moderate-income students who don’t move on to college face an even darker outlook. The unemployment rate for 16- to 19-year olds averaged 17% in 2004, the jobless rate for people over age 25 with just a high school diploma averaged 5% the same year. So far this year, those figures have jumped to 25.8% and 10.6%, respectively.

College expenses and financial aid have become increasingly larger considerations for parents and students, driving more qualified students away from enrolling in four-year colleges.