Boom, bust and echo

There is a really nice chart in Worthwhile Canadian Initiative (WCI) about youth unemployment over time (in Canada). I was too young to actually be influenced much by the recession in the 1980's but the recession of the early 1990's (about 6% higher than it is today) dramatically influenced my career trajectory. As a young person, I never imagined that I would end up in the United States.

However, poverty has a way of changing opinions and I headed south for employment reasons. I am struck by how different the tenor of the times was: the articles that were linked to in the WCI post suggest that the issue is greed among those in the older generations. But, back in the day, we were much more likely to hear that young people were unmotivated or spoiled.

I think that we are actually seeing signs of the economic power of young people today that the narrative has shifted so far from what it used to be. But, as a card carrying member of Generation X, I can definitely attest that careers were pushed back and we started a lot later than everyone else simply because jobs were so hard to find.

"What is this 'Canada' of which you speak?"

Following up on Joseph's last post, I remember a discussion about careers I had with a group of friends including Joseph a few years ago. I was looking to make a change and Joseph asked if I'd considered the Canadian term for substitute teaching. I looked at him as though he had suggested I apply for a job scraping roadkill. It took several minutes for him to convince me that where he came from, substitute teaching was actually a sought-after career.

This is consistent with Canada's approach toward teaching in general. Canadians have long worked under the assumption that, if you give teachers security, respect and good salaries, you will attract good teachers. This is just one of the ways that Canada has done the opposite of what education reformers have recommended in this country. Here advocates like Joel Klein and Michelle Rhee insist that without charter schools and the option of mass firings the education system is doomed and yet, by the reformers' own favorite metrics, our northern neighbor consistently kicks our ass.

Demographically, economically, culturally and historically, Canada would seem to be the obvious country to look to when trying to determine the effectiveness of potential U.S. policies but it has been conspicuously absent from a number of debates. Before we start looking across half the world to countries with radically different situations and backgrounds, isn't it possible that we can learn something from a spot closer to home?

Canadian Exceptionalism

Megan McArdle has a comment on whether a social safety net will spur entrepreneurship:

But when we try to look at the net effect, it's hard to see much dynamism coming out of the places with a generous safety net. Rates of entrepreneurship and labor mobility are far lower in Europe than they are in America--and one of the many factors restraining European labor mobility is thought to be the social safety net, both because of the difficulty of moving between benefit system, and because the benefits lessen the urgency of say, relocating in order to find a job.

I will now speak at the level of personal anecdote; I beg the reader’s indulgence but I think it is a worthy point to make. I have lived in both the United States and Canada (as any reader of this blog will probably have figured out). At a purely personal level I am way more comfortable with economic risk when I am resident in Canada.

Why? Because the cost of a hospitalization is so high here in the United States as compared with elsewhere. There are two factors that I think are key. One is simply that medical care is intrinsically expensive here in the United States. But the second is the bargaining power of groups. Look at what the difference is between the billed price and what private insurance pays.

Now imagine that you feel dizzy one day. Dizziness can be a sign of a stroke or a heart attack. But if you are unemployed and not covered then what do you do? Visiting the hospital is a guarantee of some sort of economic crisis. But many events can be stopped with early treatment (or at least highly mitigated).

If we had the features of an efficient market, that would be different. But has anybody ever tried to shop around for better prices in the midst of a serious health event? Yet the United States links health insurance with employment. Furthermore, rescission tends to happen at the time of the event; in cases of gross fraud that is perfectly understandable but the examples of very minor book-keeping errors leading to rescission make one leery of not having somebody (like a large employer's HR department) around to advocate for you. Courts, while possible, are expensive at a time when you likely have no money.

Now this is not to say anything about elective procedures or non-urgent medicine; things like elective eye surgery can be areas that markets can exist. But we can already see this in Canada with things like diagnostic scans (that are not time sensitive) are increasingly handled by private firms in Canada (often paid out of pocket) for a surprisingly low cost to the consumer. The same with laser eye surgery or some forms of bariatric surgery. So a mixed economy is possible.

So I have been a lot more concerned about being employed in stable position here in the United States. In Canada I have considered (and applied for) short term and unstable positions: many of which were a lot more entrepreneurial.

After all, correlating employment risk (i.e. loss of job) with medical care cost risk (when insurance is linked to your job) seems like a bad decision. As for why we don’t see this in Europe – I have no idea. But it’s not clear that the European example is conclusive.

A Christmas Reminder

A lot of people out there have had a rough year so if you're in a position to be generous this is a good time for generosity.

From All Things Considered:

SIEGEL: Joining us from New York City's main post office is head elf, Pete Fontana. He's been working with the U.S. Postal Service Operation Santa for 15 years. Welcome to the program.

Mr. PETE FONTANA (Operation Santa Claus, U.S. Postal Service): Well, thank you for having me.

SIEGEL: And what have you noticed that's different in tone this year about the letters to Santa Claus?

Mr. FONTANA: Well, normally the letters would be what you said, you know, more like greedy-type things - big televisions, Xboxs, Wiis, things of that nature, MacBooks. This year, the letters are single moms, three kids, no winter coats, no shoes, blankets, can't pay the bills, not enough food in the pantry. So the need has changed tremendously.

SIEGEL: Perhaps you can read a couple of those letters to us, Pete.

Mr. FONTANA: Hold on. Here's one from Christopher(ph). Dear Santa, My name is Christopher. I'm 11 years old and I have a sister. Her name is Bethania(ph). She is two years old. And I have a brother who is nine months. If possible, we would like some educational toys and some winter clothes. I would like something to make my mommy happy 'cause she is getting chemotherapy after breast surgery. Something like a hat or a scarf for her. Thank you very much, Christopher.

SIEGEL: Now, your program at the post office is designed to pair children in need with volunteers who might donate gifts. I would imagine that you always see letters like this every year. Is this year really different?

Mr. FONTANA: I would say that this year there just seems to be more needy and less greedy I hate to rhyme it but it just seems that thats what the trend is.

The Homebase study looks OK but the New York Times is still a mess

Alex Tabarrok and Joseph both have posts up on a study to determine the effectiveness of a program to prevent homelessness by randomly accepting 200 of the 400 applicants the program received last summer then comparing how the accepted fare compared to the rejected. This isn't exactly how I'd set the study up but the choices the researchers seem both reasonable and ethical. As one of the researchers pointed out, this was not an entitlement; it is a small program that rejects some applicants already. All the researchers are doing is rearranging the pool.

If everything is as it seems, my only major concern is that, given our current exceptionally bad economic conditions, the results from this study might not generalize well.

But the word 'seems' is important because the NYT story that all of these posts have been based on simply isn't informative enough or well enough written for the reader to manage an informed opinion.

The story starts out ominously with this paragraph:

It has long been the standard practice in medical testing: Give drug treatment to one group while another, the control group, goes without.

Now, New York City is applying the same methodology to assess one of its programs to prevent homelessness. Half of the test subjects — people who are behind on rent and in danger of being evicted — are being denied assistance from the program for two years, with researchers tracking them to see if they end up homeless.

It might not be reasonable to expect a dissertation on the distinction between double-blind and open-label studies, but given that the subject of the article is the effectiveness and ethics of certain kinds of open-label studies, the fact that the writer may not know that there is a distinction does not bode well.

It does, however, bode accurately because the writer apparently proceeds to blur a much more important distinction, that between pilot and ongoing programs:

Such trials, while not new, are becoming especially popular in developing countries. In India, for example, researchers using a controlled trial found that installing cameras in classrooms reduced teacher absenteeism at rural schools. Children given deworming treatment in Kenya ended up having better attendance at school and growing taller.

These are pilot programs. The Indian government didn't install cameras in all their rural schools then go to the expense of randomly removing half of them, nor did the Kenyans suddenly discontinue preventative care from half their children. From a practical, ethical and analytic perspective, going from no one gets a treatment to a randomly selected sample get a treatment is radically different than going from everyone gets a treatment to a randomly selected sample get a treatment.

Putting aside the obvious practical and ethical points, the analytic approach to an ongoing program is different because you start with a great deal of data. You don't have to be a Bayesian to believe that data from other sources should affect the decisions a statistician makes, choices ranging from prioritizing to deciding what to study to designing experiments. Statisticians never work in a vacuum.

There is little doubt that the best data on this program that we could reasonably hope for would come from some kind of open-label study with random assignment but, given the inevitable concerns and caveats that go with open-label studies, exactly how much better would that data be? What kind of data came out the original pilot study? What kind of data do we have on similar programs across the country? And most importantly, what's the magnitude of the effect we seem to be seeing?

On that topic we get the following piece of he said/she said:

Advocates for the homeless said they were puzzled about why the trial was necessary, since the city proclaimed the Homebase program as “highly successful” in the September 2010 Mayor’s Management Report, saying that over 90 percent of families that received help from Homebase did not end up in homeless shelters.

...

But Seth Diamond, commissioner of the Homeless Services Department, said that just because 90 percent of the families helped by Homebase stayed out of shelters did not mean it was Homebase that kept families in their homes. People who sought out Homebase might be resourceful to begin with, he said, and adept at patching together various means of housing help.

Before we can ask if this proposed selection effect can explain the impact of Homebase, it would help if we had the slightest idea what that impact was. It's true that over 90 percent of the families in question did not end up in shelters. It is also true that 99.99 percent of the people who took homeopathic remedies for their colds did recover. Neither number is particularly useful without the proper context.

The proper context shouldn't be that difficult to find. We have more than enough information to estimate the likelihood of families in these financial situations ending up in shelters. Of course, there is no upper bound on a possible selection effect, but I'm going to weight the possibility differently if the comparison rate turns out to be 80 percent than I would if it were 40 percent.

None of this is a criticism of the actual research. Other than my concern about generalizing 2010 economic data, this looks like a good study. Governments and large organizations should always be on the lookout for ways that experimental design and sampling theory can improve our data.

But I can't say with any certainty that this is a good place for this kind data gathering because I'm getting my information from a badly written story. There are three papers I read frequently: the LA Times, the New York Times and the Wall Street Journal, and of those, the one that is most likely to run lazy, low-context, he said/she said pieces is the old gray lady.

Merit?

From the comments section of Worthwhile Canadian Initiative:

Actually one of my favourite questions on the 2008 US Jump$tart survey is this one:

Don and Bill work together in the finance department of the same company and earn the same pay. Bill spends his free time taking work-related classes to improve his computer skills; while Don spends his free time socializing with friends and working out at a fitness center. After five years, what is likely to be true?

a) Don will make more because he is more social.
b) Don will make more because Bill is likely to be laid off.
c) Bill will make more money because he is more valuable to his company.*
d) Don and Bill will continue to make the same money.

Not at all obvious to me that (c) is the right answer.

[Note: the * next to (c) indicates that it is the correct answer for the key.] I can certainly think of examples where this answer would seem to be contradicted by empirical reality. And that is before one worried about outsourcing or what happens if the company completely changes software platforms in a major re-org.

The post and comments are worth reading, throughout.

Randomized Social Experiments

In a really fascinating post, Alex Tabarrok discusses controversy about a random clinical trial set within a housing program. It's a very interesting example (and his quote from E. E. Peacock, Jr. was priceless). But it does bring up a serious question: in many areas of social policy there is no realistic way to blind experiments and, similarly, the idea of informed consent is hard to implement in practice.

Consider this point as well: the only way you will be able to get rational actors to consent to an experiment on housing is if all of the options in the intervention are equal or better than the status quo. That means you can never experiment to see if the current plans are working, insofar as informed consent is required. Nor can one really argue that coercion is not present given that people enter these programs out of desperation.

I think the real difference is that we have two different sources of utility: the user of the program and the agency (via the taxpayer) who is implementing the program. So it becomes a complex problem because the status quo might benefit the user (marginally) and harm the taxpayer/state/agency (greatly) but you will never get users to consent to testing how they will do with fewer benefits (or at least the incentives are wrong). Compare this with drug trials: only the patient’s ability to benefit is typically considered (although cost effectiveness may matter later in the approval process).

But the alternative is simply to not know what the right answer is and to risk getting stuck at a very suboptimal policy point. And that seems to be the wrong answer as well.

"I am not now and have never been a constructionist"

(This post also appears at Education and Statistics.)

After my last post thought I should run this titular disclaimer. For those of you not up on the subject, here's a definition from the well-written Wikipedia entry on the subject:

Constructivist teaching is based on constructivist learning theory. This theoretical framework holds that learning always builds upon knowledge that a student already knows; this prior knowledge is called a schema. Because all learning is filtered through pre-existing schemata, constructivists suggest that learning is more effective when a student is actively engaged in the learning process rather than attempting to receive knowledge passively. A wide variety of methods claim to be based on constructivist learning theory. Most of these methods rely on some form of guided discovery where the teacher avoids most direct instruction and attempts to lead the student through questions and activities to discover, discuss, appreciate and verbalize the new knowledge.

Don't get me wrong. For the right topic, executed the right way with the right teacher and class, this can be a great, wonderful, spectacular and really good approach to education. Unfortunately, education reformers (particularly the current crop), are not good at conditional problems. They tend instead to fall into the new tool camp (you know the saying, "to a man with a new hammer, the whole world is a nail.").

Worse yet, (and I'm afraid there's no nice way to say this) many of the educational theorists don't have a firm grasp on the subjects they are working with. This is never more plain than in constructionist science classes that almost entirely eschew lectures and traditional reading assignments and instead have the students spend their time conducting paint-by-numbers experiments, recording the results and performing a few simple calculations.

To most laymen, that's what science is: stuff you do while wearing a lab coat. Most people don't associate science with forming hypotheses, designing experiments, analyzing results and writing papers and, based on my limited but first hand experience, many science educators don't give those things much thought either.

The shining exception to the those-who-can't-teach-teach-teachers rule is George Polya. Though best known as an educational theorist, Polya was a major Twentieth Century mathematician (among his other claims to fame, he coined the term "central limit theorem") so he certainly fell in the those-who-can camp.

But it it important to note that Polya advocated guided discovery specifically as a way of teaching the problem solving process. I suspect that when it came to simply acquiring information, he would have told his students to go home and read their textbooks.

"Reasons to teach what we teach"

I've got a long post up on Education and Statistics on the reasons for including a concept in the mathematics curriculum and how those reasons should affect what we teach and how we teach it.

Here's an excerpt:

Where a topic appears on this list affects the way it should be taught and tested. Memorizing algorithms is an entirely appropriate approach to problems that fall primarily under number one [Needed for daily life]. Take long division. We would like it if all our students understood the underlying concepts behind each step but we'll settle for all of them being able to get the right answer.

If, however, a problem falls primarily under four [helps develop transferable skills in reasoning, pattern-recognition and problem-solving skills], this same approach is disastrous. One of my favorite examples of this comes from a high school GT text that was supposed to develop logic skills. The lesson was built around those puzzles where you have to reason out which traits go with which person (the man in the red house owns a dog, drives a Lincoln and smokes Camels -- back when people in puzzles smoked). These puzzles require some surprisingly advanced problem solving techniques but they really can be enjoyable, as demonstrated by the millions of people who have done them just for fun. (as an added bonus, problems very similar to this frequently appear on the SAT.)

The trick to doing these puzzles is figuring out an effective way of diagramming the conditions and, of course, this ability (graphically depicting information) is absolutely vital for most high level problem solving. Even though the problem itself was trivial, the skill required to find the right approach to solve it was readily transferable to any number of high value areas. The key to teaching this type of lesson is to provide as little guidance as possible while still keeping the frustration level manageable (one way to do this is to let the students work in groups or do the problem as a class, limiting the teacher's participation to leading questions and vague hints).

What you don't want to do is spell everything out and that was, unfortunately, the exact approach the book took. It presented the students with a step-by-step guide to solving this specific kind of logic problem, even providing out the ready-to-fill-in chart. It was like taking the students to the gym then lifting the weights for them.

This is a bit of a work in progress so if you have any relevant experience in mathematics education (and, yes, experience as a student definitely counts), I would greatly appreciate it if you went by and joined the conversation.

Interesting Passage

Via Tyler Cowen of Marginal Revolution fame comes an interesting article on school choice. I was particularly struck by this passage:

This approach helps explain why choice advocates — inclined to approach choice-based reform not as a regulatory question, but as one of justice and rights — have spent so much less time considering the dynamics of deregulation than have pro-market reformers in sectors like transportation, telecommunications, and cable television. Because education reformers have approached choice not as a matter of political economy but as a moral crusade, they have favored grand, sweeping claims over empirical reality.

I think that this has a lot of explanatory power for the postive treatment that charter schools are getting in the press and among pundits. Mark has written about this extensively on this blog.

I think breaking apart the moral and the policy dimensions would be a positive development. The issues with under-servicing of some groups are broad but there is a risk that alternatives could make things worse.

Homework and treatment levels

Previously on OE, we've talked about how norming, placebo and volunteer effects can cause methods like lottery-based analyses to miss significant sampling effects. Here's one more area of potential concern.

The basic problem here is familiar to clinical researchers. You're trying to determine the optimal level of a drug or therapy or exercise regime or type of food. There is an ideal dosage, the level of treatment you would prescribe if your only concern was optimizing the effect. Then there is what we might call a realistic dosage, one that takes into account factors like side effects and compliance. These levels vary from group to group which means the process of finding the right dosage is sensitive to sampling issues. The appropriate level is likely to be different for a group of college students and a group of senior citizens.

Teachers assigning homework face essentially the same problems. Assuming that goal is to improve students' score on a standardized test, there is an ideal homework assignment for each lesson, a subset of the problems available in the text that will tend to maximize the average score of a class (to keep things simple we'll limit the discussion, with no loss of generality, to homework assignments derived from the textbook).

The ideal assignment assumes that students have unlimited time and will complete all the assigned problems. The realistic assignment would take into account factors like time constraints, demands from other classes, compliance, burnout and parental pushback (I can tell you from experience that parents complain if they feel their children are being given excessive work and I would argue this is a good thing -- teachers should remember that their students' time also has value). The realistic assignment would optimize the class's average score when these constraints are in place.

Like the previously discussed norming, placebo and volunteer effects, a sampling/treatment level interaction with the homework assignment here can interfere with the ability of lottery-based analysis to detect sampling effects.

To see how this would work, consider a charter school like one in the KIPP system that very publicly acknowledges that students will be expected to do large amounts of homework each night (some schools even require parents to sign contracts agreeing to sign off on the students' homework). Students who apply for this school are aware of these requirements, as are their parents. This certainly raises the possibility that the optimal realistic homework level might be higher for these students, particularly if this is one of those schools that aggressively culls out the students who can't handle the workload.

If this is the case and if both the charter school and the public schools assign the optimal levels of homework for every class, you will have a sample based effect that will completely evade detection by a lottery-based analysis. That analysis will compare the performance of those who applied for the charter and were accepted against those who applied but lost the lottery. Since the rejected students will receive a treatment level that was optimized for the general population while the accepted students will receive a treatment level optimized to their particular subgroup, we expect the charter school students to do better, leading the lottery-based analysis to incorrectly conclude that there is no selection effect.

I used homework in this post to keep things simple but the principle applies just as well to most elements of the educational models of many highly praised charter schools -- longer days, Saturday instruction, extended school years, holding students responsible for more material on tests. Put bluntly, these schools get their results by giving students lots of work. Because of these results, these schools are often held up as examples for the rest of the educational system, but when you start with a self-selected sample then counsel out the students in that sample who have trouble keeping up, you should at least consider the possibility that the optimal workload for your student body is higher than the optimal workload for the general populace.

This doesn't mean that all high performing charter schools are simply running on undetected selection effects and it certainly doesn't mean that charter schools with high standards should lower them. What is it does mean is that selection effects in education show up in subtle and complex ways. It remains extraordinarily difficult to resolve these issues through observational means. Any study that claims to have settled them should be approached with great caution.

Arnold Kling

Arnold Kling is a skeptic about the importance of peer effects:

Back when the SAT was just math and verbal, I described us as living on the other side of a 250-point SAT gradient--in the better high schools, the SAT's were 1200, and in our high school they were 950. (I took the view that my own kids did not need high-achieving peers in order to do well on their own.)

I am not sure whether this is persuasive (although it does match my personal experience). But it is true that this is the sort of behavior that I would expect from peer-effects skeptics (and if we saw more of it there would be a much smaller premium on good schools for housing costs.

Thomas Jefferson on income inequality

From a letter to James Madison (courtesy of Brad DeLong):

I am conscious that an equal division of property is impracticable, but the consequences of this enormous inequality producing so much misery to the bulk of mankind, legislators cannot invent too many devices for subdividing property, only taking care to let their subdivisions go hand in hand with the natural affections of the human mind. The descent of property of every kind therefore to all the children, or to all the brothers and sisters, or other relations in equal degree, is a politic measure and a practicable one. Another means of silently lessening the inequality of property is to exempt all from taxation below a certain point, and to tax the higher portions or property in geometrical progression as they rise.

Whenever there are in any country uncultivated lands and unemployed poor, it is clear that the laws of property have been so far extended as to violate natural right. The earth is given as a common stock for man to labor and live on. If for the encouragement of industry we allow it to be appropriated, we must take care that other employment be provided to those excluded from the appropriation. If we do not, the fundamental right to labor the earth returns to the unemployed. It is too soon yet in our country to say that every man who cannot find employment, but who can find uncultivated land, shall be at liberty to cultivate it, paying a moderate rent. But it is not too soon to provide by every possible means that as few as possible shall be without a little portion of land. The small landholders are the most precious part of a state.

Profundity adjacent

I'm not a fan of the BBC's the Changing World. The show has interesting topics but the episodes I've caught have an annoying habit of almost saying something profound and insightful then veering off at the last minute. This tendency to be profundity adjacent is particularly notable because, on KPCC, Changing World airs a couple of hours after This American Life, a show with an extraordinary ability to uncover the genuinely important aspects of a complicated story.

The criticism certainly holds for this episode on chess. Repeatedly, I found myself caught up in an interesting thread only to have it end right before it got to the good part. Still, they were interesting threads and if your interests include abstract strategy games and their role in fields like education, you should probably check this out.




http://www.thechangingworld.org/archives/2010/44.php

On Salmon's blog, even the comments are quotable

Felix Salmon has a good post on the economic impact of extending unemployment benefits but for me the most insightful observation was found in the comments:

What’s missing from this discussion is the part about optimization of deployment of resources. If a skilled worker, say a skilled aluminum welder, can stay on unemployment long enough to find a skilled aluminum welder job at higher pay and productivity, that’s good for GDP and has a higher multiplier. But if that worker has to settle for a security guard job because he or she must pay the rent, then there is a net loss of productivity, GDP, and related multipliers.