Distributions

John Cook has another insightful post today on data distributions. It is an area that I know that I could stand to develop a fuller intuition about.

One point that I think comes out of his post is the idea that no real data ever fully fits a theoretical distribution. Ever. After all, all real data has noise in it. While this seems to be a obvious point, it is possible to get reasonable patterns of residuals using different distributions to fit data. Which could mean it is either distribution, or possibly neither.

Even worse, real data is sometimes a mixture of distributions based on a latent (or non-latent variables). Andrew Gelman has an example with height -- male and female adult heights are both approximately normally distributed but the combination of the two is not (there is a nice picture on page 14 of "Data Analysis using regression and multilevel/hierarchical models").

Even worse, there may be factor (e.e. genetic) that result in different mean adult heights. So you can get "fat tails". This is not a small problem as it means that your models will wildly underestimate the probability of an extreme result. I was not a huge fan of the Black Swan, but this point was correct (and, to be fair, it was the central theme of the book).

All of which is to say that I am definitely going to have to think more about this issue and , hopefully, see cases where I am not making the correct assumptions.

Complex processes are hard to simplify

One thing that I have noticed is that simple narratives about employment often gloss over important details. Mark has an example with teacher employment. Yves Smith has another one, where she points out that employers who are having trouble finding employees often manage this only by offering very poor terms of employment:

This does not add up. If the company can afford to spend ten weeks training people (and the additional cost of setting up a course), that suggests it could have offered more than $13 an hour, particularly given the opportunity cost of the orders it could have filled if it had had people on board sooner. The article later notes that Mechanical Devices “hire[s] through staffing agencies to help control health-care costs and maintain flexibility.” Um, that means they fire people as soon as orders fall.

I think that this issue is an old idea that we often forget. The jobs that are difficult to fill (in a functional economy) either require complex skills/credentials or have a known downside. I would worry that Yves Smith understates the case with her example: the worst of jobs are the ones where you never actually become unemployed but for which you don’t have a predictable income. Economists back to Adam Smith have noted that a lack of job security tends to raise wages.

Are we surprised that experiments with unpredictable income and low wages don't always work out? After all, there is a minimum income threshold below which basics like food, shelter and hygiene may start to become difficult to maintain.

In the same sense, why are we worried that it is hard to break into education? Many professions are hard to break into without, for example, credentials. Neither lawyers nor medical doctors can practice without the specific credential? Why is the lack of a graduate degree making it hard to become a teacher in competive markets an issue?

I suppose we could re-imagine the economy from the ground up. But top down approaches to comprehensive social and economic reform don't always result in wild successes and may generate adverse effects in the process.

This is not to say that there are not real issues to be resolved that can occur due to technological change. I just worry when the proposed solutions to a complex problem are extremely simple. That suggests either an agenda or a failure to appreciate all of the moving parts involved.

Now, before I get to be labeled an arch-conservative, I tend to think that slow and incremental reform is the best choice in most cases. There can be specific cases where the slow reform approach has failed completely and where the options are limited, but I'd rather be cautious rather than aggressive in finding them.

A better-late-than-never edition of reasons not to trust Naomi Schaefer Riley

As we used to say back in the Ozarks, I've been busier than a one-legged man at in an ass-kicking contest, so I still don't have time to go through the logical and factual problems with this piece in the Wall Street Journal Op-Ed section, but I have to take a moment to point out that Riley defines 'impossible' in much the same way that Newman defines 'rarely.'

In the spring of 1989 Wendy Kopp was a senior at Princeton University who had her sights set on being a New York City school teacher. But without a graduate degree in education or a traditional teacher certification, it was nearly impossible to break into the system. So she applied for a job at Morgan Stanley instead.

"Nearly impossible' in this case means 'mildly inconvenient.' It was no big deal. Lots of people did it. I know this because at around the same time Ms. Kopp was cranking out spreadsheets in Lotus 1-2-3, I was teaching high school despite the fact that I had neither an education degree nor traditional teacher certification.

I'm not saying Ms. Riley is woefully ignorant in her area of supposed expertise. I'm not saying that she's a liar. I'm just saying we've got the possibilities narrowed down.

Social Security Thought

I am not an economist and I don’t necessary have a strong understanding of personal finance. Mark brought up the issue of wealth transfer as a part of the whole social security debate. But I will admit to one additional element of puzzlement when it comes to privatizing social security.

Namely, precisely what would one invest in? If one is investing in government bonds (the safest instruments) then isn't the government just writing an IOU to the bondholders? The only difference I can see between a personal account investing in US treasury bonds and social security is that social security can be amended, like in 1983, without a default on United States bonds.

Or we could invest in stocks, but them we have risk due to issues like decades with a net negative return. I'd always naively assumed that the real role of programs like social security was to smooth out variability in returns (as the government can afford to plan for the real long run) and that'd seem to be defeated by personal accounts.

Hiltzik puts SS in plain English

Michael Hiltzik is one of the reasons I still feel that, despite cruel budget cuts and bad management, the LA Times is a better paper than the NYT. The latter gives us stories like this. Hiltzik gives us this:

What trips up many people about the trust fund is the notion that redeeming the bonds in the fund to produce cash for Social Security is the equivalent of "the government" paying money to "the government." Superficially, this resembles transferring a dollar from your brown pants to your gray pants — you're no more or less flush than you were before changing pants.

But that assumes every one of us contributes equally to "the government," and by equal methods — you, me and the chairman of Goldman Sachs.

The truth is that there are two separate tax programs at work here — the payroll tax and the income tax — and they affect Americans in different ways. The first pays for Social Security and the second for the rest of the federal budget.

Most Americans pay more payroll tax than income tax. Not until you pull in $200,000 or more, which puts you among roughly the top 5% of income-earners, are you likely to pay more in income tax than payroll tax. One reason is that the income taxed for Social Security is capped — this year, at $106,800. (My payroll and income tax figures come from the Brookings Institution, and the income distribution statistics come from the U.S. Census Bureau.)

Since 1983, the money from all payroll taxpayers has been building up the Social Security surplus, swelling the trust fund. What's happened to the money? It's been borrowed by the federal government and spent on federal programs — housing, stimulus, war and a big income tax cut for the richest Americans, enacted under President George W. Bush in 2001.

In other words, money from the taxpayers at the lower end of the income scale has been spent to help out those at the higher end. That transfer — that loan, to characterize it accurately — is represented by the Treasury bonds held by the trust fund.

The interest on those bonds, and the eventual redemption of the principal, should have to be paid for by income taxpayers, who reaped the direct benefits from borrowing the money.

So all the whining you hear about how redeeming the trust fund will require a tax hike we can't afford is simply the sound of wealthy taxpayers trying to skip out on a bill about to come due. The next time someone tells you the trust fund is full of worthless IOUs, try to guess what tax bracket he's in.

Quote of the day

In Unqualified Offerings, Thoreau made the following quote about reform in higher education:

In any other line of business, a person who says “How hard can it be to do this efficiently?” is usually a clueless idiot. Ditto for many aspects of higher education.

It gives one pause.

I am not certain that word means what they think it does

From the New York times comes the news that Once a Leader, U.S. Lags in College Degrees. Let us see what is meant by the word lags:

The United States used to lead the world in the number of 25- to 34-year-olds with college degrees. Now it ranks 12th among 36 developed nations.

I suppose that it is true that the United States is lagging behind the number one country. Who is that?

Canada now leads the world in educational attainment, with about 56 percent of its young adults having earned at least associate’s degrees in 2007, compared with only 40 percent of those in the United States. (The United States’ rate has since risen slightly.)

So a small country with a highly credential based society and a large network of Universities and Colleges with financial incentives to expand is actually able to beat the United States in proportion of degrees?

I am curious as to whether this change is due to Canadian improvements or US decline? But, either way, being in the top half of developed countries is an odd definition of "lags".

Finally, let's look at the proposed remedy:

The group’s first five recommendations all concern K-12 education, calling for more state-financed preschool programs, better high school and middle school college counseling, dropout prevention programs, an alignment with international curricular standards and improved teacher quality.

I am unclear exactly how these things directly link with the proportion of college graduates in the general population. One might think that a better place to start would be to look at what the Canadians are actually doing. After all, they are only a short drive north . . .

OT: Paizo

I think it has been a year since I did an off-topic post on role-playing games. One year ago the company Pazio released a new role-playing game based on the d20 system (August 2010). In that time they have managed to produce a pretty amazing set of high quality products. Nice art, high production values and a very well written set of books. It reminds me of the high quality work that Iron Crown Enterprises did in the 1980's in their Middle Earth books.

Well worth checking out!

Too many talks

I have four talks this year at the conference I am attending. I now officially promise that this is a high water mark. In future years, I plan to actually relax at the conference and plan to spend at least some of the time networking.

It's unclear what I was thinking (or if thought was part of the process).

The role of luck in science

Candid Engineer has a nice question about the role of luck in science. It'sa good question and it is quite true that careers tend to be sensitive to initial conditions. I don't think that this is entirely true in science but it applies to many fields. Michael Lewis describes this phenomenon (being in the right place at the right time leading to better career opportunities) occurring to him mostly due to a single major trade early his career as a band trader in "Liar's Poker". Heck, he describes being hired entirely due to lucky meeting with the wife of a senior member of the corporation.

There is no way to eliminate luck in science. Even the assignment of reviewers to one's grants and papers can have an important influence on one's ultimate success.

When I was a physicist, I got some very odd readings in some of my experiments. We tried varying key parameters, replicated them and always got the same fascinating results. However, other labs had trouble finding the same thing and, as a result, I never published these results. It's good thing -- the error was finally detected in the analysis software!! Was it bad luck that the software had a bug in it for the exact type of data we had but worked well on the standard testing samples? Of course, with enough work or a flash of insight the bug might have been detected with less work. But ruling things out starts from most likely to least likely and, in this case, we guessed wrong.

So the best one can do is to work hard, focus on the best betsand roll with the punches. I'll leave you with a quote from physioprof (that I thought was the best take on the issue so far):

I hate that saying as well, because it doesn’t capture the most important aspect of “attracting good luck”. In order to maximize your chances of getting lucky, you need to maximize your exposure to beneficial risk. This has much more to do with the judicious application of hard work in wisely chosen directions than it does the sheer volume of hard work.

More on Trower

The tenure articles posted by round-table Mark tenure really do have the best of quotes. Let us return to the quote by Cathy A. Trower:

Research shows that Generation X values qualities that are in conflict with this system: collaboration, not competition; transparency, not secrecy; community, not autonomy; flexibility, not uniformity; diversity, not homogeneity; interdisciplinary structures, not disciplinary silos; and family-work life balance, not “publish or perish” careers.

There was so much wrong with this quote that I decided it deserved a second post. By setting her argument up this way, Cathy seems to be associating the academy with: competition, secrecy, autonomy, uniformity, homogeneity, disciplinary silos and poor work-life balance. Now it is possible that some of these issues could be altered (in a positive direction) by the removal of tenure.

But some of them are likely to move the opposite direction if tenure is abolished. I am unclear about the conditions in the Gamma quadrant, but are we sure that work-life balance is going to be improved by reducing job security? People feel less pressure to work hard when they do not have job security? Or what about competition -- does this mean that if private industry wanted to create a more competitive environment they would offer more job security?

Now I don't want to construct a straw person argument here there are three points that I think are really important. One, if one wants to re-envision the academy it is necessary to fully spell out the alternatives and not just attack one feature that people find unfair. Ideally, this would be done with the clear understaniding of how successful top-down reforms typically are. Do note that this is a critique that begins with the idea of competition as being bad!

Two, it is odd to attack a key instituition of the American academy at the point when it is at the peak of it's success. Look carefully at the Academic Rankings of World Universities (a Chinese project, by the way) and see how many American schools are ranked in the top 100 (in 2009, it was 8 of the top 10 universities with the other two being British). Sure, the methodology can be criticized but it's not a sign of complete failure to rank so strongly in international rankings.

Three, I find it odd how people simplify the academic enivironment as if there was a single system across the entire United States. Working in an NIH funded biomedical research shop is very different than teaching english at a community college. The stresses and solutions are very different. Recognition of this compexity would do a lot to refine the arguments being presented.

But, to be fair, I am not sure that the virutes of competition, autonomy and disciplinary focus are things that we want to get rid of (and I am sure that work-life balance is unlikely to improve with less job security). That the list of problems, itself, contains virtues is a rather interesting dilemma and it does rather make me wonder what the end state looks like.

University Towns

One interesting feature of American life is the University Town. These have survived and thrived, even as towns based around corporations have tended to be troubled (See: Michigan, Flint). However, unlike company towns, university towns rely on the ability to convince people to move there (as universities are historically reluctant to recruit their own graduates as faculty).

This is one of the key problems with statements like this one, made by Mark C. Taylor:

A middle ground will address most of the problems. After a trial period of three to five years, faculty members who merit promotion should be given seven-year renewable contracts. For this system to work effectively, these reviews must be rigorous and responsible.

I am not saying that you could not get this to work. But it is going to change the dynamics of these positions considerably. People are often willing to move cross country for the chance of permanent employment. But who is willing to do a major relocation to a small town for a series of rotating contracts? Even more important, these are people who have spent a decade (or more) in school and who are ill positioned to make major risks. And, under these conditions, buying a house counts as financial circumstances may make the decision not a renew contracts happen in clusters.

It is true that, ideally, there would be no global financial considerations placed on the decision to renew a contract. But that seems like an idealistic assumption.

Now, it is true that universities in big cities don't have this issue. However, in the United States and Canada (where I have experience), it is the universities in large (and diverse) cities that can afford to not have tenure now. You can decide you like Toronto as a city and be willing to find another job if the contracts at the University of Toronto don't work out. It's hardly the same at Lakehead University (in the small and very blue collar city of Thunder Bay) where relocation is likely to be hundreds of miles should employment fall through and alternative employment options are thin.

So one item that is being missed in the tenure debate is the heterogeneity of universities, themselves, and how much local conditions matter.

Academia and Diversity

Mark had a link to a tenure round-table that was a good reflection of the current argument about tenure that seems to be making the rounds. It's a rich area for discussion but I wanted to focus on an argument by Cathy A. Trower:

Research shows that Generation X values qualities that are in conflict with this system: collaboration, not competition; transparency, not secrecy; community, not autonomy; flexibility, not uniformity; diversity, not homogeneity; interdisciplinary structures, not disciplinary silos; and family-work life balance, not “publish or perish” careers.

Curiously enough, by making the argument that the institution of academia is in conflict with the current values of Generation X might be one of the best arguments to leave well enough alone. Consider, the principle of diversity is that diversity, by trying a lot of different approaches to problems and being inclusive of a wide range of viewpoints, allows us to find solutions more efficiently. The idea that a long standing institution is taking a wildly different approach than that of the current default approach actually suggests that keeping the system in place is a way to enhance diversity.

This is not to make me an apologist for the academy. There are many issues with the modern academic system that could use resolution. But the argument that it is in conflict with the values of a particular generation seems to be one of the weaker arguments for doing away with the whole enterprise.

I'm too busy to comment on this at length...

but the NYT has a roundtable on tenure. Pretty much the same old lines. With the exception of Kezar (who does, at least, ask some good, tough questions), no one adds anything original or insightful to the discussion, but there is, I suppose, some value in seeing where the debate has come to a dead stop.

• At Stake: Freedom and Learning

  Cary Nelson, University of Illinois at Urbana-Champaign

• Unsustainable and Indefensible

  Mark C. Taylor, Columbia University

• No Tenure, No Nothing

  Adrianna Kezar, University of Southern California

• Reducing Intellectual Diversity

  Richard Vedder, economist, Ohio University

• How to Start Over

  Cathy A. Trower, Harvard School of Education

One final(?) reason not to trust teaching metrics

Don't be too afraid (or perhaps relieved) at the thought of OE dropping the subject of education and education reform. I think I can speak for Joseph here when I say that neither of us plan to drop the topic until pundits stop saying stupid things about it.

What follows might be a final reason because I think we've got a pretty good framework for discussing the dangers of making radical changes based on the proposed metrics for measuring teacher performance. Do a keyword search on education to see what I mean.

A few months ago, Joseph and I had a marathon phone conversation where we sketched out the attributes a well designed study of teacher performance would have to have in order to cope with self-selection, interactions, social dynamics, nesting and the myriad other challenges that go with this problem. You will occasionally find an educational study that attempts to deal with one or two of these challenges but, as far as I can tell, no educational study address all of them.

Some of these problems have been widely discussed. Others have been examined in some detail in this blog. But there is at least one that we haven't gotten around to: the question of how certain teachers mesh with certain classes.

Every teacher, no matter how skilled or experienced, will do better with some types of classes than with others. Different classes have different needs. Some need a fast pace; some need patience; some need freedom to explore; some need structure and discipline. After you've been teaching for a while you learn to read classes and adjust your style and presentation but no teacher is equally good under all conditions.

This raises two serious problems for the educational researcher: first, how do you assign classes to teachers and students to classes in a way that protects you from aliasing problems?; second, how do you interpret the results?

Here's a example, let's say teacher A does well with remedial math but does badly with calculus. Teacher B is great with calculus but simply can't communicate with the remedial kids. How do you score those teachers? You could use the mean and conclude that both teachers were doing an average job -- a conclusion that is pretty much wrong in all four cases. You could take the higher of the scores, working under the assumption that administrators are competent managers and therefore know enough to put teachers in classes that match their abilities. Or you might decide that since the purpose is to catch bad teachers we should take the low score. Or you could pick calculus because it's more advanced. Or you could pick remedial because it affects those kids most likely to be left behind.

Of course, all of this presupposes that we have this kind of information available when we try to measure teacher performance (which we never do). It also assumes that we want to have a serious discussion using meaningful data, that we want to be honest and fair, and that we actual care about the quality of our kids' education.

Perhaps it's just the lateness of the hour and the weight of a long week, but I find it increasingly difficult to hold onto that last set of assumptions.




(I'm writing this under the gun and corners are being cut. I will try to go back and add links when the storm passes.)