The irrationality of engineers

Andrew Gelman has a new post pointing out a contradiction that keeps popping up in the pop econ world. You should read the whole piece (as well as Joseph's take on it) but this part in particular caught my eye:

The key, I believe, is that "rationality" is a good thing. We all like to associate with good things, right? Argument 1 has a populist feel (people are rational!) and argument 2 has an elitist feel (economists are special!). But both are ways of associating oneself with rationality. It's almost like the important thing is to be in the same room with rationality; it hardly matters whether you yourself are the exemplar of rationality, or whether you're celebrating the rationality of others.

This is an excellent point but I'd go further. The economist's 'rationality' is, as mentioned before, analogous to the statistician's 'significance.' Both are highly specific terms that are just close enough to their general usage to get people into trouble.

I previously gave some silly examples of 'irrational' decision making, here's a more practical one: you're an engineer designing a missile. You're concerned with accuracy, range, speed, payload and cost. The 'rational' approach would go something like this -- for each attribute, come up with a function that assigns its level a dollar value, then pick design x where a(x) + r(x) +s(x) + p(x) - cost(x) is optimized. I suspect that most engineers would find this 'rational' approach highly irrational and would usually choose, instead, to hold three or four of these attributes to an acceptable threshold and optimize the remaining one or two. (check out the cheap beer post for more on thresholds and irrationality.)

There's nothing wrong with having a specialized, field specific definition for rationality; there is something wrong with using confusion over different meanings of the term to make yourself look good.

Popular Micro-economics

Andrew Gelman writes:

Pop economists (or, at least, pop micro-economists) are often making one of two arguments:

1. People are rational and respond to incentives. Behavior that looks irrational is actually completely rational once you think like an economist.

2. People are irrational and they need economists, with their open minds, to show them how to be rational and efficient.

Argument 1 is associated with "why do they do that?" sorts of puzzles. Why do they charge so much for candy at the movie theater, why are airline ticket prices such a mess, why are people drug addicts, etc. The usual answer is that there's some rational reason for what seems like silly or self-destructive behavior.

Argument 2 is associated with "we can do better" claims such as why we should fire 80% of public-schools teachers or Moneyball-style stories about how some clever entrepreneur has made a zillion dollars by exploiting some inefficiency in the market.

[sorry for the general link, there seemed to be technical issue linking to the specific article]

I think that this insight is quite compelling and exposes a real issue with popular micro-economics. After all, these two positions mostly contradict each other!

I see this contraction arising for different reasons than Andrew does. I think that economists are often forced to make strong assumptions in order to deal with the sorts of complex problems that they look at. So they presume rationality on the part of all actors when trying to explain problems (i.e., descprive work). But when they want to improve matters they shift and reject this assumption (as it would suggest everything is already optimized). So they make different (strong and unverifiable) assumptions depending on whether they are trying to explain behavior or give guidence to improve outcomes.

What seems to be more alarming to me that is that this pool of economists often don't "sanity check" the effect size of their analysis. Perhaps it is my background in epidemiology, but if I see a hazard ratio of 5 then I am immediately suspicious that it is too good to be true. However, Ray Fisman can see an analysis that suggests firing 80% of teachers and not necessarily wonder if perhaps there is an overly strong assumption in his analysis (like the pool of new potential teachers not having finite limits or in the real sensitivity of the evaluation process).

Anybody's culpa?

In what might be considered a commissioned blog post, Jonathan Robinson manages to dig up some comments that Steven Levitt would probably like to rebury.



It is, of course, possible to make too much of a bad call or even a bad argument. If the penalty for being wrong is too high it inhibits the conversation, but there does have to be some kind of accountability. When you're this far off on something this big, you really ought to do something: modify your reasoning, concede that your assumptions may need some work, maybe even just acknowledge the error but dismiss it as an anomaly.

Instead, we have a debate where, as Paul Krugman has repeatedly pointed out, no one ever has to admit he or she is wrong. You will occasionally see someone step up, but it's strictly done on a volunteer basis.

Maybe we need to go beyond the honor system on this one.

Subtle satire or ugly glimpse into the mind of the NYT -- you be the judge

Intentional or not, Christoph Niemann has perfectly captured New York journalists strange belief that the rest of the country is fascinated by the minutia of NYC (the relative size of Park Slope and Africa is particularly telling). As at least one commenter noted, this would seem to be a remake of the classic Saul Steinberg cartoon without the self-awareness.

California Education

Janet D. Stemwedel quotes Jerry Brown

I have reviewed the Mercer compensation study and have reflected on its market premises, which provide the justification for your proposed salary boost of more than $100,000. The assumption is that you cannot find a qualified man or woman to lead the university unless paid twice that of the Chief Justice of the United States. I reject this notion.

This is a very odd American idea: that without a huge salary, no competent person can ever be found. I am not sure that this idea passes the "laugh test" as seen by Governor Brown's comments. It also has some very pernicious effects on things like income inequality and the perception of fairness (this has not been the best time for California universities from a fiscal point of view).

I am happy to discuss these matters from first principles but the idea that qualified people cannot be found for less than absurd amounts of money is just berserk.

Grade Inflation

From the New York Times:

Another must-hear from This American Life

Another really outstanding piece of journalism from TAL, this time focusing on the extraction of natural gas from Pennsylvania's Marcellus shale. You can download it for free today and tomorrow but I'd encourage you to donate a few bucks if you can. They do good work.

Felix Salmon addresses something that's been bothering me for a long time

You should read the whole thing, but this passage in particular caught my eye:

The deeper thing going on here is what might charitably be called a momentum trade, or what might less charitably be called a sell-low strategy. First you pick your sector, then you pick funds in that sector. If the funds do well, that’s great; if they do badly, then you sell those funds and instead you buy funds which did do well over the time period in question. Then, next year, presumably, you rinse and repeat.

I realize this is not my field and there are a lot of subtle questions here about random walks and efficiency, but a great deal of the financial advice I hear reminds me of an ad from the late Nineties. In it a man is listening to the radio and hears about a stock that has just doubled in price. He rushes to his computer to buy some of this newly expensive stock only to discover that his cut-rate internet provider is down. He howls in anguish. (even at the time I remember thinking "so this is what a bubble looks like.")

I understand that there's more to investing than buy low/sell high, but many investment strategies seem to strive for the opposite. Wouldn't there have to be an unreasonably big momentum factor for this to make sense?

Wise Bloodsport

OE readers have the coolest noms de plume.

Modeling assumptions

From Matt Yglesias:

I’ll note, however, that you might be a freshwater economist if you think it makes sense to reassure us that a deflationary spiral is impossible because your model says so even though deflationary spirals do, in fact, occur in human history. To me, a model that denies the possibility of something happening that does, in fact, happen indicates that you’re working with a flawed model.

I can't comment on whether or not this is a fair assessment of the work in question. But it is always a good idea to "reality check" model outputs and ensure that the distribution of events generated by the model looks something like real data. If important events occur in real data that your model dismisses as impossible than model misspecification or missing confounding variables should be immediately suspected.


EDIT: Noah Smith also comments and it is well worth the read. He traces these conclusions to some rather strong assumptions . . .

Challenges of causal inference in complex systems

From Felix Salmon:

“Spend less money, create more jobs” is the kind of world one normally finds only in Woody Allen movies, and it’s a profoundly unserious stance for any politician to take. Spending cuts, whether they’re implemented by the public sector or the private sector, are never going to create jobs. And there’s simply no magical ju-jitsu whereby government spending cuts get reversed and amplified, becoming larger private-sector spending increases.

I think that one of the difficulties in macroeconomics is that you have complex systems that are not subject to experimentation. So you are forced to try and use observational studies and analogies with microeconomics to try and determine the causal effects of policies. Even instruments are questionable as they also rely on unverifiable, strong assumptions.

The inability to have a consensus on the counter-factual is pernicious and causes no end of trouble. Consider the tax increases passed at the beginning of the Clinton administration. Are they responsible for the late-1990's boom, unrelated to it, or did they act to slow it down (making the current economy smaller than it could have been)? How would you know this?

Cross country comparisons are possible but you have both confounding factors and effect measure modification. Changing the tax rate in Sweden might have different consequences than in the United States due to both different cultures (confounding) and to differences in current tax rates (effect modification). So, by picking different analogies and different models for the observational data, we can end up with some really strange claims being made about how economies work.

It is not an area with easy solutions. But I think to agree with Felix that the model he is critiquing is making heroic assumptions about the influence of tax levels on economic growth.

Harnessing the power of tax evasion to address climate change

From the comment section for a recent post on the crisis in Greece:

sub-divided said...

The thing I know about Greek tax-evasion is that they know where you live when you hook into the electric grid. So a family I know, owning a small island, avoids paying any taxes by running everything on wind and solar power.

Case Crossover paper and time trends

There was a new paper e-published recently in Pharmacoepidmeiology and Drug safety that used the case-crossover study design:

"Purpose
Elevated levels of phosphorus (P) and calcium (Ca) have been shown in observational studies to be associated with an increased risk of adverse clinical outcomes including mortality. Vitamin D sterols have been shown to increase the risk of hypercalcemia and hyperphosphatemia in clinical trials. We sought to explore these risks in real-world clinical practice.
Methods
We employed a case–crossover design, which eliminates confounding by non-time-varying patient characteristics by comparing, within each patient, vitamin D doses before the event with those at an earlier period. Using this method, we estimated the risk of hypercalcemic (Ca ≥ 11 g/dL) and hyperphosphatemic (P ≥ 8 g/dL) events for patients at different dose quartiles of vitamin D relative to patients not on a vitamin D sterol.
Results
There was a dose-dependent association between vitamin D dose quartile and risk of hypercalcemia or hyperphosphatemia. In adjusted analyses, each increase in vitamin D quartile was associated with a multiple of hypercalcemia risk between 1.7 and 19 times compared with those not on vitamin D and a multiple of hyperphosphatemia risk between 1.8 and 4.
Conclusion
Use of vitamin D sterols is associated with an increased risk of hypercalcemic and hyperphosphatemic events in real-world clinical practice. Other potential predictors of these events, such as phosphate binder use and dialysate Ca levels, were not examined in this analysis."

It seems to be an interesting paper but I have one concern. If you look at the discussion section of the paper, the authors note that:

In our sensitivity analysis, we used 1-month periods to assess vitamin D exposure. In this analysis, estimates of the association between vitamin D dose and risk of events were smaller than those in the primary analysis, particularly for hypercalcemia. One possible explanation for this finding is that the average 2-month exposure measure is a superior indicator, compared with the 1-month assessment, of both the dose and duration of vitamin D exposure. As well, it could be that some dose changes in the month prior to the event had already occurred in response to increasing Ca levels and that, for this reason, the dose 2 months prior to the event is a more accurate reflection of the dose that gave rise to the hypercalcemic or hyperphosphatemic event.

Another explanation that I did not see addressed is the possibility that there is a time trend occuring. If the frequency of vitamin D administration (or the dose) increased with time then you would expect to see smaller estimates in the sensitivity analysis as well. But it would be an artefact of changing exposure over time.

That being said, it was a real pleasure to read a well justified use of the case-crossover design in a medication paper. Hopefully this is a sign that there will be more use of with-in person study designs in the future in epidemiology. The ability to handle time invariant confounders is a serious advantage of this approach.

Transformations

Frances Woolley has a post on the use of the inverse hyperbolic sine transformation for handling wealth as a variable (skewed and with lots of zeros).

The post is worth reading and the comments are really interesting. In particular, Chris Auld makes a very good case for simplicity and interpretability as a desirable property of statistical models in several of the comments.

There is also a thought provoking discussion of how to parameterize wealth that involves the sort of deep thinking about variables that we should do more of in epidemiology. In particular, in what sense is it reasonable to consider a person (especially in a country like Canada with strong entitlement programs) to truly have zero wealth.

Definitely worth the read.

The Chameleon XLE

James Surowiecki has posted a thoughtful piece on Greece's culture of tax evasion, but it was this fairly trivial though amusing detail that caught my eye.

Even while dealing with protests and open riots, the new Greek government is trying to change things. It is rationalizing its tax-collection system. It has simplified taxes and done away with some of the loopholes. And it has stepped up its enforcement efforts in ways large and small—tax officials have, for instance, been sending helicopters over affluent neighborhoods looking for swimming pools, as evidence of underreported wealth. These efforts have made some difference: the self-employed seem to be reporting more of their income, and the evaders have had to step up their game. (There’s now a burgeoning market in camouflage swimming-pool covers.)

Perhaps there's a marketing opportunity here...


Chamelion XLE - Watch more Funny Videos