Comments, observations and thoughts from two bloggers on applied statistics, higher education and epidemiology. Joseph is an associate professor. Mark is a professional statistician and former math teacher.
Tuesday, February 8, 2011
It has reached the point where not only do sitcoms portray Ponzi schemes...
Monday, February 7, 2011
There might just be a lesson here somewhere.
As for HuffPo, it gets lots of money, great tech content from Engadget and TechCrunch, hugely valuable video-production abilities, a local infrastructure in Patch, lots of money, a public stock-market listing with which to make fill-in acquisitions and incentivize employees with options, a massive leg up in terms of reaching the older and more conservative Web 1.0 audience and did I mention the lots of money?I used to work for Earthlink. It was a great company, customer-centered, innovative, but it was trapped between AOL and Microsoft's Online, two companies with immeasurably deep pockets and no apparent concern about turning a profit.
AOL in particular was a mystery to us. It went from one bad business plan to another, all while maintaining spectacularly bad customer service. None of it mattered in the face of hundreds of millions of promotional disks.*
* According to former Chief Marketing Officer at AOL, Jan Brandt, "At one point, 50% of the CD's produced worldwide had an AOL logo on it."
Well-mannered spam
Then yesterday Lily was back with another comment, or, more accurately, the same comment, this time on another post. There was no obvious common thread to the two posts, no apparent reason that anyone would single them out. All of this made me, in equal parts, suspicious and curious, so I clicked on the profile where I found a picture of a very attractive young Asian woman and a long list of blogs that, according to Google translate, offer a wide range of questionable services.
Profile-based spam is a new one on me and I wonder how the filters are going to handle it. On the bright side, though, at least it's polite.
More on poverty and education
"I do know that I get anxious whenever a correlation analysis tries to look like a causal analysis. A frequent scenario introduces an outcome (test performance) with a highly correlated predictor (say poverty), and suggests that reducing poverty will improve the outcome. The problem is that poverty is correlated with a number of other predictors."
I think that the dense correlation is precisely the point that makes inference challenging here. I like to point out California as being a good example of a puzzling outlier in the original study. There is no a priori reason to expect that California would do as poorly as it does relative to other states. But different groups have different explanations.
One hypothesis is that "teacher tenure" leaves weak performers in the classroom for decades. This seems to be the position of people like Michelle Rhee.
Another hypothesis is low educational spending in the state. According to wikipedia, "In teaching staff expenditure per pupil, California ranked 49th of 51".
A third hypothesis is the composition of people in California just leads to lower educational performance (due to innate differences with, for example, the population of New York state).
I kind of suspect the second hypothesis as most likely, but, given how inter-related things are it is unclear how you would do inference here. And some forms of the first hypothesis would lead to policies that could make things worse if the second were true. The third would suggest no policy is likely to make an important difference.
Coming up with clever ways to distinguish between these hypotheses is not trivial.
Speaking of epicycles...
The paper argues that welfare programs expand the set of choices available to people; while that is all good according to traditional economics, if we think that people are inclined to make bad choices (“behavioral economics”), then welfare programs give people more opportunity to make bad choices and hurt themselves. This is particularly a problem because, they claim, “there are good empirical reasons to think that behavioral economics better describes the poor than it does the rest of the population” (p. 4). In other words, if poor people are more irrational, then giving them more choices will hurt them more than other people.Kwak takes it all apart, from the odd and strangely uninformed take on behavioral economics to the lack of supporting data to the ready supply of superior alternative hypotheses to the generally poor quality of the authors' reasoning. If this zombie claws its way out of the grave after Kwak is through with it then there's just no killing the damned thing.
One thing that is obvious from this paper is that Caplan and Beaulier have reached the epicycle stage.* They are no longer focused on finding the best model to fit the data; instead they are trying to salvage an intellectual framework that is based on elegant principles and appealing ideas like just desserts and efficient markets.
The shift from serving the data to serving the theory is easy to make and difficult to catch. Every new theory occasionally runs into a phenomena it has trouble explaining. Explaining them can be a good thing. The process of taking a theory into the counter-intuitive is an important, even necessary step in its evolution.
There is, however, one condition: the explanation you finally come up has got to be as good or better than any of the alternative hypotheses. Otherwise, the theory isn't evolving; it's decaying.
* The Wikipedia article on the subject suggests I'm being unfair to Ptolemy. I hope not -- I'd hate to have to come up with a new analogy.
Damn you, Paul Krugman!
Nobel Prize winners who work on Saturday are just showing off.
You can, if you’re desperate, try to explain this away by saying that there was some fundamental structural change in the early 1970s, but at that point you’re deep into epicycle territory. And there’s more — for example, Ireland went abruptly from having a stable real exchange rate against the UK to having a stable rate against Germany when it joined the European exchange rate mechanism, etc..
[Emphasis added.]
Sunday, February 6, 2011
Employment in the US
Education and Poverty
This was very clear in these posts by Andrew Gelman and Jonathan Livengood on education and poverty. There is an impressive inverse association between poverty and educational test scores. It's pretty clear that there could be a common cause (i.e. a confounder) or that the direction of the causal association could go in either direction.
However, the policy decisions are very different depending on the causal relations involved. If poor schooling is a cause of poverty then it makes sense to focus massive efforts on educational reform. But if poor educational outcomes are caused by poverty then this suggests a completely different approach. And, of course, finding a confounder is also a key point.
Now, this assumes that all parties are acting in good faith. If there was an interest in keeping outcomes poor in MS and CA (to pick two low performing states) then educational reforms that worsened outcomes might make sense. This would be a good strategy if one wanted to continue to use "poor educational outcomes" as a talking point as reforms take decades to really take effect.
Finally, one has to worry that real issues (poor educational outcomes) might be used to implement ideological views (i.e. a dislike of unions) when such reforms might be orthogonal to the issues at hand. This wastes the effort of reform without addressing the underlying issues.
Now, none of this is to accuse any group of bad faith. But it is important to realize that there are multiple viable hypotheses as to the causes of poor educational outcomes. The state that I am the most interested in is California; they seem to have very poor outcomes for a large state with a diverse (and often knowledge-based) economy. If that is a reflection of resources expended then maybe we need to focus on counter-intuitive approaches (like raising taxes to improve educational funding) if we really want to make a difference.
Going full circle in pop culture hell
Media celebrities like Oprah Winfrey, Glenn Beck and the hosts of “Morning Joe” have all raved about the musical, especially Mr. Beck, who said in an interview on Friday that he had seen it four times.
Mr. Beck has framed its appeal on his radio broadcast as a face-off between regular Americans and cultural snobs (i.e., liberals). In the interview, however, he was more fanboy than fire breather, rattling off plot points and design elements with the practiced eye of a Sardi’s regular.
“The story line is right on the money for today, which is to be your better self, that you can spiral into darkness or — ” here he quoted one of the show’s anthemic songs — “you can rise above,” said Mr. Beck, who estimated that he sees a dozen shows a year. “In fact, I just wrote an e-mail to Julie” — Ms. Taymor — “about how much I loved the new ending.
Though it's probably just a coincidence, Mr. Beck had quite a bit in common with Spider-Man's co-creator, Steve Ditko, at least when it comes to politics. Ditko was a devoted objectivist and was greatly influenced by Ayn Rand, particularly The Fountainhead and its persecuted hero. You could catch some glimpse of this in the Marvel comic, but it wasn't until Ditko left the company in the late Sixties that he was able to fully express his philosophy in a series of comics that really have to be seen to be believed.
So for fans of Steve Ditko, students of objectivism and those of you with a morbid sense of curiosity, enjoy.
Soon to be relevant quote
There are two novels that can change a bookish fourteen-year old's life: The Lord of the Rings and Atlas Shrugged. One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes, leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world. The other, of course, involves orcs.Kung Fu Monkey via Prof. Krugman.
EDIT (by Joseph): The original source is Raj Patel's "The value of nothing"
While on the subject of checkers...
Saturday, February 5, 2011
Weekend Gaming -- Behold the humble checker
Yet to calculate is not in itself to analyze. A chess-player, for example, does the one without effort at the other. It follows that the game of chess, in its effects upon mental character, is greatly misunderstood. I am not now writing a treatise, but simply prefacing a somewhat peculiar narrative by observations very much at random; I will, therefore, take occasion to assert that the higher powers of the reflective intellect are more decidedly and more usefully tasked by the unostentatious game of draughts than by all the elaborate frivolity of chess. In this latter, where the pieces have different and bizarre motions, with various and variable values, what is only complex is mistaken (a not unusual error) for what is profound. The attention is here called powerfully into play. If it flag for an instant, an oversight is committed, resulting in injury or defeat. The possible moves being not only manifold but involute, the chances of such oversights are multiplied; and in nine cases out of ten it is the more concentrative rather than the more acute player who conquers. In draughts, on the contrary, where the moves are unique and have but little variation, the probabilities of inadvertence are diminished, and the mere attention being left comparatively what advantages are obtained by either party are obtained by superior acumen.Edgar Allan Poe -- "The Murders in the Rue Morgue"
Poe's opinion on this matter is more common than you might expect. It's not unusual to hear masters of both chess and checkers (draughts) to admit that they prefer the latter. So why does chess get all the respect? Why do you never see a criminal mastermind or a Bond villain playing in a checkers tournament?
Part of the problem is that we learn the game as children so we tend to think of it as a children's game. We focus on how simple the rules are and miss how much complexity and subtlety you can get out of those rules.
Chess derives most of its complexity through differentiated pieces; with checkers the complexity comes from the interaction between pieces. The result is a series of elegant graph problems where the viable paths change with each move of your opponent. To draw an analogy with chess, imagine if moving your knight could allow your opponent's bishop to move like a rook. Add to that the potential for traps and manipulation that come with forced capture and you have one of the most remarkable games of all time.
There have been any number of checkers variants.* You could even argue that all checkers games are variants since, unlike chess, there is no single, internationally recognized version. Here are some of the best and best-known.
Spanish Checkers
Considered by many connoisseurs to be the best of the national variants. It is distinguished from the American version by a queen's (analogous to a king) ability to jump long. This makes a queen powerful but, since captures are mandatory, easier to capture.
Dama (Turkish Checkers)
Also known as Greek checkers. In this variant, pieces move vertically and horizontally instead of diagonally. Among other things this doubles the playing field.
Lasca
This variant was invented by the second World Chess Champion Emanuel Lasker. It's worth noting that Lasker (who is considered one of the best chess players of all time) would look to checkers when designing his own game.
In Lasca, captured pieces are added to the bottom of columns controlled by whichever player has the top piece. Since jumping only captures the piece on top of the column, players have to take into account not only position but also height and position.
You can get a detailed write-up by following the link or you can see Lasker's original patent application here. He comes off as rather immodest, but he was the world's best chess player and a close friend of Einstein so I guess we can let him slide.
Misère checkers
This one you probably know as suicide or give-away checkers. The object is either to lose all your pieces or have them blocked so that you can't move.
Endless Checkers (or whatever the damned thing is called)
I'm certain I've read about this somewhere though I can't seem to find any record of it. Here, when a piece reaches the last it can move to an appropriate space on the first row (think of a board wrapped around a cylinder). There are no kings in endless checkers.
I have a feeling I'm missing something. Any suggestions?
* Including one I'm particularly fond of, but that's a topic for another post.