Good piece on education funding

Kevin Carey has the details at the New Republic.

This is great news

I was listening to Marketplace on my way to the gym this evening and I heard the following:

Kai Ryssdal: You've seen them at gas stations and at the gym probably. Maybe even in the elevators at work. Small television screens showing news and weather, and more often than not, a whole lot of ads. But really, how many people do you suppose are actually paying attention to those things? Nielsen, the company behind the television rating system, well, they know. They released their first report for those little screens today.

...

Nielsen tracked the impact of ads you see when you leave the house. But how does the company know when someone running on the treadmill is actually paying attention to what's on the screen in front of her?

BRENNAN: We actually measured the proximity to the machines and that they were actually on the machine and watched the content. And then we asked them to recall what it was that they watched and what they remembered.

Turns out those ads you see when you're exercising sink in.

BOB MARTIN: This report shows that what a quarter billion impressions are generated every month.

Bob Martin is chief marketing officer at RMG Networks. It's one of the 10 networks, including Gas Station TV and the Hotel Networks, that Nielsen included in this report. Martin says the new measurements could help his network sell more ads.

Why is this great news? Because my gym keeps its channels tuned to three stations, none of which I like. If they put screens on the bikes, I can stop watching videos on my really small $24 media player.

The Complexity of the Commons

This passage from an interesting interview with Nobel Prize winner Elinor Ostrom has been making the rounds. (I picked it up from Mark Thoma). Take a look:

Fran: Do you have a favorite example of where people have been able to self-organize to manage property in common?

Elinor: One that I read early on that just unglued me—because I wasn’t expecting it—was the work of Robert Netting, an anthropologist who had been studying the alpine commons for a very long time. He studied Swiss peasants and then studied in Africa too. He was quite disturbed that people were saying that Africans were primitive because they used common property so frequently and they didn’t know about the benefits of private property. The implication was we’ve got to impose private property rules on them. Netting said, “Are the Swiss peasants stupid? They use common property also.”

Let’s think about this a bit. In the valleys, they use private property, while up in the alpine areas, they use common property. So the same people know about private property and common property, but they choose to use common property for the alpine areas. Why? Well, the alpine areas are what Netting calls “spotty.” The rainfall is high in one section one year, and the snow is great, and it’s rich. But the other parts of the area are dry. Now if you put fences up for private property, then Smith’s got great grass one year—he can’t even use it all—and Brown doesn’t have any. So, Netting argued, there are places where it makes sense to have an open pasture rather than a closed one. Then he gives you a very good idea of the wide diversity of the particular rules that people have used for managing that common land.

Fran: Why were Netting’s findings so surprising to you?

Elinor: I had grown up thinking that land was something that would always move to private property. I had done my dissertation on groundwater in California, so I was familiar with the management of water as a commons. But when I read Netting, I realized that when there are “spotty” land environments, it really doesn’t make sense to put up fences and have small private plots.

Mixed messages

From Valuing Another Degree by Jonnelle Marte

These days, however, as employers continue to cut spending, expectations for such higher pay could backfire when you're competing against less educated -- and less expensive -- candidates.

For example, entry-level teachers with master's degrees often have a harder time getting hired than those with bachelor's degrees because schools typically pay more to teachers with master's degrees, says Steven Rothberg, founder of CollegeRecruiter.com.

One of the best teacher's I've known was the wife of an army officer who had to move every few years. Despite her outstanding resume and glowing references, she had a hard time getting jobs because her years of experience made her too expensive.

This is, after all, a write in haste-repent at leisure medium

[make sure to check out the comments section]

There's a common type of blog post that consists of sarcastic dismissal of a statement followed by critical statements. Though they tend to be mean, they can be fun read and, if the criticisms are valid, can add quite a bit to the conversation.

The main problem with this type of post is that has an ugly habit of devolving into just a sarcastic dismissal and those devolved posts are not so defensible (even if I'm the one writing them). This is not only mean but cowardly. By leaving out the reasons behind your sarcasm you deny your subjects the chance to defend themselves.

A couple of days ago, I made a sarcastic comment about this passage from the New York Times. Here is the quote:

The road between the two cultures — science and literature — can go both ways. “Fiction provides a new perspective on what happens in evolution,” said William Flesch, a professor of English at Brandeis University.

To Mr. Flesch fictional accounts help explain how altruism evolved despite our selfish genes. Fictional heroes are what he calls “altruistic punishers,” people who right wrongs even if they personally have nothing to gain. “To give us an incentive to monitor and ensure cooperation, nature endows us with a pleasing sense of outrage” at cheaters, and delight when they are punished, Mr. Flesch argues. We enjoy fiction because it is teeming with altruistic punishers: Odysseus, Don Quixote, Hamlet, Hercule Poirot.

And here are comments to my post by Prof. Flesch.

My problems with Prof. Flesch's statement and more importantly with the NYT article are as follows.

Altruistic punishment has become a bit of a flavor of the month, one of those fashionable ideas that people like to work into conversations and op-ed pieces. Unfortunately this often leads to the ideas being applied where they don't fit or being used as labels for related ideas. This can (and, I would argue, should) prompt greater scrutiny and that scrutiny should always start with the question, is the concept being used correctly?

What are the requirements of altruistic punishment?

The punisher has to proceed knowing that not punishing the offender would produce a better outcome for the punisher. Outcome here needs to be broadly defined to include not only the material but also reproductive and reputational – the loan shark who has a defaulter killed is not engaged in altruistic punishment even though he probably would have collected more from a living customer; he is investing in a reputation that will make future defaults less likely.

To make sure not to confound the problem, researchers often focus on unrelated strangers with whom no future encounters are anticipated.

Altruistic punishment is normally discussed in the context of the evolution of large-scale cooperation. "Unlike other creatures, people frequently cooperate with genetically unrelated strangers, often in large groups, with people they will never meet again, and when reputation gains are small or absent." (Fehr and Gächter) The behavior suggests that individuals are biologically programmed to want to enforce the rules that make complex societies possible.

With all that out of the way, can we reasonably say that Odysseus (or, for that matter, Hamlet or Poirot) is an altruistic punisher?

The mythic world of the Odyssey was small and greatly concerned with reputation and social standing. It was clearly in Odysseus' interest to be seen as a dangerous and vengeful opponent. Acts that enhance that reputation can't really be considered altruistic punishment even if (as with Polyphemus and his father) the acts turn out to be more costly than expected.

It is even more difficult to make the case for altruistic punishment in the killing of the suitors. Here we have a group that has threatened to take his property (material), marry his wife and kill his son (genetic) and humiliate him in the process (reputational). It is the trifecta of self-interest. There was a high potential cost to this act but that alone is not sufficient to make it altruistic punishment. (besides, the consequences were based not only on the deaths of the suitors but also on the loss to those who accompanied him to Troy. He might well have faced an angry mob if he had simply told the suitors to go home.)

I admit my knowledge of the Odyssey is weak and I may well have missed some important aspect that supports the hypothesis that Odysseus was an altruistic punisher and not simply someone who pursued great goals at great costs, but they aren't obvious to the casual reader.

As for the other examples, Don Quixote probably does qualify. Hamlet is more problematic. He is acting in the interest of his father which brings in a genetic component (I'm not sure what the evolutionary psychology implications are of his father being a ghost). There's also a self-preservation aspect (perhaps more obvious in the Amleth version). As for Poirot, both his livelihood and his reputation rely on his ability to solve crimes. There are cases (most notably Curtain) where his goal is clearly altruistic punishment, but in most of his cases there is a clear element of self-interest. You could, however, make a pretty good argument for Lord Peter Whimsey.

And there are certainly examples of altruistic punishment out there. The Count of Monte Cristo is a good fit since there is no apparent material or reproductive gain and most of the revenge is carried out anonymously. Crime fiction also provides numerous protagonists (Lew Archer, Travis McGee, Matt Scudder) who take cases from strangers then continue to pursue them after their clients had stopped paying them. In one book, Scudder went so far as to make a suicide look like a murder then frame the man who was morally responsible for the death.

We could go on (anyone care to make the case for Miss Havisham?) but I don't know what it would accomplish. Like every other imaginable human behavior, altruistic punishment occurs in literature. We can take that as a not very interesting given.

There are cases where analyzing a work in terms of evolutionary psychology can provide real and unique insights. All too often, though, when an idea like altruistic punishment becomes fashionable, juxtaposition often replaces analysis. References to trendy terms like market forces or the prisoner's dilemma are stuck into arguments where they contribute little and often don't even apply (such as trying to use the prisoner's dilemma to describe a stag hunt).

We should always welcome new ideas and cross-pollination, but when we see a headline in the New York Times that starts with the words "Next Big Thing," it's probably a time to be a bit more critical.

(and if I had an issue with an New York Times article, I suppose I should have done more than slam one quote.)

And you thought the academic job market was tough

[I have a longer and hopefully fairer follow-up here. Make sure to Flesch's comment which puts his quote in context and largely invalidates my criticism.]

This New York Times article (via Cheap Talk) shows that you can get to be an English professor without ever reading the Odyssey (or even skimming the Cliff Notes):

The road between the two cultures — science and literature — can go both ways. “Fiction provides a new perspective on what happens in evolution,” said William Flesch, a professor of English at Brandeis University.

To Mr. Flesch fictional accounts help explain how altruism evolved despite our selfish genes. Fictional heroes are what he calls “altruistic punishers,” people who right wrongs even if they personally have nothing to gain. “To give us an incentive to monitor and ensure cooperation, nature endows us with a pleasing sense of outrage” at cheaters, and delight when they are punished, Mr. Flesch argues. We enjoy fiction because it is teeming with altruistic punishers: Odysseus, Don Quixote, Hamlet, Hercule Poirot.

Chait on careers in journalism

I had some posts up earlier (see here and here) on how economic changes had affected the career options for writers of fiction. Over at The New Republic, Jonathan Chait looks at old and new career paths for writers on the nonfiction side. His comments on columnists are particularly interesting:

The traditional ladder is even more problematic for columnists. Calderone notes, "a Washington column has traditionally been the reward at the end of a climb up the journalistic ladder, with stops along the way at small-town papers, medium-sized city desks and local TV newsrooms." Whereas the small town beat is merely unnecessary as a career chokepoint for national reporters, it's actually counterproductive for columnists. Writing opinion about politics and public policy is a very different skill than reporting. The almost-uniform rule of political reporters-turned-columnists is that they're awful at it. They have no idea how to construct a persuasive argument or marshal the kinds of evidence they need to make their case. Usually, their argument relies on authority -- I am asserting opinion X and you should believe me because I am a prestigious veteran reporter.

The practice of training people for one kind of work and then shifting them into something that requires a completely different set of skills is one of the more bizarre habits of the traditional journalism world. If the New York Times approached me and said that I've done a good job as a columnist and blogger for TNR, and now I should start covering the city hall beat for them, it would be nuts. I'd be horrible.

Statins in healthy individuals

This new york times article was quite worthy of thinking about. In particular, it's still not clear to me that C-reactive protein is the best possible marker of high cardiovascular risk. Nor, if the primary mechanism is inflammation, why conventional anti-inflammatory medications like aspirin are not the best therpuetic choice. Statins also lower blood pressure but they are clearly not the best anti-hypertensive medication option for healthy, low risk patients.

It is a pretty large public health experiment and I can only hope that the benefits will exceed any potential side effects of medication use.

Outsourcing English papers

Confessions of a Community College Dean has a good detailed post up the practice of off-shoring grading:

I don't buy the 'quality' argument against it, either. If radiologists in India can read images, and programmers in India can work on developing and fixing incredibly sophisticated software, then surely some smart folks in India can handle some freshman comp papers. Seriously. Other information-based industries have endured outsourcing without the quality of the work suffering. Given the inarguable indifference with which our large universities have handled undergraduate teaching for so long, to suddenly get huffy and puffy about standards is disingenuous at best.

The simple truth of the matter is that universities have engaged in a bait-and-switch with intro undergraduate classes for a long time, and have built an entire economic model on it. This may be a case in which following the model to its logical conclusion actually prompts looking more closely at the entire enterprise, which strikes me as a good idea. In the meantime, we'll keep running small classes with real faculty who actually do bond with their students, and doing it at a fraction of the cost. If folks would sneer at us a bit less, I'd be much obliged.

I got my undergraduate degrees (one in creative writing, one in math, but that's a story for another time) from a small four-year school. With a handful of exceptions, my professors were experienced, tenured teaching professionals who took undergraduate education and mentoring every bit as seriously as professors at big universities take their research.

A few years later I did stint at the state's big university, teaching undergrad courses and helping supervise the teaching assistants. I came away from that experience with the conviction that undergraduate education at most large universities is basically a scam, tricking students into buying a third-rate product with the lure of a big name and prestigious faculty members whom most of the students will never even see let alone study under.

Does this sound right?

From David Brooks (via a disbelieving Brad Delong):

Over the last 10 years, 60 percent of Americans made more than $100,000 in at least one of those years, and 40 percent had incomes that high for at least three.

Positive Reviews

Have you ever reviewed a paper and thought "wow, this paper is excellent". It's well written, relevant and obviously crafted with care. Well, know I have.

My question is what do you do with it? Do you find items to nitpick so you can show that you were rigorous or do you simply recommend that the paper proceed to publication?

Thoughts?

“Gives new meaning to company town.”

From the Asilomar International Conference on Climate Intervention Technologies reported by Jeff Goodell and brought to our attention by Brad Plumer:

It was generally agreed that for CO2-sucking technologies, private investment was not a problem [n.b., assuming we have some sort of cap-and-trade system in place]. Sunlight-reduction technologies, however, are another issue. if some company (or entrepreneur) is able to develop a new way of injecting particles into the stratosphere that becomes indispensible to the survival of the human race, well, that gives that company or person a lot of leverage.

“I’m not interested in selling my soul to some company who is going to control how much sunlight hits the planet,” said Phil Rasch, a climate modeler at the Pacific Northwest National Laboratory in Washington state. (As one audience member quipped, “Gives new meaning to company town.”) Granger Morgan, the head of the department of engineering and public policy at Carnegie Mellon University, argued that the creation of a profit motive would inevitably lead to a geoengineering lobby: “Lobbying is the last thing we need on this.”

I know there's a Schrödinger's cat joke here somewhere

Andrew Gelman has an excellent post up on the contradictory findings of two teams of researchers on the effect child-gender has on parents' political leanings.

Tyler Cowen reports the following claim from sociologists Dalton Conley and Emily Rauscher:

Using nationally-representative data from the [1994] General Social Survey, we [Conley and Rauscher] find that female offspring induce more conservative political identification. We hypothesize that this results from the change in reproductive fitness strategy that daughters may evince.

This surprised me, because less than a year ago, we reported here on a study by economists Andrew Oswald and Nattavudh Powdthavee with the exact opposite finding:

We [Oswald and Powdthavee] document evidence that having daughters leads people to be more sympathetic to left-wing parties. Giving birth to sons, by contrast, seems to make people more likely to vote for a right-wing party. Our data, which are primarily from Great Britain, are longitudinal. We also report corroborative results for a German panel.

Understanding the results (possibly) in terms of "family values"

This is a fun problem: we have two different studies, both by reputable researchers, with opposite results! I took a look at both papers and can't immediately see a resolution, but I will offer some speculations, followed by some scattered comments.

Andrew speculates that the differences in the findings can be explained by the fact that one study looked at the U.S. while the other looked at Britain with some additional data taken from Germany. Commenter DN elaborated further:

Seems to me that this is more likely a "law and order" effect in the US. Families with daughters are more likely to be concerned about 'protecting' their daughters against violent crime. These impacts are either mitigated in Germany and the UK (because of perceptions of the parties) or swamped by other perceived advantages of left-wing parties.

This brings up one of my least favourite practices in bad statistics reporting: generalizing conclusions about attitudes drawn from a specific culture or social group. One example that stayed with me (reported by the ever credulous NYT under the headline Bicycle Helmets Put You at Risk) was that of Ian Walker, a psychiatrist at the University of Bath. Walker, an opponent of helmet laws, put a sensor on his bike and rode with and without a helmet until he had been passed 2,500 times (see the curse of large numbers). To control for potential gender effects he sometimes donned a long wig (to get the full comic effect, check out Walker's picture below).

Walker found vehicles came on average 3.35 inches closer when he was wearing a helmet (for context, the average passing clearance was over four feet).

Putting aside for the moment questions about the methodology of the study and the sweeping conclusions Walker from it, the New York Times article works under the implicit assumption that despite major differences in traffic laws, road conditions, driver etiquette and education, vehicle type and biking culture, findings from that small stretch of English road are equally applicable to American highways.

Subtle issues

This recent study in Pharmacoepidemiolgy and Drug Safety suggests that long term warfarin use leads to more aggressive cancer (as opposed to never users). However, the same authors have shown that warfarin reduces the incidence of prostate cancer in this population.

Is it not just as likely that the warfarin use is more effective versus less agressive cancers as a mildly anti-cancer agent?

I'm curious as to why this interpretation of the data was not pursued

The Long Goodbye -- the networks are taking a long damned time to die

Yesterday I posted a link to Clay Shirky's essay comparing the big media empires to societies like the Roman Empire that collapsed under the weight of their own complexity. I commented that I doubted his conclusions. Here's one of the reasons why.

Back in the early Nineties, when I was still picking up some extra money as a freelance writer, I did a long feature on the surprising vitality of broadcast television in Northwest Arkansas. The vitality was surprising because broadcast television had been under a death watch for more than a decade.

These ominous prognoses had started started sometime in the mid to late Seventies, prompted by the arrival of home video and the rise of cable and satellite-based cable stations. By the beginning of the Eighties, most observers agreed that the outlook was dire. At least one columnist predicted that one of the networks (his guess was NBC) would be gone by the end of the decade. (The columnist obviously didn't have a lot of faith in that Tartikoff kid the network had just hired.)

By 1990 there were exactly twice as many networks as the columnist had predicted and one more than anyone had expected. What happened? Entertainment executives had been trying to launch a fourth network since DuMont went dark in 1956. Why was an industry facing an explosion of new competition suddenly able to support a major new player?

What industry watchers were overlooking was that broadcast stations fell into two distinct groups: VHF and UHF. For the VHF stations, cable really was a threat to a virtual monopoly. it marginally improved their picture quality but it brought in numerous new channels. UHF stations, however, had poor picture quality and low range. As businesses, they were only viable in large, heavily populated areas (and then only marginally). With cable, though, a channel 39 could compete on an equal footing with a channel 2.

This was what made Fox and later the CW viable businesses.

But to me the most remarkable thing about this story is the way the standard narrative has changed so little over the decades. For more than thirty years, there has been a steady stream of stories about reduced schedules, mergers, and deaths of networks and with the exception of the creation of the CW, every single one of them has been incorrect. Over three decades of wrong.

Just to put this in perspective, ABC, the last of the big three, started broadcasting in 1948. That was also about that the other networks started moving past the experimental stage. If we pick that as our starting point, network television has spent more than half of its life under a prognosis of imminent death. During that time the number of networks has gone from three to four to five (and might have been six if not not for some disastrous strategic errors by the heads of WB and UPN) and CBS, the network that reacted the least to the changes in the market (sticking mainly with sitcoms and hour dramas and keeping a very small web presence) is arguably the healthiest of the bunch.

Of course, this could all change in the next year or so. There is no reason to assume the current networks are any more immortal than DuMont was. All the same, we've been told since the late Seventies that this section of the sky is just about to fall. Perhaps it's time to question Mr. Little's credibility.

All others things being equal

Regression analysis is often interpreted as being the effect of changing one variable while holding all other factors constant. Sadly, when working with complex human behaviors, like in nutritional epidemiology, factors rarely stay constant. A change in one parameter can shift other parameters making inference difficult. A very good example of this was reviewed by Travis Saunders of Obesity Panacea.

Now obviously randomized experiments allow an estimate of the average causal effect of an intervention but they are expensive to run and these types of research questions raise grave issues of equiposie and adherence in the design of these experiments!

Decline and Fall of the Media Empires?

Clay Shirky has a post up comparing the current state of the big media companies to the declining days of the Romans, the Lowlands Maya, the inhabitants of Chaco canyon. I think he gets a lot wrong (hell, I think he gets most of it wrong), but he gets it wrong in a really interesting way.

Give it a look:

Bureaucracies temporarily reverse the Second Law of Thermodynamics. In a bureaucracy, it’s easier to make a process more complex than to make it simpler, and easier to create a new burden than kill an old one.

In spring of 2007, the web video comedy In the Motherhood made the move to TV. In the Motherhood started online as a series of short videos, with viewers contributing funny stories from their own lives and voting on their favorites. This tactic generated good ideas at low cost as well as endearing the show to its viewers; the show’s tag line was “By Moms, For Moms, About Moms.”

The move to TV was an affirmation of this technique; when ABC launched the public forum for the new TV version, they told users their input “might just become inspiration for a story by the writers.”

Or it might not. Once the show moved to television, the Writers Guild of America got involved. They were OK with For and About Moms, but By Moms violated Guild rules. The producers tried to negotiate, to no avail, so the idea of audience engagement was canned (as was In the Motherhood itself some months later, after failing to engage viewers as the web version had).

The critical fact about this negotiation wasn’t about the mothers, or their stories, or how those stories might be used. The critical fact was that the negotiation took place in the grid of the television industry, between entities incorporated around a 20th century business logic, and entirely within invented constraints. At no point did the negotiation about audience involvement hinge on the question “Would this be an interesting thing to try?”

Business models and secret sauces

As mentioned in the last post, today's New York Times has an article of the good people at Talx. One thing that caught my eye was this account of the company's rise:

Talx entered the field brashly, buying the industry’s two largest companies on a single day in 2002. In the next few years, it bought five more. Until then, Talx had never handled an unemployment claim, and skeptics wondered how well it could blend seven companies in an unfamiliar industry.

The Federal Trade Commission argued in a 2008 antitrust complaint that the acquisitions, which cost $230 million, had allowed Talx to “raise prices unilaterally” and “decrease the quality of services.” Talx modified some contracts to settle the case, but admitted no legal violations.

Financially, the gamble paid off: Talx was acquired three years ago by Equifax, the credit-rating giant, for $1.4 billion. But work once done locally became centralized — at a loss, critics say, of responsiveness and expertise.

If this article had been written in 2007*, I suspect the proportions would have been reversed, with two or three paragraphs on complaints and ethical lapses and the rest focused on the company's rise to dominance. Shortly before the collapse, even sober institutions like the New York Times were more inclined to focus on success and less inclined to pay attention to the man behind the curtain.

These success stories were filled with wide-eyed descriptions of innovative business plans and nimble management. There were often vague, portentous references to sophisticated analytic tools and powerful proprietary models and algorithms that could only be appreciated by a handful of Nobel Prize winners.

But when you looked closely at these companies or, better yet, sat down and talked with the people who actually came up with this stuff, you usually found yourself in the position of the trainees at a fast food restaurant in Fast Times at Ridgemont High who learned that the mysterious secret sauces of two rival chains were, respectively, thousand island dressing and ketchup & mayonnaise.

On close inspection, most innovations and business models, including the heavily hyped ones, turn out to be mundane and unimpressive: switched to cheaper suppliers, dropped expensive features or services, introduced some basic statistical tool like logistic regression, marketed a high-fee product to an underserved segment. More than a few are based on creative accounting or a cozy relationship with various regulatory agencies.

Talx' strategy was based on two simple components:

1. Most big companies will be reluctant to work with a small companies so if you buy up most of your competition you can set prices like a monopoly;

2. The penalties for breaking the law are small and the penalties for bending it are negligible relative to the potential benefits.

Neither idea was profound. Neither required a team of Ph.D.s for implementation. They were, however, wildly successful for Talx, just as they have been for any number of companies with much better P.R. departments.



* Very few were. The company doesn't even have a Wikipedia page.

Sleaziness as a business model -- first of two posts on Talx

Contesting Jobless Claims Becomes a Boom Industry

By JASON DePARLE

WASHINGTON — With a client list that reads like a roster of Fortune 500 firms, a little-known company with an odd name, the Talx Corporation, has come to dominate a thriving industry: helping employers process — and fight — unemployment claims.

Talx, which emerged from obscurity over the last eight years, says it handles more than 30 percent of the nation’s requests for jobless benefits. Pledging to save employers money in part by contesting claims, Talx helps them decide which applications to resist and how to mount effective appeals.

The work has made Talx a boom business in a bust economy, but critics say the company has undermined a crucial safety net. Officials in a number of states have called Talx a chronic source of error and delay. Advocates for the unemployed say the company seeks to keep jobless workers from collecting benefits.

“Talx often files appeals regardless of merits,” said Jonathan P. Baird, a lawyer at New Hampshire Legal Assistance. “It’s sort of a war of attrition. If you appeal a certain percentage of cases, there are going to be those workers who give up.”

When fewer former workers get aid, a company pays lower unemployment taxes.

Wisconsin and Iowa passed laws to curtail procedural abuses that officials said were common in cases handled by Talx. Connecticut fined Talx (pronounced talks) and demanded an end to baseless appeals. New York, without naming Talx, instructed the Labor Department staff to side with workers in cases that simply pit their word against those of agents for employers.

...

Advocates for the unemployed cite cases like that of Gerald Grenier, 47, who spent four years as a night janitor at a New Hampshire Wal-Mart and was fired for pocketing several dollars in coins from a vending machine. Mr. Grenier, who is mentally disabled, told Wal-Mart he forgot to turn in the change. Talx, representing Wal-Mart, accused him of misconduct and fought his unemployment claim.

After Mr. Grenier waited three months for a hearing, Wal-Mart did not appear. A Talx agent joined by phone, then seemingly hung up as Mr. Grenier testified. The hearing officer redialed and left an unanswered message on the agent’s voice mail. The officer called Mr. Grenier “completely credible” and granted him benefits.

Talx appealed, claiming that the officer had denied the agent’s request to let Wal-Mart testify by phone. (A recording of the hearing contains no such request.) Mr. Grenier won the appeal, but by then he had lost his apartment and moved in with his sister.

“That was a nightmare,” he said.

In the case of Dina Griess, Talx and its client, the subprime lender Countrywide Financial, were involved in what a judge deemed an outright fraud. Ms. Griess worked for Countrywide outside Boston and quit as it collapsed in 2008, saying she was distressed by internal investigations of lending practices. People can receive unemployment benefits if they quit for “good cause,” like unsafe working conditions, but Talx argued that Ms. Griess’s reason did not meet the legal standard.

She won benefits at a hearing that Talx and Countrywide skipped, but Talx successfully appealed, saying the Countrywide witness had missed the hearing because of a family death. Later asked under oath if that was true, the witness said, “No, it’s not.”

A Massachusetts judge reviewing the case, Robert A. Cornetta of Salem District Court, denounced the deceit and gave Ms. Griess benefits. “The court will not be party to a fraud,” he said.

[click here for a history of complaints -- Mark]

Statistics is common sense raised to a science

And Nate Silver is a very good statistician.

From Five Thirty-eight:

That de Rugy has testified before Congress on the basis of her evidence, and never paused to consider why the top five congressional districts on her list overlap with Sacramento, Albany, Austin, Tallahassee and Harrisburg, is mind-boggling. The presence of a state capital is the overwhelmingly dominant factor it predicting the dispensation of stimulus funds. This could have been discerned in literally five minutes if she had bothered to look at the apparent outliers in her dataset and considered whether they had anything in common — a practice that should be among the first things that any researcher does when evaluating any dataset.

Chait has some nice comments on the exchange here.

More on social norming

Rajiv Sethi picks up the ball from Freakonomics and runs with it:

This question lies at the heart of the lifelong work of Elinor Ostrom, co-recipient of the 2009 Nobel Memorial Prize in Economics, whose contributions I discussed in in a couple of earlier posts. Using an eclectic mix of methodological approaches, including case studies, laboratory experiments, and game theoretic models, Ostrom managed to overturn conventional wisdom regarding the "tragedy of the commons." She demonstrated the possibility of self-governance when a well-defined group of users with collective rights to an economically valuable resource were at liberty to develop their own rules, and to ostracize, expel, or otherwise sanction each other for violations.

While Ostrom's focus was on natural resources such as forests, fisheries, and pastures, her basic insights have more general relevance. For instance, institutions of self-governance are critically important in the case of urban communities that lie largely outside the reach of the formal legal system in the United States. There are parts of the country where residents do not have recourse to the courts to adjudicate contractual and other disputes. Given the very high costs of violence as an enforcement mechanism, norms backed by limited community sanctions can therefore play a crucial role.

In keeping with the season

Here's an April Fool's Day piece on joke-stealing, social norms and intellectual property (via Mark Thoma):

...Comedians have rules of their own about joke-stealing. And they impose their own punishments on thieves... Why do comedians do this? In part, because they live in a world where intellectual property law – in particular, copyright – does not help them much when a rival comedian steals a joke... lawsuits are simply too expensive and uncertain to work as an effective response... Today’s comics are intent on enforcing ownership rights. Yet they do so via social norms – informal but nonetheless powerful rules enforced by comedians on their peers... Comedians maintain a small list of commandments that every comic must follow – or risk being ostracized, boycotted, and sometimes worse. These norms track copyright law at times... More often than not, however, the norms deviate from copyright: for example, copyright protects expression but not ideas, but comedians’ norms protect expression as well as ideas...

I might quibble with the definition of social norms as rules enforced by fear of reprisal, but that's a minor point and, given my previous comments about Freakonomics, I have to give credit where credit is due.

Two thoughts on April Fool's Day blogging

1. I'd better be careful linking to any too-good-to-be-true stories.

2. Wait a minute! This is the blogosphere; how much less reliable can it get?

Skipping the math

From Felix Salmon:

Justin Fox sums up the overwhelming majority of economics papers in one sentence:

The basic form of an academic economics paper is a couple of comprehensible paragraphs at the beginning and a couple of comprehensible paragraphs at the end, with a bunch of really-hard-to-follow math or statistical analysis in the middle.

What he doesn’t (need to) mention is the way that journalists, myself included, read economics papers: we generally have no ability or inclination to try to understand the details of the formulae and regression analyses, so we confine ourselves to reading the stuff in English, and work on the general assumption that the mathematics is reasonably solid.

The problem of course is that we really have no basis for making that general assumption: we make it not because we think it’s particularly justified or justifiable, but because we don’t have any choice. What’s more, because we’re always interested in what’s new, and because we have easy access to the internet and little access to expensive journals, we gravitate to preprints at sites like SSRN, rather than papers which have gone through peer review.

Blockbusters, Franchises and Apostrophes

More on the economics of genre fiction

The story so far: last week Andrew Gelman had a post on a book that discussed the dominance of best seller lists and suggested that it was due to their increased quality and respectability. I argued that the quality and respectability had if anything decreased (here), posted some background information (here and here) then discussed how the economics of publishing from the late Nineteenth Century through the Post-War era had influenced genre fiction. The following closes with a look at where we are now and how the current state of the market determines what we're seeing at the bookstore.

As the market shrank in the last part of the Twentieth Century, the pay scale shifted to the feast and (mostly) famine distribution of today. (The century also saw a similar shift for musicians, artists and actors.) Non-paying outlets sprang up. Fan fiction emerged (non-licensed use of characters had, of course, been around for years -- Tiajuana bibles being a classic example -- but fan fiction was written for the author's enjoyment without any real expectation of payment). These changes are generally blamed on the internet but the conventional wisdom is at least a couple of decades off. All of these trends were well established by the Seventies.

With the loss of the short story market and the consolidation of publishing, the economics of writing on spec became brutal. Writing and trying to sell a novel represents a tremendous investment of time and energy with little hope of success. By comparison writing on spec in the Forties meant coming up with twelve to fifteen pages then sending them off to twenty or so potential markets. The best of these markets paid good money; the worst were hungry for anything publishable.

The shift from short story to novel also meant greater risk for the publisher (and, though we don't normally think of it in these terms, for the reader who also invested money and time). A back-pages story that most readers skipped over might hurt the sales and reputation of a magazine slightly but as long as the featured stories were strong, the effect would be negligible. Novels though are free-standing and the novel gets that gets skipped over is the novel that goes unsold.

When Gold Medal signed John. D. MacDonald they knew were getting a skilled, prolific writer with a track record artistically and commercially successful short fiction. The same could be said about the signing of Donald Westlake, Lawrence Block, Joe Gores and many others. Publishing these first time authors was a remarkably low risk proposition.

Unfortunately for publishers today, there are no potential first time authors with those resumes. Publishers now have to roll the dice on inexperienced writers of unknown talent and productivity. In response to that change, they have taken various steps to mitigate the risk.

One response was the rise of the marketable blockbuster. The earliest example I can think of is the book Lace by Shirley Conran. If memory serves, Lace got a great deal of attention in the publishing world for Conran's huge advance, her lack of fiction-writing experience, and the role marketing played in the process. The general feeling was that the tagline ("Which one of you bitches is my mother? ") came first while the book itself was merely an afterthought.

More recently we have Dexter, a marketer's dream ("He's a serial killer who kills serial killers... It's torture porn you can feel good about!"). The author had a few books in his resume but nothing distinguished. The most notable was probably a collaboration with Star Trek actor Michael Dorn. The first book in the series, Darkly Dreaming Dexter was so poorly constructed that all of the principals had to act completely out of character to resolve the plot (tip for new authors: when a character casually overlooks her own attempted vivisection, it's time for a rewrite*).

The problems with the quality of the novel had no apparent effect on sales, nor did it prevent the character from appearing in a successful series of sequels and being picked up by Showtime (The TV show was handled by far more experienced writers who managed to seal up almost all of the plot holes).

The point here is not that Darkly Dreaming Dexter was a bad book or that publishing standards have declined. The point is that the economics have changed. Experienced fiction writers are more rare. Marketable concepts and franchises are more valuable, as is synergy with other media. The markets are smaller. There are fewer players. And much of the audience has a troublesome form of brand loyalty.

Normally of course brand loyalty is a plus, but books are an unusual case. If you convince a Coke drinker to also to drink Sprite you probably won't increase his overall soda consumption; you'll just have cannibalization. But readers who stick exclusively with one writer are severely underconsuming. Convince James Patterson readers to start reading Dean Koontz and you could double overall sales.

When most readers got their fiction either through magazines or by leafing through paperback racks, it was easy to introduce them to new writers. Now the situation is more difficult. One creative solution has been apostrophe series such as Tom Clancy's Op Center. Other people are credited with actually writing the books but the name above the title is there for branding purposes.

Which all leads us back to the original question: Why did thrillers become so dominant?

They tend to be easily marketable.

They are compatible with franchises.

They lend themselves to adaptation as big budget action movies.

Their somewhat impersonal style makes them suitable for ghosting or apostrophe branding.

They are, in short, they are what the market is looking for. As for me, I'm looking for the next reprint from Hard Case, but I might borrow the latest Turow after you're done with it.


* "Is that a spoiler?"
"No, sir. It was spoiled when I got here."

p.s. I was going to tie in with a branding situation Slim Jim snacks faced a few years ago but this post is running a bit long. Maybe I'll get back to it later.

Vanishing media

Brad Plummer shares the following examples of how ephemeral some of our art and images are:

Actually, though, we don't even need to consider the apocalypse. The fragile state of digital storage is already causing trouble. NASA has a few people racing to recover old images from its Lunar Orbiter missions in the 1960s, which are currently stored on magnetic tapes and may not be long for this world. And the National Archives is struggling to preserve its digital records, which tend to rot faster than paper records.

A related tale of disintegrating media comes from Larry Lessig's Free Culture—though this one has a twist. There are a lot of films that were made after 1923 that have no commercial value anymore. They never made it to video or DVD; the reels are just collecting dust in vaults somewhere. In theory, it shouldn’t be too hard to digitize these films and put them in an archive. But alas, thanks to the Sonny Bono Copyright Term Extension Act that was passed by Congress in 1998, any film made after 1923 won't enter the public domain until at least 2019.

That means these films are still under copyright, and anyone who wanted to restore them would have to track down the copyright-holders (not always easy to do) and probably hire a lawyer. And who's going to go through that much trouble just to restore some obscure movie that only a few people might ever watch? Yet a lot of these older movies were produced on nitrate-based stock, and they'll have dissolved by the time 2019 rolls around, leaving nothing behind but canisters of dust. It's sort of tragic.

It's also a perversion of the original intent of copyright laws. Copyrights like patents are government imposed monopolies that dampen commerce and development of new works. Intellectual property rights were seen, in the words of Jefferson, as a necessary evil to balance the interests of the creators with those of the general public by granting temporary these monopolies.

The suggestion that extending these monopolies for almost a century is meant to protect the interests of creators is absurd. The vast majority of these rights are held by companies like Disney or Time-Warner, companies that frequently screwed over the original creators and are now spending more money lobbying to keep the rights than they did to actually acquire them. This is particularly egregious for Disney, a company founded on adaptations of public domain works.

Another outstanding (and tragic) economics story from This American Life

"A car plant in Fremont California that might have saved the U.S. car industry. In 1984, General Motors and Toyota opened NUMMI as a joint venture. Toyota showed GM the secrets of its production system: how it made cars of much higher quality and much lower cost than GM achieved. Frank Langfitt explains why GM didn't learn the lessons – until it was too late."

Currently available for a free download.

Let's talk about sex

More cool stuff from the New York Times' best science writer (not that the others have set the bar that high)

The real thing

Jaime Escalante dies at 79; math teacher who challenged East L.A. students to 'Stand and Deliver'

Jaime Escalante, the charismatic former East Los Angeles high school teacher who taught the nation that inner-city students could master subjects as demanding as calculus, died Tuesday. He was 79.

Today's pointer

I am on the road (trasporting 3 pets solo -- don't ask) so blooging is very light.

But John D Cook brought up an interesting point today that should not be missed. It's a grey area but it is worth being very careful about just how much effort there is involved trying to improve medical care and how many barriers need to be crossed.

It's a difficult balance!

Hey, I used to work there

The Decline of the Middle (Creative) Class

I suggested in an earlier post that the rise to dominance of the thriller had not been accompanied by a rise in quality and reputation. In this and the next post, I'll try to put some foundations under this claim.

Popular art is driven by markets and shifts in popular art can always be traced back, at least partly, to economic, social and technological developments as well as changes in popular taste. The emergence of genre fiction followed the rise of the popular magazine (check here for more). Jazz hit its stride as the population started moving to cities. Talking pictures replaced silents when the technology made them possible.

Crime fiction, like science fiction first appeared in response to demand from general interest magazines like the Strand then moved into genre specific magazines like Black Mask and a few years later, cheap paperbacks. The demand for short stories was so great that even a successful author like Fitzgerald saw them as a lucrative alternative to novels. There was money to be made and that money brought in a lot of new writers.

It seems strange to say it now but for much of the Twentieth Century, it was possible to make a middle class living as a writer of short fiction. It wasn't easy; you had to write well and type fast enough to melt the keys but a surprisingly large number of people managed to do it.

Nor were writers the only example of the new creative middle class. According to Rosy McHargue (reported by music historian Brad Kay) in 1925 there were two hundred thousand professional musicians in the United States. Some were just scraping by, but many were making a good living. (keep in mind that many restaurants, most clubs and all theaters had at least one musician on the payroll.) Likewise, the large number of newspapers and independent publishers meant lots of work for graphic artists.

I don't want to wax too nostalgic for this era. Sturgeon's Law held firmly in place: 95% of what was published was crap. But it was the market for crap that made the system work. It provided the freelance equivalent of paid training -- writers could start at least partially supporting themselves while learning their craft, and it mitigated some of the risk of going into the profession -- even if you turned out not to be good enough you could still manage food and shelter while you were failing.

It was also a remarkably graduated system, one that rewarded quality while making room for the aforementioned crap. The better the stories the better the market and the higher the acceptance rate. In 1935, Robert E. Howard made over $2,000 strictly through magazine sales. Later, as the paperback market grew, writers at the very top like Ray Bradbury or John O'Hara would also see their stories collected in book form.

Starting with Gold Medal Books, paperback originals became a force in 1950. This did cut into the magazine market and hastened the demise of the pulps but it made it easier than ever before to become a novelist. It was more difficult (though still possible) to make a living simply by selling short stories, but easier to make the transition to longer and more lucrative works.

It was, in short, a beautifully functioning market with an almost ideal compensation system for a freelance based industry. It produced some exceptionally high quality products that have generated billions of dollars and continue to generate them in resales and adaptations (not to mention imitations and unlicensed remakes). This includes pretty much every piece of genre fiction you can think written before 1970.

The foundation of that system, the short story submarket, is essentially dead and the economics and business models of the rest of the publishing industry has changed radically leading to the rise of marketing, the blockbuster mentality and what I like to call the Slim Jim conundrum.

Tune in next time.

Now I'm going to spend the rest of the day wondering what a giant deodorant gun looks like

From the Guardian via TNR:

Beijing is to install 100 deodorant guns at a stinking landfill site on the edge of the city in a bid to dampen complaints about the capital's rubbish crisis. ...

Thrillers on Economics -- a quick digression

I've been working on a series of posts about the economics of crime novels (see here and here) and it got me thinking about economics in crime novels. I'm no expert but here's my incomplete survey.

George Goodman (a.k.a. "Adam Smith") once bemoaned the absence of business in American literature with the notable exception of John P. Marquand. With all due respect to the estimable Marquand (himself no stranger to the pulps), Goodman might have found what he was looking for if he had spent less time in high-end bookstores and more time in his corner drugstore looking at the books with the lurid covers.

Of the many crime novels built around businesses, the best might be Murder Must Advertise, a Lord Whimsey by Dorothy L. Sayers. The story is set in a London ad agency in the Thirties, a time when the traditional roles of the aristocracy were changing and "public school lads" were showing up in traditional bourgeois fields like advertising.

Sayers had been a highly successful copywriter (variations on some of her campaigns are still running today) and has sometimes been credited with coining the phrase "It pays to advertise." All this success did not soften her view of the industry, a view which is probably best captured by Whimsey's observation that truth in advertising is like yeast in bread.

But even if Sayers holds the record for individual event, the lifetime achievement award has got to go to the man whom many* consider the best American crime novelist, John D. MacDonald.

Before trying his hand at writing, MacDonald had earned an MBA at Harvard and over his forty year writing career, business and economics remained a prominent part of his fictional universe (one supporting character in the Travis McGee series was an economist who lived on a boat called the John Maynard Keynes). But it was in some of the non-series books that MacDonald's background moved to the foreground.

Real estate frequently figured in MacDonald's plots (not that surprising given given their Florida/Redneck Riviera settings). His last book, Barrier Island, was built around a plan to work federal regulations and creative accounting to turn a profit from the cancellation of a wildly overvalued project. In Condominium, sleazy developers dodge environmental regulations and building codes (which turned out to be a particularly bad idea in a hurricane-prone area).

Real estate also figures MacDonald's examination of televangelism, One More Sunday, as does almost every aspect of an Oral Roberts scale enterprise, HR, security, public relations, lobbying, broadcasting and most importantly fund-raising. It's a complete, realistic, insightful picture. You can find companies launched with less detailed business plans.

But MacDonald's best book on business may be A Key to the Suite, a brief and exceedingly bitter account of a management consultant deciding the future of various executives at a sales convention. Suite was published as a Gold Medal Original paperback in 1962. You could find a surprising amount of social commentary in those drugstore book racks, usually packaged with lots of cleavage.


* One example of many:

“To diggers a thousand years from now, the works of John D. MacDonald would be a treasure on the order of the tomb of Tutankhamen.” - KURT VONNEGUT

All Cretans are ad execs

This ad reminded of the Liar's Paradox. Not exactly the same thing, but the juxtaposition of messages -- romanticized images of cars brainwash you into desiring hollow status symbols/look at the romanticized images of our cars -- certainly plays to the irony-impaired.

My best subject used to be recess

David Elkind has a good op-ed piece out today on the loss of unstructured playtime in many schools.

One consequence of these changes is the disappearance of what child-development experts call “the culture of childhood.” This culture, which is to be found all over the world, was best documented in its English-language form by the British folklorists Peter and Iona Opie in the 1950s. They cataloged the songs, riddles, jibes and incantations (“step on a crack, break your mother’s back”) that were passed on by oral tradition. Games like marbles, hopscotch and hide and seek date back hundreds of years. The children of each generation adapted these games to their own circumstances.

Yet this culture has disappeared almost overnight, and not just in America. For example, in the 1970s a Japanese photographer, Keiki Haginoya, undertook what was to be a lifelong project to compile a photo documentary of children’s play on the streets of Tokyo. He gave up the project in 1996, noting that the spontaneous play and laughter that once filled the city’s streets, alleys and vacant lots had utterly vanished.

For children in past eras, participating in the culture of childhood was a socializing process. They learned to settle their own quarrels, to make and break their own rules, and to respect the rights of others. They learned that friends could be mean as well as kind, and that life was not always fair.

I have some quibbles with the essay and strong objections to a couple of points but most of what Elkind has to say here is valid and important.

The fundamental assumption of all educational debates needs to be that children are naturally curious and creative, that evolution has programmed them to learn and explore. Strategies that do a good job capitalizing on that curiosity and creativity will be successful and sometimes the best way to do that is to simply get out of the kids' way.

Another reminder that improbable events are probable

From Jonathan Chait:

Brian Kalt, a law professor and former college classmate of mine, has developed his own law of presidential facial hair:

I thought you might be interested in the following ironclad law of American presidential politics. I call it Kalt’s Law: “Under the modern two-party system, if a candidate has facial hair, the Republican always has as much, or more, than the Democrat.”

Excellent primer on the economics of genre fiction.

In the introduction to Science Fiction by Gaslight, Sam Moskowitz does a really good job explaining how changes in publishing led to the creation of most of today's popular fiction genres. It's an interesting book if you can find a copy.

I'll try to tie this in with the thriller thread (see here and here) in an upcoming post.

Advice from Andrew Gelman

Whom I always defer to on non-literary matters:

They also recommend composite end points (see page 418 of the above-linked article), which is a point that Jennifer and I emphasize in chapter 4 of our book and which comes up all the time, over and over in my applied research and consulting. If I had to come up with one statistical tip that would be most useful to you--that is, good advice that's easy to apply and which you might not already know--it would be to use transformations. Log, square-root, etc.--yes, all that, but more! I'm talking about transforming a continuous variable into several discrete variables (to model nonlinear patterns such as voting by age) and combining several discrete variables to make something continuous (those "total scores" that we all love). And not doing dumb transformations such as the use of a threshold to break up a perfectly useful continuous variable into something binary. I don't care if the threshold is "clinically relevant" or whatever--just don't do it. If you gotta discretize, for Christ's sake break the variable into 3 categories.

This all seems quite obvious but people don't know about it. What gives? I have a theory, which goes like this. People are trained to run regressions "out of the box," not touching their data at all. Why? For two reasons:

1. Touching your data before analysis seems like cheating. If you do your analysis blind (perhaps not even hanging your variable names or converting them from ALL CAPS), then you can't cheat.

2. In classical (non-Bayesian) statistics, linear transformations on the predictors have no effect on inferences for linear regression or generalized linear models. When you're learning applied statistics from a classical perspective, transformations tend to get downplayed, and they are considered as little more than tricks to approximate a normal error term (and the error term, as we discuss in our book, is generally the least important part of a model).Once you take a Bayesian approach, however, and think of your coefficients as not being mathematical abstractions but actually having some meaning, you move naturally into model building and transformations.

I don't know if I entirely buy point 2. I'm generally a frequentist and I make extensive use of transformations (though none of them are linear transformations).

Fighting words from Andrew Gelman

Or at least a fighting summary of someone else's...

[I've got a meeting coming up so this will have to be quick and ugly and leave lots of plot threads dangling for the sequel]

From Andrew's reaction to Triumph of the Thriller by Patrick Anderson:

Anderson doesn't really offer any systematic thoughts on all this, beyond suggesting that a higher quality of talent goes into thriller writing than before. He writes that, 50 or 70 years ago, if you were an ambitious young writer, you might want to write like Hemingway or Fitzgerald or Salinger (if you sought literary greatness with the possibility of bestsellerdom too) or like James Michener, or Herman Wouk (if you sought fame and fortune with the possibility of some depth as well) or like Harold Robbins or Irving Wallace (if you wanted to make a business out of your writing). But the topselling authors of mysteries were really another world entirely--even though their books were ubiquitous in drugstore and bus-station bookracks, and even occasionally made their way onto the bestseller lists, they barely overlapped with serious fiction, or with bestselling commercial fiction.

Nowadays, though, a young writer seeking fame and fortune (or, at least, a level of financial security allowing him to write and publish what he wants) might be drawn to the thriller, Anderson argues, for its literary as well as commercial potential. At the very least, why aim to be a modern-day Robbins or Michener if instead you can follow the footsteps of Scott Turow. And not just as a crime novelist, but as a writer of series: "Today, a young novelist with my [Anderson's] journalistic knack for action and dialogue would be drawn to a crime series; if not, his publisher would push him in that direction."

1. I'd argue (and I think most literary historians would back me up) that in terms of literary quality, crime fiction was at its best from about the time Hammet started writing for Black Mask to either the Fifties or Sixties, a period that featured: Chandler; Ross and John D. MacDonald; Jim Thompson; Ed McBain; Donald Westlake; Joe Gores; Lawrence Block* and a slew of worthies currently being reprinted by Hard Case.

2. Crime writing was fairly respected at the time. Check out contemporary reviews (particularly by Dorothy Parker). It was even possible for Marquand to win a Pulitzer for a "serious" novel while writing the Mr. Moto books.

3. There is an economic explanation for both the drop in quality and the surge in sales, but that will have to wait. I have a meeting at one of the studios and I need to go buy a pair of sunglasses.

*Those last three did their best work more recently but they were a product of the pulps.

p.s. Here's an illustrative passage from the NYT on the literary respect a mystery writer might achieve back before thrillers were the dominant genre:

Ross Macdonald's appeal and importance extended beyond the mystery field. He was seen as an important California author, a novelist who evoked his region as tellingly as such mainstream writers as Nathanael West and Joan Didion. Before he died, Macdonald was given the Los Angeles Times's Robert Kirsch Award for a distinguished body of work about the West. Some critics ranked him among the best American novelists of his generation.

By any standard he was remarkable. His first books, patterned on Hammett and Chandler, were at once vivid chronicles of a postwar California and elaborate retellings of Greek and other classic myths. Gradually he swapped the hard-boiled trappings for more subjective themes: personal identity, the family secret, the family scapegoat, the childhood trauma; how men and women need and battle each other, how the buried past rises like a skeleton to confront the present. He brought the tragic drama of Freud and the psychology of Sophocles to detective stories, and his prose flashed with poetic imagery. By the time of his commercial breakthrough, some of Macdonald's concerns (the breakdown between generations, the fragility of moral and global ecologies) held special resonance for a country divided by an unpopular war and alarmed for the environment. His vision was strong enough to spill into real life, where a news story or a friend's revelation could prompt the comment "Just like a Ross Macdonald novel."

It was a vision with meaning for all sorts of readers. Macdonald got fan mail from soldiers, professors, teenagers, movie directors, ministers, housewives, poets. He was claimed as a colleague by good writers around the world, including Eudora Welty, Andrey Voznesensky, Elizabeth Bowen, Thomas Berger, Marshall McLuhan, Margaret Laurence, Osvaldo Soriano, Hugh Kenner, Nelson Algren, Donald Davie, and Reynolds Price.

Assumptions

We always talk about how hard it is to actually try and verify the assumptions required for missing data techniques to yield unbiased answers. Still, it really is a breath of fresh air when somebody tries to give some (data driven) guidance on whether or not an assumption really is reasonable. That was the case with a recent PDS article:

Marston L, Carpenter JR, Walters KR, Morris RW, Nazareth I, Petersen I. Issues in multiple imputation of missing data for large general practice clinical databases. Pharmacoepidemiol Drug Saf 2010 (currently an epub)

They nicely make the case that blood pressure data is likely to be missing at random in these databases. Given my thoughts that BP data is underused, this is actually a pretty major advance as it allows more confidence in inferences from these large clinical databases.

Good show, folks!

More questions about the statistics of Freakonomics

Felix Salmon is on the case:

There’s a nice empirical post-script to the debate over the economic effects of classifying the Spotted Owl as an endangered species. Freakonomics cites a study putting the effect at $46 billion, but others, including John Berry, who wrote a story on the subject for the Washington Post, think it’s much closer to zero.

And now it seems the Berry side of the argument has some good Freakonomics-style panel OLS regression analysis of the microeconomy of the Pacific Northwest to back up its side of the argument. A new paper by Annabel Kirschner finds that unemployment in the region didn’t go up when the timber industry improved, and it didn’t go down when the timber industry declined — not after you adjust for much more obvious things like the presence of minorities in the area.

Comparing Apples and furniture in a box

In a previous post on branding, I used Apple as an example of a company that, because of its brand, can charge a substantial premium for its high-quality products. In this New Yorker post, James Surowiecki compares Apple to companies that take the opposite approach.

For Apple, which has enjoyed enormous success in recent years, “build it and they will pay” is business as usual. But it’s not a universal business truth. On the contrary, companies like Ikea, H. & M., and the makers of the Flip video camera are flourishing not by selling products or services that are “far better” than anyone else’s but by selling things that aren’t bad and cost a lot less. These products are much better than the cheap stuff you used to buy at Woolworth, and they tend to be appealingly styled, but, unlike Apple, the companies aren’t trying to build the best mousetrap out there. Instead, they’re engaged in what Wired recently christened the “good-enough revolution.” For them, the key to success isn’t excellence. It’s well-priced adequacy.

These two strategies may look completely different, but they have one crucial thing in common: they don’t target the amorphous blob of consumers who make up the middle of the market. Paradoxically, ignoring these people has turned out to be a great way of getting lots of customers, because, in many businesses, high- and low-end producers are taking more and more of the market. In fashion, both H. & M. and Hermès have prospered during the recession. In the auto industry, luxury-car sales, though initially hurt by the downturn, are reemerging as one of the most profitable segments of the market, even as small cars like the Ford Focus are luring consumers into showrooms. And, in the computer business, the Taiwanese company Acer has become a dominant player by making cheap, reasonably good laptops—the reverse of Apple’s premium-price approach.

True models?

The p-value discussion started by an arcile authored by Tom Siegfried has generatesd a lot of discussion. Andrew Gelman has tried to round up many of the discussion points.

But the best part of the post (besides showing the diversity out there) was hidden at the bottom. Andrew comments:

"In all the settings I've ever worked on, the probability that the model is true is . . . zero!"

Well, he is most certainly correct in pharamcoepidemiology as well. I see a lot of variation over how to handle the biases that are inherent in observational pharmacoepidemiology -- but the focus on randomized drug trials should be a major clue that these associations are tricky to model. As a point of fact, the issue of confounding by indication, channeling bias, indication bias or whatever else you want to call it is central to the field. And the underlying idea here is that we can't get enough information about participants to model the influence of drugs being channeled to sicker patients.

So I wish that, in my field as well, people would realize that the relationships are tricky and no model is ever going to be absolutely correctly specified.

The curse of large numbers and the real problem with p-values

(Some final thoughts on statistical significance)

The real problem with p-values isn't just that people want it to do something that it can't do; they want it to do something that no single number can ever do, fully describe the quality and reliability of an experiment or study. This simply isn't one of those mathematical beasts that can be reduced to a scalar. If you try then sooner or later you will inevitably run into a situation where you get the same metric for two tests of widely different quality.

Which leads me to the curse of large numbers. Those you who are familiar with statistics (i.e. pretty much everybody who reads this blog) might want to skip the next paragraph because this goes all the way back to stat 101.

Let's take simplest case we can. You want to show that the mean of some group is positive so you take a random sample and calculate the probability of getting the results you saw or something more extreme (the probability of getting exactly results you saw is pretty much zero) working under the assumption that the mean of the group was actually zero. This works because the bigger the samples you take the more the means of those samples will tend to follow a nice smooth bell curve and the closer those means will tend to group around the mean of the group you're sampling from.

(For any teachers out there, a good way of introducing the central limit theorem is to have students simulate coin flips with Excel then make histograms based on various sample sizes.)

You might think of sampling error as the average difference between the mean of the group you're interested in and the mean of the samples you take from it (that's not exactly what it means but it's close) . The bigger the sample the smaller you expect that error to be which makes sense. If you picked three people at random, you might get three tall people or three millionaires, but if you pick twenty people at random, the chances of getting twenty tall people or twenty millionaires is virtually are next to nothing.

The trouble is that sampling error is only one of the things a statistician has to worry about. The sampled population might not reflect the population you want to draw inferences about. Your sample might not be random. Data may not be accurately entered. There may be problems with aliasing and confounding. Independence assumptions may be violated. With respect to sample size, the biases associated with these problems are all fixed quantities. A big sample does absolutely nothing to address them.

There's an old joke about a statistician who wakes up to find his room on fire, says to himself "I need more observations" and goes back to sleep. We do spend a lot of our time pushing for more data (and, some would say, whining about not having enough), but we do that not because small sample sizes are the root of all of our problems but because they are the easiest problem to fix.

Of course "fix" as used here is an asymptotic concept and the asymptote is not zero. Even an infinite sample wouldn't result in a perfect study; you would still be left with all of the flaws and biases that are an inevitable part of all research no matter how well thought out and executed it may be.

This is a particular concern for the corporate statistician who often encounters the combination of large samples and low quality data. It's not unusual to see analyses done on tens or even hundreds of thousands of sales or customer records and more often than not, when the results are presented someone will point to the nano-scale p-value as an indication of the quality and reliability of the findings.

As far as I know, no one reviewing for a serious journal would think that p<0.001 means that we're 99.9% sure that a conclusion is true, but that's what almost everyone without an analytic background thinks.

And that is a problem.

Another chapter in the New Republic Debate

Check it out here.

Silence

I'm in the process of moving from one corner of the United States to the other. Blogging may be extremely light for the next 2 weeks.

Apologies in advance!

Interesting variable taxation idea from Thoma

From Economist's View:

Political battles make it very difficult to use discretionary fiscal policy to fight a recession, so more automatic stabilizers are needed. Along those lines, if something like this were to be implemented to stabilize the economy over the business cycle, I'd prefer to do this more generally, i.e. allow income taxes, payroll taxes, etc. to vary procyclically. That is, these taxes would be lower in bad times and higher when things improve, and implemented through an automatic moving average type of rule that produces the same revenue as some target constant tax rate (e.g. existing rates).