Twelve years ago at the blog --How to lie with statistics -- rare cinema history edition

 

Friday, February 18, 2011

From David Leonhardt (via DeLong):

When I read that John Boehner, the speaker of the House, had said that the federal government added 200,000 federal workers under President Obama, I wondered, “Really? Where?” I’m not aware of any major federal hiring initiatives since January 2009.

... It turns out that the 200,000 number is simply incorrect.

...

Second, Mr. Boehner was starting his clock in December 2008, the month before Mr. Obama became president. The Bureau of Labor Statistics conducts its monthly survey during the week that contains the 12th day of each month, so there is no reason to start the clock in December 2008 as opposed to January 2009. On Jan. 12, 2009, George W. Bush was still president.

To be more accurate there wasn't an honest reason. As Leonhardt points out later, 11,000 jobs were added by President Bush in his last month in office. Speaker Boehner was interval shopping, one of the most effective and time-honored methods of lying with statistics. (Given that, by Leonhardt's estimate, Boehner went from 57,000 actual jobs to a claim of 200,000, he used lots of effective and time-honored methods of lying with statistics.)

Interval shopping is based on the idea that if you can adjust the period being studied, you can make something look much better or worse than it actually is. For example, if you take one day off of the service record of the Titanic, it looks like a remarkably safe form of transportation.

The method also allows you to have a great deal of fun with denominators. You will often see people in positions of responsibility pointing to a period of growth that started just after a disastrous collapse and ends just before the next one. The worse that initial collapse was, the better your growth rate looks.

Interval shopping can be particularly effective when the groups being compared are at different stages of life. You can, for example, use it to argue that a product is less reliable than one that was introduced a couple of years later, not taking into account the difference in average ages, or you could 'prove' the mental inferiority of one immigrant group over another by comparing test scores, not taking into account the higher proportion of non-English-speaking first generation immigrants.

For beautiful example of egregious interval shopping, check out this excerpt from a rebuttal to Gore Vidal written by Peter Bogdanovich in the New York Review of Books:

Now I’m getting in a foul mood because I’m reading this sentence again: “The badness of so many of Orson Welles’s post-Mankiewicz films ought to be instructive.” That’s another of those glib, sweeping statements that play right into the reader’s lack of information and is written so as to presume a general critical atmosphere, which in this case is not just superficial, it is decidedly untrue, which makes it all the more offensive and irresponsible on Gore’s part. Almost everyone with any sense knows that Orson Welles is a great director and that Herman Mankiewicz was a talented hack,* but for the record, here is a list of the movies Orson Welles has directed since Citizen Kane:

The Magnificent Ambersons

The Stranger

The Lady from Shanghai

Macbeth

Othello

Mr. Arkadin (Confidential Report)

Touch of Evil

The Trial

Chimes at Midnight (Falstaff)

The Immortal Story

F for Fake

And these are all of Herman Mankiewicz’s post-Welles films:

Rise and Shine

Pride of the Yankees

Stand by for Action

Christmas Holiday

The Enchanted Cottage

The Spanish Main

A Woman’s Secret

The Pride of St. Louis

One of the surest signs of interval shopping is the arbitrary start point, but the key to making it work is finding an arbitrary point that doesn't look arbitrary. Here Bogdanovich is able to make use of a sloppy writing by Vidal. The phrase "post-Mankiewicz" implies that there is some special significance to these films coming after Citizen Kane. If Vidal were comparing Welles' post-Mankiewicz films to his pre-Mankiewicz films (which he obviously isn't), or if he were arguing that Welles was changed by working with Mankiewicz (which seems unlikely, though I'd need to get behind the paywall to be sure), then the wording would have been appropriate. Here, though, we simply have Vidal saying "post-Mankiewicz" when he means "non-Mankiewicz."

This small bit of imprecision on Vidal's part gives Bogdanovich the opportunity to use Kane as the start point for his interval (and Peter Bogdanovich has never been one to pass up on opportunity). When comparing careers you would normally look at entire careers. This interval includes all of Welles' films and less than half of Mankiewicz's.

To make matters worse, the intervals aren't even close to the same length for the two men. Mankiewicz drank himself to death in 1953. Welles died in 1985 (the last film on Bogdanovich's list was released in 1974).

More importantly, though, this list includes all of Orson Welles' career as a director barring some shorts and TV work, while it leaves out most of Mankiewicz's major accomplishments as a writer and producer. Even in his final, declining, alcohol-soaked years, Mankiewicz still managed a good picture or two, but a list of films that he wrote or produced before Kane would include Dinner at Eight, Million Dollar Legs (with W.C. Fields) and three out of four of the Marx Brothers' best movies Monkey Business, Horse Feathers and Duck Soup.

And when you leave out Duck Soup, that's just going too far.



*Bogdanovich's senseless group here include Mankiewicz collaborators and admirers such as Alexander Woollcott, Heywood Broun, Dorothy Parker, Robert E. Sherwood, George S. Kaufman, Marc Connelly and Nunnally Johnson, but that's a topic for another post and perhaps another blog.

Update: The conversation continues here and here.