Friday, February 18, 2011

How to lie with statistics -- rare cinema history edition

From David Leonhardt (via DeLong):
When I read that John Boehner, the speaker of the House, had said that the federal government added 200,000 federal workers under President Obama, I wondered, “Really? Where?” I’m not aware of any major federal hiring initiatives since January 2009.

... It turns out that the 200,000 number is simply incorrect.


Second, Mr. Boehner was starting his clock in December 2008, the month before Mr. Obama became president. The Bureau of Labor Statistics conducts its monthly survey during the week that contains the 12th day of each month, so there is no reason to start the clock in December 2008 as opposed to January 2009. On Jan. 12, 2009, George W. Bush was still president.
To be more accurate there wasn't an honest reason. As Leonhardt points out later, 11,000 jobs were added by President Bush in his last month in office. Speaker Boehner was interval shopping, one of the most effective and time-honored methods of lying with statistics. (Given that, by Leonhardt's estimate, Boehner went from 57,000 actual jobs to a claim of 200,000, he used lots of effective and time-honored methods of lying with statistics.)

Interval shopping is based on the idea that if you can adjust the period being studied, you can make something look much better or worse than it actually is. For example, if you take one day off of the service record of the Titanic, it looks like a remarkably safe form of transportation.

The method also allows you to have a great deal of fun with denominators. You will often see people in positions of responsibility pointing to a period of growth that started just after a disastrous collapse and ends just before the next one. The worse that initial collapse was, the better your growth rate looks.

Interval shopping can be particularly effective when the groups being compared are at different stages of life. You can, for example, use it to argue that a product is less reliable than one that was introduced a couple of years later, not taking into account the difference in average ages, or you could 'prove' the mental inferiority of one immigrant group over another by comparing test scores, not taking into account the higher proportion of non-English-speaking first generation immigrants.

For beautiful example of egregious interval shopping, check out this excerpt from a rebuttal to Gore Vidal written by Peter Bogdanovich in the New York Review of Books:
Now I’m getting in a foul mood because I’m reading this sentence again: “The badness of so many of Orson Welles’s post-Mankiewicz films ought to be instructive.” That’s another of those glib, sweeping statements that play right into the reader’s lack of information and is written so as to presume a general critical atmosphere, which in this case is not just superficial, it is decidedly untrue, which makes it all the more offensive and irresponsible on Gore’s part. Almost everyone with any sense knows that Orson Welles is a great director and that Herman Mankiewicz was a talented hack,* but for the record, here is a list of the movies Orson Welles has directed since Citizen Kane:

The Magnificent Ambersons

The Stranger

The Lady from Shanghai



Mr. Arkadin (Confidential Report)

Touch of Evil

The Trial

Chimes at Midnight (Falstaff)

The Immortal Story

F for Fake

And these are all of Herman Mankiewicz’s post-Welles films:

Rise and Shine

Pride of the Yankees

Stand by for Action

Christmas Holiday

The Enchanted Cottage

The Spanish Main

A Woman’s Secret

The Pride of St. Louis

One of the surest signs of interval shopping is the arbitrary start point, but the key to making it work is finding an arbitrary point that doesn't look arbitrary. Here Bogdanovich is able to make use of a sloppy writing by Vidal. The phrase "post-Mankiewicz" implies that there is some special significance to these films coming after Citizen Kane. If Vidal were comparing Welles' post-Mankiewicz films to his pre-Mankiewicz films (which he obviously isn't), or if he were arguing that Welles was changed by working with Mankiewicz (which seems unlikely, though I'd need to get behind the paywall to be sure), then the wording would have been appropriate. Here, though, we simply have Vidal saying "post-Mankiewicz" when he means "non-Mankiewicz."

This small bit of imprecision on Vidal's part gives Bogdanovich the opportunity to use Kane as the start point for his interval (and Peter Bogdanovich has never been one to pass up on opportunity). When comparing careers you would normally look at entire careers. This interval includes all of Welles' films and less than half of Mankiewicz's.

To make matters worse, the intervals aren't even close to the same length for the two men. Mankiewicz drank himself to death in 1953. Welles died in 1985 (the last film on Bogdanovich's list was released in 1974).

More importantly, though, this list includes all of Orson Welles' career as a director barring some shorts and TV work, while it leaves out most of Mankiewicz's major accomplishments as a writer and producer. Even in his final, declining, alcohol-soaked years, Mankiewicz still managed a good picture or two, but a list of films that he wrote or produced before Kane would include Dinner at Eight, Million Dollar Legs (with W.C. Fields) and three out of four of the Marx Brothers' best movies Monkey Business, Horse Feathers and Duck Soup.

And when you leave out Duck Soup, that's just going too far.

*Bogdanovich's senseless group here include Mankiewicz collaborators and admirers such as Alexander Woollcott, Heywood Broun, Dorothy Parker, Robert E. Sherwood, George S. Kaufman, Marc Connelly and Nunnally Johnson, but that's a topic for another post and perhaps another blog.

Update: The conversation continues here and here.

1 comment:

  1. Bogdanovich is, of course, an unabashed fan of Orson Welles, which is no sin in my book. His fandom has sadly blinded him. How is this for a statistic?

    No. of Oscar nominations for Welles - 2
    Oscar wins - 1

    No. of Oscar nominations for Mankiewicz - 2
    Oscar wins - 1

    The "talented hack" remark rings hollower when you take into account that the second nomination for Mankiewicz was for "Pride of the Yankees", one of the listed films!

    By the way, lest you think that I am swayed solely by numbers of awards (Welles has more), here is a funny sidelight. To perhaps emphasize the perfidy of either organization, it should be noted that two of Orson Welles later nominations were for his work in the movie "Butterfly". It was nominated for a Golden Globe for BEST Supporting Performance by an Actor and for a Razzie for WORST Supporting Performance by an Actor.

    The other hole in this theory is numbers. Some sexist might say something along the lines of "Men are better musicians than women". Let's look at that statement on the basis of recorded work. While it is certainly true that there are more male names than female ones on lists made by critics and musicians, it is an unfair argument, because, quite simply, many more men have recorded. The more at-bats one has, the greater possibility of hitting a home run.

    If one looks at the sheer amount of material Mankiewicz wrote, he wins handily over Welles. It is the select few that can turn out a prodigious amount of material AND have an overwhelming percentage of it adjudged genius level or thereabouts.

    Mankiewicz was hired at a time that movies were the ONLY visual medium around. People loved the new medium, therefore many, many people needed to write and write a lot. In Vaudeville, if you had a successful act, you could tour for years and never change it, because it was live, could not be preserved by the amateurs and due to a circuit of theaters, it could not be centralized. Radio and movies didn't slow Vaudeville, it killed it. Radio killed the visual acts and the verbal ones could do their act a few times at best as guests, but if one of these performers got a series, you HAD to have new material. There were hours in every day and every day needed programming to fill some or all of those hours. One man was hired on radio, because someone missed a gig and a panicky station owner literally stopped him on the street and asked him, "Can you do anything!?", and fortunately, he was a pianist; he was hired permanently not long afterwards. Radio also brought about convenience. The entertainment came to you, right into your home.

    Movies added the bonus of seeing as well as hearing celebrities and celebrities-to-be. it too needed to be fed a lot of material. They needed people that could write and write quickly and Mankiewicz fit the bill. Did some who found work skate by with a minimum of talent? Yes. Was some of Mankiewicz's work less than memorable? Yes. However, Mankiewicz's career did something that Welles' could not have done, which is to say that it spanned the era of movies from silents to sound. Any number of people lost their jobs because they might have been able to write title cards, but could not write screenplays. Mankiewicz was a journalist, theater critic, playwright AND a screenwriter and a member of the "Algonquin Round Table".*

    Is this the resume' of a talented hack?

    *One of the members of the ART was Harpo Marx, one of the few members that not only didn't write, but left school early. When asked why he, a member of a comedy team that many of the members might have considered lowbrow, was a member, he replied, "Well, they needed someone to listen."