What's a winning strategy for a lottery (and what does winning really mean)?

Two interesting stories on lotteries, one recent, the other old but still relevant.

The first, from Felix Salmon, tells of a type of lottery with a positive expected value if you bought your tickets at the right time :

Because the jackpot was basically never won, it couldn’t just keep on rising indefinitely. So Cash WinFall had a mechanism for distributing it: when the jackpot rose above $2 million, it would “roll down” into smaller prizes. For instance, if you got five out of six numbers correct in a normal week, you would win $4,000; in a roll-down week, you would win $40,000.

A bunch of what can only be called professional lottery players jumped on this quirk, and would buy up hundreds of thousands of dollars’ worth of tickets in roll-down weeks, when the swollen jackpot was certain to get distributed. By buying so many tickets, they pretty much guaranteed that they would buy enough winning tickets to turn a profit — in a typical roll-down week, they would win back 15% to 20% more than they gambled.

Weirdly, the big risk here was the 1.4% chance that the jackpot would be won — as happened, for instance, on July 10, 2008. That worked out very well for the winner, Wenxu Tong, the general partner of a company called Tong’s Fortunelot Limited Partnership, who took home nearly $2.5 million. But all the other consortiums trying to game the system that week all did very badly, losing hundreds of thousands of dollars.

There was a tinge of scandal to Estes’s reporting. “Cash WinFall isn’t being played as a game of chance,” she quoted Mohan Srivastava as saying. “Some smart people have figured out how to get rich while everyone else funds their winnings.’’ And a few days after her story appeared, the Boston Globe ran an editorial under the headline “Lottery game is fatally flawed; treasurer should shut it down”. The argument? In any lottery game, according to the paper, “the odds should be stacked equally against rich and poor”. And eventually, earlier this year, Cash WinFall was indeed phased out.

Now Gregory Sullivan, the state inspector general, has written a 25-page report on the Cash WinFall game, which is well worth reading; Estes, naturally, has written it up for the Globe, under the headline “Lottery officials knew about Cash WinFall’s flaws, IG says”. She never mentions, however, the report’s conclusion: those “flaws” ended up being very profitable for the state, and were a way for Massachusetts to get significant lottery revenues not only from the poor but also from the rich.

The second, from This American Life back in 2007, looks at the impact of winning a lottery as reported by a man whose job it was to buy those jackpots for a lump sum payment:

What Ed found was that, if lottery winners felt like they could relate to him, could trust him, then they'd be much more willing to do the deal, no matter what the terms. And Ed found it wasn't hard to get them comfortable because they actually had a lot in common.


Ed Ugel: It was just a natural fit for me. One of the biggest things that helped me was my intimate understanding of the mind of a gambler.

A fellow I did a deal with in Florida, we went to a notary to get certain pages of the contracts, all the contracts needed to be signed. Some of the signature pages needed to be notarized. And this notary happened to also sell lottery tickets. And when we went and got these pages notarized, he whipped out a wad of bills out of his pocket and bought 1,000 scratch tickets while the notary slash lottery ticket salesperson was notarizing the signature pages on his contract to sell me his annuity from the lottery win. He was sitting there. He didn't even look at the contracts. He's just scratching away. He couldn't even wait to get away from the booth. I'll never forget the way the notary looked at me sort of in awe. And yeah, it was a little daunting, a little bizarre seeing it. But I knew just who this cat was. I'd been that guy. And I would like to think that maybe I was a little bit better than scratch tickets in the airport, but I don't know that scratch tickets in the airport are that much classier than video poker machines in the strip clubs of Oregon.

Is there a political historian in the house?

It seems unusual to announce a VP pick more than two weeks before the convention. How common is this sort of thing?

My problems with Matt Yglesias' Slate piece

Joseph recently had a post on Matt Yglesias's Slate piece on data. I pointed out in the comments that I wasn't that impressed. Here's a more detailed explanation.

I realize that Yglesias is writing for a general audience and that can muddle the issue, but I keep getting the feeling that he really doesn't get the subtlety here, that he misses the fundamental asymmetry between significance and insignificance and the importance of what Friedman and Rowe are saying.

I don't want to get caught up in the weeds of interpreting (let alone defending) p-value, but if you see a contextually significant result you pretty much know that there's a relationship there. It may not be causal or useful and it may not be what it appears, but there's something.

If you don't see a significant result, you don't know anything. Maybe there's nothing there or maybe there's too much noise or the relationship is outside the range of data or you could be using techniques that miss nonlinear relationships (an argument for CHAID and CART but that's a topic for another post). You certainly wouldn't draw a conclusion like burning fossil fuels doesn't cause heat.

As for the thermostat scenario, Yglesias focuses on this as an example of observational analysis missing a relationship. That's true but it's a very specialized case of a wide spread problem (lots of things cause us to miss relationships). Furthermore, it misses the real profundity of the analogy: it is the job of self-regulating systems to put forces in opposition, to set up destructive interference. The better the system works, the less appears to happen.

And finally, while I'm in a picky mood,  Yglesias should realize that lots of natural sciences (astronomy, meteorology) make very limited use of experiments.

Two essential posts on the Ryan pick

Nate Silver looks at the electoral implications.

Jonathan Chait handles the historical context.

Nate Silver answers a question I've had for a while

From Nate Silver's analysis of Ryan's selection as VP (emphasis added)

The economy? Well, it isn’t very good. But it also doesn’t appear to be getting much worse, and some recent signs — like the July jobs report — suggest a slight brightening of the outlook. It’s not quite the case that incumbent presidents are favored to win unless there is an outright recession, but that also isn’t that far from the truth. Incumbent presidents tend to get the benefit of the doubt from voters, especially when, as in Mr. Obama’s case, they are regarded as likable, their party is in its first elected term, they are perceived as competent on foreign affairs and they have avoided major scandals.

I'd noticed that while assessments of political history often talked about a one-term president, I didn't hear much about how long a party held the White House. which always struck me as an important potential factor in a model. It's good (though hardly surprising) that Silver is considering it, but I do have to wonder how many of the experts on CNN are being as thorough.

Harm Reduction

DrugMonkey has a post on legalization efforts for marijuana in Washington state.  I think that this might be my public health background, but why is the focus on on harm reduction and not the legal status of the drug? I have no trouble believing that use of this substance may have adverse effects over the long term.  But so do legal substances like tobacco and alcohol.  Furthermore, how does the harm stack up to the harm done by a term in a prison?  A lifetime of reduced employment opportunities, acculturation in a brutal environment as well as being a victim of the violence that occurs in jails.

Why not look for middle grounds?  A heavily taxed substance that minors are prevented from buying (the tobacco model)?  Decriminalization so that use equals fines and not police breaking down one's door (the Canadian model)?

Why is the focus not on maximizing public health outcomes?

Another data point on the competence of NBC Universal*

I was thinking about the Christian Slater show My Own Worst Enemy the other day (not a sentence I write that often) specifically I was thinking about how much valuable Olympic ad space NBC burned on the show in 2008 before panicking at a slow start and pulling the plug in the middle of the first story arc.

On today's All Things Considered, Andrew Wallenstein had a nice piece on NBC's attempts to use the Olympics to promote its fall schedule, complete with some embarrassing examples that had slipped my mind:

Sure, you may be spending a lot of time with NBC this summer. But they'd like you to start thinking about the fall, too. That's why what seems like every third commercial during the Olympics is for the network's own shows. Animal Practice, Go On, Chicago Fire ... they're all on NBC's fall schedule, and you may feel like you're hearing about them — complete with lame sports metaphors — as much as you're hearing about gymnastics and track.

But the network isn't stopping at mere promotion; they're airing entire episodes of select shows after primetime coverage of the competition. Some will even start their seasons right after the closing ceremonies.

You may ask yourself: If the Olympic Games are such a powerful viewer magnet, why not schedule all the fall shows to start right after they end?

The answer: Because it almost never works. NBC has tried again and again over the years to use the Olympics as a launch pad for other programming. But do you remember Father Of The Pride in 2004? Or Conviction in 2006? Didn't think so. Both are examples of shows that failed to take off after being heavily promoted during Olympic coverage.

There are a number of theories as to why it doesn't work. First, there's the distinct possibility that none of the shows NBC has tried to launch out of the Olympics were all that good. Or maybe it's the fact that the Olympics provide what NBC's rivals dismiss as a "rented audience," meaning they're the kinds of viewers who flock to the Olympics but aren't interested in much else.

Along similar lines, TV by the Numbers looked at the track record of the new shows NBC promoted during the 2008 Olympics:

My Own Worst Enemy (the Christian Slater curse is born!) lasted 9 episodes
Knight Rider, lasted 17 episodes
Kath & Kim, lasted 17 episodes
America's Toughest Jobs (reality, started late August, season/series finale Oct)
Crusoe (started after ATJ) lasted 12 episodes.

As mentioned before, NBC has been badly run for years. No network has ever done more to deserve fourth place. By comparison, CBS is much better run and the small upstart Weigel is even more impressive. In theory, we should see huge salaries at CBS, loads of market interest and good press for Weigel and heads rolling like bowling balls at NBC Universal. Instead we're getting one out of three. It took years and thirty plus million in severance to get rid of Zucker and the chances are good that you've never heard of Weigel outside of this site.






* The people who brought you Battleship -- the Movie and this remarkable piece of business logic.

Yglesias on the Dangers of Observational Data

Matt Yglesias has a piece on the Dangers of Data that really should be the Dangers of Observational Data!  True randomized or quasi-randomized experiments, when you can do them, have none of these limitations ascribed to the thermostat problem (and, in physics, an experiment is how you would figure out what the thermostat actually does). 

I am also amazed by the different foci that fields put on different methodological issues.  In observational pharmacoepidemiology we are obsessed with the issue of confounding by indication and constantly worry that it is leading to non-trivial amounts of bias.  The concept behind confounding by indication is awfully similar to the problem described by Milton Friedman's thermostat.  But I never hear economists bring that up as a major issue with observational data; perhaps because they lack experiments to tell them how often an observational estimate is wildly inaccurate (whereas in pharmacoepidemiology these experiments are slow and rare rather than non-existent). 

None of this is to say that you cannot do valid inference with observational data -- you most definitely can.  But it does highlight the need to be very, very careful. 

An almost perfect example of modern political reporting (and I don't mean that in a good way)

From NPR, sadly, Ari Shapiro reports that both candidates have tried to associate themselves with the Clinton legacy. That's a valid, newsworthy topic, but to cover it you have to acknowledge some basic asymmetries. Having a popular Democratic ex-president is more likely to help the Democrats. By the same token, it's easier for Obama to point out similarities between his policies and Clinton's than it is for Romney.

You won't, however, get that from Shapiro. In his report, both parties are pursuing the same strategy with apparently the same risks, potential rewards and level of rhetoric consistency.

As long as Bill Clinton has been on the public stage, there have been people of both parties willing to say negative things about him. But this year, even high-profile Republicans are waxing nostalgic about the Clinton years. This was Newt Gingrich on CNN after Democrats announced that the former president will have a prime speaking spot at the convention.

NEWT GINGRICH: President Clinton got four consecutive balanced budgets. President Obama has had huge deficits. So I think having Bill Clinton there is going to remind people of a Democrat they used to like, and may in fact shrink Obama by comparison.

SHAPIRO: On the campaign trail, Mitt Romney has been applauding the Clinton welfare program as an accomplishment for the ages. Here he was in Illinois this week.

MITT ROMNEY: One of the things that happened in the last couple of decades was one of the greatest bipartisan successes we've seen. And that was President Bill Clinton and Republicans coming together to reform welfare.

SHAPIRO: Republicans who praise Bill Clinton can show that they are not mindless partisans. It's a way of saying: There are Democrats I like, just not the one in office right now. Of course, the Democrat in office right now has tried to co-opt Clinton's legacy, too.

PRESIDENT BARACK OBAMA: My theories have been tested. Last time they were tried was by a guy named Bill Clinton.

SHAPIRO: On the campaign trail, Barack Obama makes it sound like he's running to continue the Clinton administration.

(APPLAUSE)

OBAMA: And that's why I'm running for a second term as president of the United States, to go back to what works.

Do we really have to go to the tape here? Do we have to compare the "bad things" Democrats said about Clinton (irresponsible, opportunistic triangulator) to those said by Republicans (drug dealing, Manchurian Candidate who raped his lesbian wife and countless other women and had his best friend murdered)? Do we have to review the statements that Republicans like Gingrich made about Clinton's policies at the time? Do we need to point out that Obama "makes it sound like he's running to continue the Clinton administration" because his policies actually are largely a continuation of Clinton's policies, and where there are difference on things like taxes, Clinton is generally to the left of Obama?

As James Fallows, Paul Krugman and many others have noted, journalistic fairness does not consist of saying the same number of good things and bad things about both parties. That's not balance; it's cowardice and it's undermining our ability to have a productive discussion (and in a democracy that's a bad thing to undermine).

the clothesline paradox

Shane Greenstein has some smart things to say about orphan technologies:

Does the clothesline paradox apply to information technology? There is a relationship between the clothesline paradox and digital dark matter, but there is also a subtle and important difference. It is important to keep those differences straight. It makes a difference to several contemporary policy debates.

That will take some explaining. There are some terms to define.

The clothesline paradox comes from energy economics. For a definition here is a quote from this well-meaning article from the Whole Earth Catalogue:

If you take down your clothes line and buy an electric clothes dryer the electric consumption of the nation rises slightly. If you go in the other direction and remove the electric clothes dryer and install a clothesline the consumption of electricity drops slightly, but there is no credit given anywhere on the charts and graphs to solar energy which is now drying the clothes.

In other words, standard approaches to economic measurement do not count inputs that lack a price. The clothesline paradox leads to under-counting the importance of activity that uses free endowments from nature. It is one of the quirks of modern economic measurement. No price equals no value.

Why does that matter? For one, out of sight leads to out of mind. The sun does not have a firm that lobbies on its behalf. Policy conversation tends to favor existing firms with seemingly big economic contributions, and tends to underestimate the importance of the free.

Perhaps more importantly, when a new technology (which uses the free inputs) substitutes for existing economic activity, on first glance it looks like the new technology brings about a decline in total economic activity. That appearance is misleading, of course, because the savings goes into other economic activity, but those gains are diffuse and difficult to identify.

A third aspect matters as well. An unpriced input tends to come without restrictions on use. There are very few laws governing the proper use of the sun. (Ok, there are a few zoning laws, actually, but you get my point).

This bothers Tim O’Reilly. Why? He argues that open source software suffers from a clothesline paradox. Open source software is an input into one trillion dollars worth of activity, he estimates. Yet, because the open source is unpriced, it gets little or no credit.

Simon Phillips of InfoWorld liked this point, and gave a catchy label for the effect, calling open source software “The Stealth Stimulus Package.” Phillips goes on to compare the open source software and the clothesline paradox, arguing for their similarity.

There is a good insight there, but also an interesting oversight. The interesting insight is worth stressing. Open source software deserves credit, but there is more. This comparison reminds me of an old column in this space, one devoted to “digital dark matter.” Digital dark matter are “important building blocks of the digital economy that we do not measure using standard tools.” (Indeed, the first example of digital dark matter in that essay is open source, where the lack of price is the source of the issue.)

There is a key difference between using the sun and using open source software, however. Nobody has to invest in the sun in order to keep the light coming. Not so for some digital dark matter. Fail to invest, and stuff will not arise.

For my take on a related subject, check here.

This ought to reassure all those doubters of the uncertainty effect

Some of the comments following our recent post on Glenn Hubbard's op-ed were rather skeptical. To recap, Hubbard argued that uncertainty over Obama's possible implementation of policies similar to those under Clinton "reduced GDP by 1.4 percent in 2011 alone."

Now Mike Konczal has dug in the source material to explain where these numbers come from:

That said, the rate is elevated starting around 2009. Why is that? The uncertainty index consists of three parts. The first a news search for articles on policy uncertainty, which we'll return to in a minute. The second part has to do with disagreements among economic forecasters. And the last part is "the number of federal tax code provisions set to expire in future years." Tax code provisions set to expire are weighted by the formula 0.5^((T+1)/12), where T is the number of months until the tax code expires. That means these provisions weigh more in the analysis as they get closer to expiring -- those with more time left have weights approaching 0, and those close to expiration approach 1.

Having an interest in text mining techniques (n-grams and that sort of thing) I was wondering how they approached the problem. It turns out their methods were a little less sophisticated than you might expect.

A Magic Trick

But now for that magic trick. How do they construct the search of newspaper articles for their index, which generates a lot of the movement?

Their news search index is constructed with four steps. They first isolate their search to a set of articles from 10 major newspapers (USA Today, the Miami Herald, the Chicago Tribune, the Washington Post, the Los Angeles Times, the Boston Globe, the San Francisco Chronicle, the Dallas Morning News, the New York Times, and the Wall Street Journal). They then search articles for the term "uncertainty" or "uncertain." They then filter again for the word "economic" or "economy." With economic uncertainty flagged, they then filter again for one of the following words to identify government policy: "policy," "'tax," "spending," "regulation," "federal reserve," "budget," or "deficit."

See the problem? We don't know what specific stories are in their index; however, we can use their search terms listed above to find which articles would have likely qualified. Let's take a story from their first listed paper, USA Today, "Obama taking aim at GOP pledge on campaign trail," from August 28, 2010 (for the rest of this post, I'm going to underline the words in quotes that would trigger inclusion in their policy uncertainty index):

Brendan Buck, a spokesman for the House GOP lawmakers who crafted the pledge, said "it's laughable that the president would try to lecture anyone on spending." [....] Buck said the pledge was developed to address voter worries about high unemployment and record levels of government spending and debt.

"While the president has exploded federal spending and ignored Americans who are asking, 'Where are the jobs?', the pledge offers a plan to end the economic uncertainty and create jobs, as well as a concrete plan to rein in Washington's runaway spending spree," Buck said. Spokespeople for the conservative movement tell reporters that President Obama's policies are causing economic uncertainty. Reporters write it down and publish it. Economic researchers search newspapers for stories about economic uncertainty and policy, and create a policy uncertainty index out of those talking points. The conservative movement then turns around and points to the policy uncertainty index as scientifically justifying their initial talking points about Obama and uncertainty as well as the need to implement their policies. Taa-daa! Magic.

"The employees’ incentives are aligned more with the markets than they are with Zuckerberg"

Felix Salmon delves into some of the issues I was alluding to in this earlier post.

From Facebook’s Faustian bargain

The point here is that although Facebook might be controlled by Zuckerberg individually, it’s still nothing without its thousands of employees. And those thousands of employees have entered into a bargain with Zuckerberg: they’ll accept relatively modest salaries, and work hard, because Zuckerberg is giving them substantial amounts of equity in the company. Once Facebook went public, every single Facebook employee became acutely aware of the company’s share price, what direction it was going in, what that move was doing to their net worth, and what public investors wanted to see from the company (revenues, and profits, rising sharply).

As such, despite his voting control at board level, it’s actually really hard for Zuckerberg to keep his employees focused on long-term platform-building, rather than short-term obsession over the share price. For one thing, they don’t own the company; many of them are going to leave, at some point, and so their time horizon is necessarily going to be shorter than Zuckerberg’s. And at any company with broad share ownership and a public share price, employees are always going to pay a huge amount of attention to whether it’s going up or going down.

On top of that is the classic Silicon Valley problem — which is that employees are always searching for the new new thing, the company where they can get early-stage equity and make themselves a fortune. Or, at the very least, join a mature company like Apple where the stock can still rise enormously. If Facebook’s stock is going down rather than up, its employees will start looking for other opportunities, and the company will find it much harder to attract talent.

If you want to know what's wrong with the movie industry

Listen to this interview with Peter Berg, director of Battleship (sorry, no transcript). Lots of troubling information if you listen for it but the one that jumped out at me was the part about thirteen minutes in where the Berg says (and I'm inclined to believe him) that the studio actually pressured him to go from the original 150 million to 200. There were all sorts of elaborate rationales for this, but in the final analysis, none of them undercut the conclusion that the business model here is deeply screwed up.

When is a conclusion credible: health care edition

I think that critical thinking skills are needed when discussing health outcomes now more than ever.  Consider this prelude to reporting study results by Matt Yglesias:

Conservatives don't like Medicaid because they believe programs that tax the rich to transfer resources to the poor are bad for long-term economic growth and violate principles of cosmic justice. But since nobody likes to admit to the existence of a tradeoff, conservatives have lately taken to mounting the bizarre argument that giving health care to low-income poor people doesn't improve health outcomes.

 Now, let us consider just what this would likely imply.  If Medicaid, a conservative, inexpensive, and rationed health care system cannot improve outcomes what help is there for any possible level of health care?  After all, this care is focused on the sickest possible patients who have the fewest resources to handle a medical condition on their own.

So one would actually begin to wonder if any health care at all is effective at all.

Now it turns out that more careful studies are showing that Medicaid is a pretty cost effective way of saving lives.  But, if we believed the original (flawed) studies, wouldn't the really exciting take home point be that modern medicine is ineffective at saving lives?  That is a hige area of GDP that we could simply stop and re-allocate to more productive activities.

It really was an odd argument, all around. 

The uncertainty paradox

Via DeLong, Glenn Hubbard has been making the point that uncertainty hurts the economy. Here's what he had to say in the WSJ:

As a consequence, uncertainty over policy—particularly over tax and regulatory policy—slowed the recovery and limited job creation. One recent study by Scott Baker and Nicholas Bloom of Stanford University and Steven Davis of the University of Chicago found that this uncertainty reduced GDP by 1.4% in 2011 alone, and that returning to pre-crisis levels of uncertainty would add about 2.3 million jobs in just 18 months.

(Matthew O'Brien points out that this uncertainty was driven by the debt ceiling crisis and the European crisis, but that's somewhat off-topic for this post.)

Without getting into the weeds here (or mentioning how close the idea of fear of uncertainty affecting the economy comes to sounding like Keynes talking about animal spirits), my question here is how have Romney decisions affected uncertainty. I'm sure Hubbard would argue that Romney has a good chance of winning in November which would mean we would have a good chance of seeing Romney's policies implemented, but Romney has been extremely reluctant to release specifics.

If you need to make a business or investment decision that depends on tax policy, you have no way of accurately evaluating that decision until the election is decided and, in the event of a Romney victory, actual proposals are made in 2013. We're talking six months here, closer to to twelve if you include time for the legislation to pass and the dust to settle.  Therefore doesn't Hubbard's point about uncertainty imply that Romney is hurting the economy by refusing to supply essential information to businesses and investors?

(Much of this applies to Paul Ryan as well.)

I can imagine valid arguments for keeping policies broad and open-ended early in the process and I can imagine valid arguments for the need to reduce uncertainty.

But for the life of me, I can't imagine a valid argument for both.