This ought to reassure all those doubters of the uncertainty effect

Some of the comments following our recent post on Glenn Hubbard's op-ed were rather skeptical. To recap, Hubbard argued that uncertainty over Obama's possible implementation of policies similar to those under Clinton "reduced GDP by 1.4 percent in 2011 alone."

Now Mike Konczal has dug in the source material to explain where these numbers come from:

That said, the rate is elevated starting around 2009. Why is that? The uncertainty index consists of three parts. The first a news search for articles on policy uncertainty, which we'll return to in a minute. The second part has to do with disagreements among economic forecasters. And the last part is "the number of federal tax code provisions set to expire in future years." Tax code provisions set to expire are weighted by the formula 0.5^((T+1)/12), where T is the number of months until the tax code expires. That means these provisions weigh more in the analysis as they get closer to expiring -- those with more time left have weights approaching 0, and those close to expiration approach 1.

Having an interest in text mining techniques (n-grams and that sort of thing) I was wondering how they approached the problem. It turns out their methods were a little less sophisticated than you might expect.

A Magic Trick

But now for that magic trick. How do they construct the search of newspaper articles for their index, which generates a lot of the movement?

Their news search index is constructed with four steps. They first isolate their search to a set of articles from 10 major newspapers (USA Today, the Miami Herald, the Chicago Tribune, the Washington Post, the Los Angeles Times, the Boston Globe, the San Francisco Chronicle, the Dallas Morning News, the New York Times, and the Wall Street Journal). They then search articles for the term "uncertainty" or "uncertain." They then filter again for the word "economic" or "economy." With economic uncertainty flagged, they then filter again for one of the following words to identify government policy: "policy," "'tax," "spending," "regulation," "federal reserve," "budget," or "deficit."

See the problem? We don't know what specific stories are in their index; however, we can use their search terms listed above to find which articles would have likely qualified. Let's take a story from their first listed paper, USA Today, "Obama taking aim at GOP pledge on campaign trail," from August 28, 2010 (for the rest of this post, I'm going to underline the words in quotes that would trigger inclusion in their policy uncertainty index):

Brendan Buck, a spokesman for the House GOP lawmakers who crafted the pledge, said "it's laughable that the president would try to lecture anyone on spending." [....] Buck said the pledge was developed to address voter worries about high unemployment and record levels of government spending and debt.

"While the president has exploded federal spending and ignored Americans who are asking, 'Where are the jobs?', the pledge offers a plan to end the economic uncertainty and create jobs, as well as a concrete plan to rein in Washington's runaway spending spree," Buck said. Spokespeople for the conservative movement tell reporters that President Obama's policies are causing economic uncertainty. Reporters write it down and publish it. Economic researchers search newspapers for stories about economic uncertainty and policy, and create a policy uncertainty index out of those talking points. The conservative movement then turns around and points to the policy uncertainty index as scientifically justifying their initial talking points about Obama and uncertainty as well as the need to implement their policies. Taa-daa! Magic.