How a hit show loses (or perhaps "loses") money

This is one of those cases where threads may or may not be colliding. The following could be yet another story of Hollywood cost spirals. We have lots of unambiguous evidence that things are getting out of control. What's more, these examples it into a larger narrative of cost shooting up in the very areas where technology are to be driving it down.

On the other hand, we could have just another instance of a far older and better established genre, the Hollywood accounting story. There is a long history of films that obviously turned huge profits being labeled as money losers when the time came to pay contributors their share.

Regardless of which version you pick, it is interesting and always useful to look at some actual numbers when trying to follow a business story.

From the Hollywood Reporter

Puzzle and Problem-solving videos [Doublet edition] – now slightly less beta

First the usual caveats. These videos still aren't all that pleasing to either the eye or the ear (which doesn't leave a lot of senses to engage). The plan is still to focus first on concept then on specific content while hopefully keeping the production values at least adequate. For example, recording the audio in a relatively quiet closet-sized hallway to get a reasonably clean track and ignoring the weird acoustics and stilted, choppy delivery that comes from wrestling with a jury-rigged arrangement while trying to narrate.

The concept is a series of math video (initially concentrating on puzzles) that focus less on specific problems and more on problem-solving. The video embedded here talks about analyzing problems to see what makes some easy and others difficult, then seeing if we can use that information to suggest strategies for tackling the more challenging ones. In the follow-up ("turn GRASS GREEN" -- also from Carroll), I talk about flipping problems and working forwards then backwards then forwards... In the Kakuro video I discuss finding footholds. In a couple based on Dudney puzzles, I cover mixing algebraic and trial-and-error solutions to be better guessers. You get the picture.

I'm more or less satisfied with the concept and content (or at least with the direction they're headed) but production and promotion still have a long way to go and I'm not entirely certain how to proceed. A few years ago, if you found a good niche and posted some videos of acceptable quality, there was a decent chance that you'd find an audience through organic search. Based on conversations with people who've worked with SEO, that's very difficult now between the competition and Google changing its algorithms to crack down on people gaming the system (which creates a lot of collateral damage among small players).

I'll be exploring other ways of promoting the videos starting here.If you're interested in the approach I outlined earlier or just in puzzles in general, please check this out (feedback is always appreciated) and keep an eye out for future installments. If you like what you see in terms of content, spread the word around. I'm getting advice from some acquaintances who work in video production so the quality on that side should definitely be improving.

Maybe I'll get around to a post on...

- The extraordinary value of name recognition in the 21st Century built around this post from Ken Levine.

- Exploring (or at least seconding) the points Paul Campos makes in "The key to a more egalitarian society is for everyone to go to elite colleges."

- Indulging in a bit of schadenfreude on how the establishment press's quarter century of tolerating increasingly shoddy and sensationalistic reporting is catching up with them in the form of the Trump campaign, as explained by Mr. Pierce.

- I have great respect for the crowd at FiveThirtyEight (when they're good they're very good), but given their track record, is there any question you'd less like to hear them tackle than Why Did The 'Stop Trump' Movement Fail?

- Connectography is the name of an actual book and not just the title for a parody TED Talk.

- Tierney Sneed does a good job pointing out the limits of the Goldwater analogy. Jon Huntsman proves her point.

Dean Dad on Zero Sum Performance

This is Joseph.

From Dean Dad:

In most states or systems with performance funding, the overall level of funding -- the pie to be sliced, if you prefer -- is either flat or declining.  Which means that if everyone improves by the same five percent, then everyone gets the same zero percent increase.  You may be making progress, but you’re still essentially running in place.  Worse, if you improve by three percent but the statewide average is five, you actually lose ground.

I find these sort of systems to be extremely tough environments to build motivation and success in, so I am glad that they are being scrutinized.  One issue is that it creates some very perverse incentives.  Consider this in a human resources context -- you do well at your job, get promoted, and now you are at the bottom of the ranking for the new rile you are in.  If the bottom group tends not to survive (long term) it suggests promotion is bad.  Or that politics will be played to make a promotion survivable, which can be pretty toxic. 

Where I have seen this system thrive is in very high reward environments.  Only one actress could be cast to play "Rey" in the new star wars film (no matter how good the second best applicant was), but there are no shortage of volunteers because the pay-off is so great. 

But placed into a less highly leveraged environment and it is a recipe for lowering motivation and, occasionally, penalizing the colleges that take on the toughest challenges.  We all want to think that we are so awesome that we can do amazing things with tough problems, and sometimes people do, but it can be a thin line between that and being set up to fail. 

Context only counts if it shows up in the first two dozen paragraphs

The New York Times has a good piece on the impact of voter ID laws but I do have a problem with a few parts (or at least with the way they're arranged).

Stricter Rules for Voter IDs Reshape Races

By MICHAEL WINES and MANNY FERNANDEZ MAY 1, 2016

SAN ANTONIO — In a state where everything is big, the 23rd Congressional District that hugs the border with Mexico is a monster: eight and a half hours by car across a stretch of land bigger than any state east of the Mississippi. In 2014, Representative Pete Gallego logged more than 70,000 miles there in his white Chevy Tahoe, campaigning for re-election to the House — and lost by a bare 2,422 votes.

So in his bid this year to retake the seat, Mr. Gallego, a Democrat, has made a crucial adjustment to his strategy. “We’re asking people if they have a driver’s license,” he said. “We’re having those basic conversations about IDs at the front end, right at our first meeting with voters.”

Since their inception a decade ago, voter identification laws have been the focus of fierce political and social debate. Proponents, largely Republican, argue that the regulations are essential tools to combat election fraud, while critics contend that they are mainly intended to suppress turnout of Democratic-leaning constituencies like minorities and students.

In the third paragraph, we have two conflicting claims that go to the foundation of the whole debate. If election fraud is a significant problem, you can make a case for voter ID laws. If not, it's difficult to see this as anything other than voter suppression. This paragraph pretty much demands some additional information to help the reader weigh the claims and the article provides it...

More than twenty paragraphs later.

Mr. Abbott, perhaps the law’s most ardent backer, has said that voter fraud “abounds” in Texas. A review of some 120 fraud charges in Texas between 2000 and 2015, about eight cases a year, turned up instances of buying votes and setting up fake residences to vote. Critics of the law note that no more than three or four infractions would have been prevented by the voter ID law.

Nationally, fraud that could be stopped by IDs is almost nonexistent, said Lorraine C. Minnite, author of the 2010 book “The Myth of Voter Fraud.” To sway an election, she said, it would require persuading perhaps thousands of people to commit felonies by misrepresenting themselves — and do it undetected.

“It’s ludicrous,” she said. “It’s not an effective way to try to corrupt an election.”

I shouldn't have to say this but, if a story contains claims that the reporter has reason to believe are false or misleading, he or she has an obligation to address the issue promptly. Putting the relevant information above the fold is likely to anger the people who made the false statements, but doing anything else is a disservice to the readers.

When is the presumption of reasonableness reasonable?

From Wikipedia [emphasis added]

Since the first corps was established in 1990, more than 42,000 corps members have completed their commitment to Teach For America. In September 2015, the organization reached a milestone of 50,000 corps members and alumni, who have collectively taught more than 5 million students across the nation.

Unless I lost a zero somewhere, that comes to presumably not that much above 100 students per teacher. If we're just talking about the two years of official TFA service, that seems a low but not out of the question  if you had more elementary than secondary classes in the mix.

What bothers me is that, in order to get to a reasonable number, I have to assume that the writer meant something he or she didn't actually say. I have to change 

In September 2015, the organization reached a milestone of 50,000 corps members and alumni, who have collectively taught more than 5 million students across the nation.

to

In September 2015, the organization reached a milestone of 50,000 corps members and alumni, who collectively taught more than 5 million students across the nation during their two year commitments.

In this case, I think the change is reasonable because I don't find the alternative credible (specifically that the average career total of current and former TFA members is a little over one hundred taught). I am certainly open to changing my mind on this point as new evidence comes in, but, for now, I'm going to stick with the second version.

There is, however, a real danger in automatically assuming people meant something more reasonable than what they actually said, particularly when the people in question are not very honorable and are aware that you'll be shading things in their favor. Which brings us to this repost from 2012

_____________________________________________

Following up the follow-up

Following up on Joseph's latest, I actually think the problem here is more James Stewart than Paul Ryan. Ryan's budgets have been fairly obvious attempts to form a more Randian union. That's not surprising coming from an avowed follower of Ayn Rand. Ryan also comes from a Straussian tradition so I'm not exactly shocked that he would try to sell proposals that are likely to increase the deficit as a path to fiscal responsibility.

But that's OK. The Ryan plan is exactly the kind of bad idea that our national immune system ought to be able to handle. Liberals should savage its underlying values (Rand is always a hard sell); centrists and independents should spend their time pointing out the endless ways that the numbers don't add up and the evidence contradicts the basic arguments; respectable conservatives should damn it with faint praise or simply avoid the topic. The Republicans would then come back with a new budget, hopefully a proposal based on valid numbers and defensible assumptions, but at the very least one that obscures its flaws and makes a cosmetic effort at advancing its stated goals.

For Ryan's proposals to maintain their standing as serious and viable, the system has to have broken down in an extraordinary way. Specifically, the centrists such as James Stewart have had to go to amazing lengths to make the budget look reasonable, up to and including claiming that Ryan intends to take steps that Ryan explicitly rules out (from James Kwak):

Stewart is at least smart enough to realize that a 25 percent rate is only a tax increase if you eliminate preferences for investment income (capital gains and dividends, currently taxed at a maximum rate of 15 percent):

“Despite Mr. Ryan’s reluctance to specify which tax preferences might have to be curtailed or eliminated, there’s no mystery as to what they would have to be. Looking only at the returns of the top 400 taxpayers, the biggest loophole they exploit by far is the preferential tax rate on capital gains, carried interest and dividend income.”

So give Stewart credit for knowing the basics of tax policy. But he is basically assuming that Ryan must be proposing to eliminate those preferences: “there’s no mystery as to what they would have to be.”
Only they aren’t. Stewart quotes directly from the FY 2012 budget resolution authored by Ryan’s Budget Committee. But apparently he didn’t notice this passage:

“Raising taxes on capital is another idea that purports to affect the wealthy but actually hurts all participants in the economy. Mainstream economics, not to mention common sense, teaches that raising taxes on any activity generally results in less of it. Economics and common sense also teach that the size of a nation’s capital stock – the pool of saved money available for investment and job creation – has an effect on employment, productivity, and wages. Tax reform should promote savings and investment because more savings and more investment mean a larger stock of capital available for job creation.”

In other words, taxes on capital gains should not be increased, but if anything should be lowered.

These distortions aren't just journalistic laziness or rhetorically overkill on Stewart's part; it's essential to a narrative that writers like Stewart have built their careers on.

Here's Paul Krugman:

But the “centrists” who weigh in on policy debates are playing a different game. Their self-image, and to a large extent their professional selling point, depends on posing as high-minded types standing between the partisan extremes, bringing together reasonable people from both parties — even if these reasonable people don’t actually exist. And this leaves them unable either to admit how moderate Mr. Obama is or to acknowledge the more or less universal extremism of his opponents on the right.

The point about self-image and professional selling points is remarkably astute and when you combine those with the decline in fact-checking, diminishing penalties for errors, and a growing trend toward group-think, you get a journalistic system that loses much of its ability to evaluate policy ideas.

And for a democracy that's a hell of a loss.

Actually, Whole Foods are both high-price and high-volume

I have said before, I am in many ways generally sympathetic to the utopian urbanists. The problems they address are real and substantial and many of their proposed solutions make a lot of sense to me. That said, the movement, at least in the form that makes it into the popular press, often seems overly narrative-driven, romanticized,and derived and debated from a top decile viewpoint

Cities are all much nicer if you have money, and this is even more true in high density upscale places like New York and San Francisco. Unfortunately, many of the pieces you read advocating the charms of city living are written from the perspective of a six or seven-figure income. There is nothing this is nothing wrong with this perspective as long as the writers maintain a degree of self-awareness, but this is often not the case.

Recently, we've been hearing a lot of arguments claiming we can  get the costs of housing down to reasonable middle-class levels in places like NYC by aggressively building those cities up. Without that middle-class target, the argument would certainly be valid. With the target, the number of new housing units necessary would appear to be huge, particularly since many if not most of the specific proposals we've seen so far focus on upscale housing with relief for middle and lower class markets to come through trickle down effects.

But getting the prices down is only half the problem. We also to consider what living in these hyper-dense cities would look from the vantage of a median income, which brings us to this (with the usual caveats about quoting something you read in one of these pop sci sites).

Indeed, that’s what Colin Ellard, a neuroscientist at the University of Waterloo and director of its Urban Realities Laboratory, has found in his own work. Five years ago, Ellard became interested in a particular building on East Houston Street — the gigantic Whole Foods “plopped into” a notoriously textured part of lower Manhattan. As described in his book, titled Places of the Heart: The Psychogeography of Everyday Life, Ellard partnered with the Guggenheim Museum’s urban think tank to analyze what happens when someone “turns out of a tiny, historic [knish] restaurant” and encounters a full city block with nothing but “the long, blank façade of the Whole Foods Market.”

In 2011, Ellard led small groups on carefully planned Lower East Side walks to measure the effect of the urban environment on their bodies and minds. Participants recorded their response to questions at each stopping point and wore sensors that measured skin conductance, an electrodermal response to emotional excitement. Passing the monolithic Whole Foods, people’s state of arousal reached a nadir in Ellard’s project. Physiologically, he explained, they were bored. In their descriptions of this particular place, they used words like bland, monotonous, and passionless. In contrast, one block east of the Whole Foods on East Houston, at the other test site — a “lively sea of restaurants with lots of open doors and windows” — people’s bracelets measured high levels of physical excitement, and they listed words like lively, busy, and socializing. “The holy grail in urban design is to produce some kind of novelty or change every few seconds,” Ellard said. “Otherwise, we become cognitively disengaged.” The Whole Foods may have gentrified the neighborhood with more high-quality organic groceries, but the building itself stifled people. Its architecture blah-ness made their minds and bodies go meh.

First, as mentioned before, there are lots of reasons to worry about the homogeneity of the people conducting this research, making these proposals, and writing these articles.

With apologies for the snark, if you were to fund grants in the field of "first world problems," I can't imagine anything better than the psychological effects of the lack of architectural charm of a Whole Foods in the middle of a picturesque Manhattan street.This also brings up familiar social science concerns about unrepresentative populations and generalizing from outliers, but, putting all of that aside, let's  assume the results (which seem reasonable enough) are valid and see where they lead us.

Given the numbers being thrown around, it would seem that street-accessible retail, having limited capacity to build up, would largely be forced into one of two models: high price or high-volume.
Either way, retailers will have to make every expensive square foot pay for itself. We've already seen something like this in gentrifying neighborhoods where longstanding and often beloved mom-and-pop businesses are forced out to make way for chain stores and high-end boutiques. Barring some fairly draconian regulation (which would very much go against current conventional wisdom), it's hard to imagine the proposed hyper-dense cities not taking these trends to a new extreme.

Would this be a bad thing? That depends. I don't want to get all nostalgic about some neighborhood pizza joint (if anything, getting New Yorkers to stop going on about their neighborhood pizza places would be a national good). Antiquated and inefficient business model are supposed to go away. I'm not prepared to take a policy stand based on charm and sentimental appeal.

But, of course, I'm not a utopian urbanist. I've always been highly skeptical of these narratives, generally finding them to be overoptimistic and sometimes mutually contradictory. Increasingly dense cities are often held up as a panacea, curing all of our ills be they economic, environmental or cultural. That alone makes me nervous. Add on to that a romanticized, idyllic quality -- music in the cafes at night and innovation in the air – that has been seldom actually observed and is not at all the direction some of these policies seem to be headed.

At least not for the bottom nine deciles.

Damn, apparently Krugman has gotten to the College Humor staff

I was going to save this for later, but events seem to be picking up speed...



One of the things that impresses me about College Humor that, though they very much come from a millennial point of view, they are more than willing to satirize millennial culture.

The Monsanto House of the Future

More on the retro-future thread.

Maybe "thread" is too strong a word. So far, it has mainly been fragments. Hopefully, this is building up to a coherent thesis about attitudes toward technological progress, particularly about how the huge advances of the late 19th/early 20th centuries and the Post-War era lead people to internalize the idea that every aspect of the world was changing at an ever-accelerating pace. (I also want to tie in the closing of the Western frontier with the opening of the technological one, but that's definitely a topic for another day.)

I've argued that for almost a hundred years, technological progress reliably outpaced prediction while for the past thirty or forty, it's been the opposite. Though it's dangerous to pin these things down,1967 is around the time when our expectations started exceeding our advances.

From Wikipedia:

The Monsanto House of the Future (also known as the Home of the Future) was an attraction at Disneyland in Anaheim, California, USA, from 1957 to 1967. It was part of Disney's Tomorrowland

It was sponsored by Monsanto Company. The design and engineering of the house was done jointly by Monsanto, Massachusetts Institute of Technology, and Walt Disney Imagineering. The MIT faculty members were architects Richard Hamilton and Marvin Goody, and building engineer Albert G. H. Dietz. The fiberglass components of the house were manufactured by Winner Manufacturing Company in Trenton, New Jersey, and were assembled into the house on-site.

The attraction offered a tour of a home of the future, set in the year 1986, and featured household appliances such as microwave ovens, which eventually became commonplace. The house saw over 435,000 visitors within the first six weeks of opening, and ultimately saw over 20 million visitors before being closed.

The house closed in 1967. The building was so sturdy that when demolition crews failed to demolish the house using wrecking balls, torches, chainsaws and jackhammers, the building was ultimately demolished by using choker chains to crush it into smaller parts. The reinforced polyester structure was so strong that the half-inch steel bolts used to mount it to its foundation broke before the structure itself did.

For a firsthand account, check out this post by Ken Levine.

The “Prisoner's Dilemma” moratorium

[Perhaps the nicest thing about having a blog is being able to shoot off an angry post when a news story annoys you.]

I'm not sure what the best way to get the ball rolling here would be (perhaps a kickstarter?) but we need  to have a strictly enforced rule that no journalist or pundit is allowed to mention the prisoner's dilemma for the next five or ten years, however long it takes to learn to use it properly and, more importantly, discover that game theory consists of more than that one concept.

If we could just get writers to stop mistaking a stag hunt for prisoner's dilemma, it would be a massive improvement. I see this all the time and it drives me crazy.

Then there are ideas like Schelling focal points. As mentioned before, while most political commentators have had what can only be described as a humiliating season, a handful (notably Josh Marshall, Jonathan Chait and Paul Krugman) have actually enhanced their reputations. One of Krugman's best posts of the year was this game theory-based analysis of the over-reaction to the results in Iowa (more on my reaction here).

Not to name any names, but a lot of writers would be looking better now if they had spent more time thinking about focal points and less time claiming to see inflection points.

Remembering the St. Francis Dam

I'm not going to spend a lot of time pointing out the obvious. Anyone who's been following the pubic works debate should find something of interest in this Newsweek article:

Eighty-eight years ago, the St. Francis Dam burst in the middle of a March night, killing nearly 500 people [431 according to Wikipedia -- MP]. There are some images of the aftermath, but numbers tell the story better: 12.4 billion gallons of water rising to the furious height of 140 feet, surging 54 miles to the Pacific Ocean, an inland tsunami 2 miles wide leveling towns in its path. Some thought a saboteur had dynamited the dam. This would be easier to believe than the dam failing and people dying senselessly. But that was the case. And given the sorry state of American infrastructure, something similar could be the case again: the St. Francis Dam as portent, not aberration. ...

The dam burst on its sides, so that a strangely picturesque center section remained, standing there as a lone man might on a deserted train platform. Morbidly nicknamed “the Tombstone,” this vertical slab of concrete was dynamited to bits after a boy climbing the structure fell and died (another boy had thrown a snake at him). The stated reason for the demolition was public safety, but as Jon Wilkman wrote in his excellent book on the St. Francis Dam disaster, Floodpath, “it was a memorial to a failure the leaders of Los Angeles preferred to forget.”

"Likely not peer reviewed"

Occasionally gross but otherwise safe for work.



From the good people at College Humor.

Suspensions and calm-down rooms in Boston

If you've been following the no-excuses debate, you should definitely check out this story by Peter Balonon-Rosen.

Last year, state data show, the school imposed 325 suspensions overall. Massachusetts schools, on average, out-of-school suspended one in 33 students. UP Academy Holland out-of-school suspended one in 11.

...

Of Holland’s 233 in- and out-of-school suspensions so far this year, 117 were for first- and second-graders, according to the records supplied to WBUR. The school has about 250 students in those grades. Students with disabilities substantial enough to keep them out of regular classrooms were suspended 37 times, the records show.

...

The teacher pointed to the school’s philosophy of punishing even small infractions, like rolling their eyes, sucking their teeth or not sitting in “scholar ready position” as setting students off.

A few points before we go on. These suspension rates are the  result of a deliberate policy by the administrators of these schools. If the CEO (yes, they have a CEO) of UP Education Network (formerly "Unlocking Potential" and I'll stop with the parentheses now) wanted to cut suspensions in half, he could do it with a conference call. That call hasn't been made yet because, though public pressure is starting to build, it still does not outweigh the benefits for the administrators.

Of course, there is also a regulatory component, particularly in cases involving the disabled.

Those processes include notifying parents before the student is suspended, holding a hearing on the suspension and, in certain cases, determining whether the student’s disability caused the behaviors, in which case suspension is forbidden. By state law, parents can appeal the suspension to the district superintendent.

For instance, Boston Public Schools’ Code of Conduct lists specific measures: Before any suspension, school staffers must document that they’ve tried discipline that keeps the student in class. Principals must notify the district superintendent in writing before any suspension of a student in kindergarten through third grade. And a student’s parents or guardians can appeal the suspension to Boston Superintendent Tommy Chang.

...

But UP Academy Holland, while still considered a public school, no longer reports to the superintendent of Boston Public Schools. So, rather than notifying Chang, principals are told to notify Given, the UP CEO. And if parents want to appeal, they would appeal to the CEO, not to a public official.

Calm-down rooms (which we discussed previously) are also present, along with the obligatory claim that the are seldom used.

Jayden was sent to one of the school’s “calm-down rooms.” A first-floor calm-down room is a former storage closet, still labeled “STORAGE” on an outside sign, that’s been turned into a dedicated space for timeouts.

Sometimes students stay in there alone while a staff member waits outside. The door has a small window for observation, although not every corner is visible through it. If a student is in the room longer than half an hour, the staff must notify the principal.

Malikka Williams’ son Malik attended UP Academy Holland in kindergarten. She remembers the first time she saw one of the calm-down rooms.

“Tears just started coming down my eyes, because it reminded me of a hospital ward room for psychiatric,” says Williams. “And I remember at that moment I said, ‘My God, this is not OK.’ ”

School administrators say they put students there only when they pose a danger to themselves or others. Principal Peddie says it allows students to calm down.

“We give students the space and the opportunity to self-regulate and really put themselves in a position where they feel as if they can be successful,” he says.

Quick note on the word "successful." This is a small thing but it illustrates a distinctive trait of the education reform movement. The language of the movement is very much modern corporate-speak, relentlessly aspirational. Certain words such as "success" are repeated with such frequency that they start to lose meaning.

Saying that a crying, traumatized six-year-olds should "put themselves in a position where they feel as if they can be successful” is entirely in keeping. It also shows a tendency to treat small children as components and statistics, not as people.

Eventually, Williams says, she was getting texts and calls almost every day to pick up Malik. More often than not, it wasn’t a formal suspension, just a demand that she come take him out of school. Sometimes, she says, a staff member told her that if she didn’t take Malik, they’d call EMS to do it.

“It got to the point that my phone would ring and my nerves became shot,” says Williams. “I do feel that through the numerous suspensions, calls and emergency removal threats, that you were pushing my son out.”

In January, Malik transferred to another school. Since then, he has not been suspended once.

“My son, when we pick him up, he runs to the car…,” Williams says with tears in her eyes. “He says, ‘Mommy, I had an excellent day!’ I’m so happy. Happy he’s happy.”

Good educators always strive for more excellent days.

A missing data parable that links to secondary data use

This is Joseph.

Thomas Lumley points out a terrifying example of mean imputation gone terribly wrong. 

It turns out that the source of this problem was the algorithms from a private corporation that mapped out zip code level data, coupled with the risks of secondary data use by people who don't understand what the data was collected to do:

Earlier this week, I reached Thomas Mather, a co-founder of MaxMind, via email. I told him Joyce Taylor’s story, and how I’d discovered MaxMind’s involvement in the IP mapping part of it. I asked him if he knew anything about the default coordinates that were placing unidentified IP addresses on the Taylor’s property.

Mather told me that “the default location in Kansas was chosen over ten years ago when the company was started.”
 

He continued: “At that time, we picked a latitude and longitude that was in the center of the country, and it didn’t occur to us that people would use the database to attempt to locate people down to a household level. We have always advertised the database as determining the location down to a city or zip code level. To my knowledge, we have never claimed that our database could be used to locate a household.”

This is a common problem with big data projects in which you attempt to repurpose a data source for something other than what it is intended for.  And this re-use had some pretty severe consequences -- the people living at the "missing data address" were visited by a lot of officials -- all relying on the addresses that the IP address was registered to. 

The new plan, to put the defaults in the middle of a lake, should result in less pain for the residents of the unfortunately located property.  I am curious if we will soon see divers at that lake looking for fraudsters, though. 

But the secondary lesson of this story (above and beyond mean imputation rarely being an optimal missing data strategy) is just how risky it can be use data for an unintended purpose without understanding just how that data arises and what the limitations actually are. 

It still counts as commuting when you do it in an airplane

In all of these posts about urban development, there is an underlying idea that I should probably spell out explicitly more often. My concern here is not so much that the urbanists are wrong, but that this and many other public policy discussions are being conducted under extremely dangerous conditions.

This is a hugely complicated issue but the major points that worry me can be broken down to four or five basic categories (depending on how you split them):

The overwhelming majority of the people conducting this discussion come from a remarkably homogeneous group (economically, geographically, educationally, and culturally). Furthermore, this group lands on the far end of the spectrum on any number of relevant dimensions. This invariably leads to distortions and blind spots;

This discussion has come to be dominated by a simple and elegant narrative with notable utopian elements. Not coincidentally, this narrative is remarkably appealing to the group conducting the discussion;

Much, if not most, of the supporting evidence for the narrative comes from a handful of outliers such as the Bay Area. What's more, the great success of some of these outliers may have more to do with having been in the right place at the right time for various industry booms than with the cities having pursued any particular policy.;

There is also a morality play at work here. This is often unavoidable in public policy debates but it becomes very dangerous when the judging is done by a homogeneous and insular group and the focus is on the sins of those on the outside.

Which brings us to our case in point. The worker who lives 25 miles outside of St. Louis, Missouri and commutes to work is seen as engaging in wasteful and environmentally costly behavior, but the Manhattan-based executive who flies once or twice a month to cities on the West Coast is not, despite the trip being roughly one hundred times longer and the mode or travel not being by any stretch of the imagination green.

From Duncan Clark [emphasis added]

The new paper, published in the journal Environmental Science and Technology, finally pins some numbers on all this theory by examining the impact over different time periods of various different modes of transport. The results are illuminating.

According to the paper, if we focus just on the impact over the next five years, then planes currently account for more global warming than all the cars on the world's roads – a stark reversal of the usual comparison. Per passenger mile, things are even more marked: flying turns out to be on average 50 times worse than driving in terms of a five-year warming impact.

If we shift to a 20-year time frame, things look completely different. The short-term impacts have largely died down and the plane looks considerably better – helped along by a quirk of atmospheric chemistry which sees nitrous oxide pollution from the aircraft engines causing cooling during this period by destroying methane in the air. The paper even suggests that for any time frame longer than 20 years, flying is typically greener per kilometre than driving (although when I phoned to check this, one of the authors of the report confirmed my suspicion that this isn't true in Europe, where fuel-efficient cars are more popular).

If we compare by miles traveled and we assume the average automobile fuel efficiency remains constant,  air travel is much worse than auto travel in the short term and slightly better in the long term, but both of those assumptions are questionable.


In an age of hybrids, plug-ins and increasingly viable electrics, it should just take time or a good regulatory push to get us to where Europe is. On the question of miles traveled... As we mentioned in a previous discussion of drunk driving, using per-mile comparisons when discussing modes of transportation with wildly different ranges is problematic at best. The decision to travel long distances and the decision to go by air are closely related and the causality goes both ways.

It would be easy to get mired in the details, but we can be fairly sure that a sharp reduction of air travel would encourage people to look for closer substitute destinations or to eliminate trips entirely (in the 21st Century, there is little reason for insisting that people be physically present for a meeting). Cutting back on air travel would certainly seem to be the environmentally sensible thing to do, but for this post, I'm less interested in going green than in going meta.Is the larger discussion healthy and productive.

Homogeneity is not always a bad thing. Sometimes a group of people of like backgrounds and similar minds can converge on a common vision and produce something wonderful and innovative. In these cases, isolation and even insularity can help get things started. (Chicago was a theatrical backwater before the Post-War renaissance.)  At some point, though, there has to be cross-fertilization. Otherwise conjectures become implausible, judgements become biased and ideas become stale.

Following the debate on urban renewal, food stamps, tax policy, education reform, etc. I constantly notice, not only that a plurality and possibly a majority of stories seem to come from the same perspective (top quartile, Ivy League, upscale neighborhood in a high-density city), but that the writers have no idea how unrepresentative their experiences (including bicoastal living)can be.