Comments, observations and thoughts from two bloggers on applied statistics, higher education and epidemiology. Joseph is an associate professor. Mark is a professional statistician and former math teacher.
Wednesday, June 29, 2011
If you're serious about education reform...
From Mike Konczal:
Thursday, June 23, 2011
Adoption studies as Rorschach test
On Mr. Mankiw’s specific point, though, not all economists have the same view of the research on parents that he does. Bruce Sacerdote at Dartmouth has done one of the most-cited studies, and it finds that parents can make an enormous difference. From the abstract:I look at this and see a cautionary tale about the perils of drawing inferences from non-representative samples, but I'm a statistician and that's how we see most things.
I analyze a new set of data on Korean American adoptees who were quasi-randomly assigned to adoptive families. I find large effects on adoptees’ education, income and health from assignment to parents with more education and from assignment to smaller families. Parental education and family size are significantly more correlated with adoptee outcomes than are parental income or neighborhood characteristics. Outcomes such as drinking, smoking and the selectivity of college attended are more determined by nurture than is educational attainment.
Another data dump
Felix Salmon points out another one of those things that make me nervous about the NYT.
Salmon also directs us to another great piece from the New Yorker.
Jonathan Chait has some smart things to say about the asymmetry of the political debate.
(Blogger's spell check just stopped working. The rest of the post may be a challenging read.)
Motorola's example doesn't really generalize. GE's run was better explained by other factors. Are there any Six Sigma success stories that don't fall in these two categories?
Dana Goldstein continues to bring objectivity and balance to the education debate.
Tuesday, June 21, 2011
more thoughts on geoengineering
When you have a plan that covers ocean acidification, maybe you can talk me out of supporting a carbon tax.
Ocean life on the brink of mass extinctions: study
OSLO (Reuters) – Life in the oceans is at imminent risk of the worst spate of extinctions in millions of years due to threats such as climate change and over-fishing, a study showed on Tuesday.
Time was running short to counter hazards such as a collapse of coral reefs or a spread of low-oxygen "dead zones," according to the study led by the International Programme on the State of the Ocean (IPSO).
"We now face losing marine species and entire marine ecosystems, such as coral reefs, within a single generation," according to the study by 27 experts to be presented to the United Nations.
"Unless action is taken now, the consequences of our activities are at a high risk of causing, through the combined effects of climate change, over-exploitation, pollution and habitat loss, the next globally significant extinction event in the ocean," it said.
Scientists list five mass extinctions over 600 million years -- most recently when the dinosaurs vanished 65 million years ago, apparently after an asteroid struck. Among others, the Permian period abruptly ended 250 million years ago.
"The findings are shocking," Alex Rogers, scientific director of IPSO, wrote of the conclusions from a 2011 workshop of ocean experts staged by IPSO and the International Union for Conservation of Nature (IUCN) at Oxford University.
Fish are the main source of protein for a fifth of the world's population and the seas cycle oxygen and help absorb carbon dioxide, the main greenhouse gas from human activities.
Jelle Bijma, of the Alfred Wegener Institute, said the seas faced a "deadly trio" of threats of higher temperatures, acidification and lack of oxygen, known as anoxia, that had featured in several past mass extinctions.
A build-up of carbon dioxide, blamed by the U.N. panel of climate scientists on human use of fossil fuels, is heating the planet. Absorbed into the oceans, it causes acidification, while run-off of fertilizers and pollution stokes anoxia.
Extraordinary interview with Eric Sterling
Though we generally don't think about it in these terms, the war on drugs should be part of the education debate. The communities on the wrong side of the achievement gap are also the ones that have paid the price for congress's pathetic need to sound tough a quarter century ago. Here's a representative sample:
STERLING: The amounts ended up being very, very small, instead of a high-level quantity...
CONAN: Because the idea of the law, originally, was to go after kingpins.
STERLING: Exactly. We - the Justice Department had very broad discretion, and we recognized that the federal should be focused on the highest-level traffickers. The first proposal that we had used data that the DEA had suggested in terms of how they evaluate, internally, their highest-level traffickers. Those numbers were objectionable to a congressman from Louisville, Kentucky, who said: We'll never use this law in Louisville. And unfortunately, no one, given the speed of this, said: But congressman, Louisville is not Holly - Miami. It's not the center of the drug trade. Nobody goes to Louisville to do a major cocaine deal. Of course, we don't need it.
Monday, June 20, 2011
Bond villains can be so hard to shop for
Now their feelings are hurt
If you want a great example of the kind of mean things that people are saying about Groupon in the run-up to its IPO, you could do a lot worse than Rocky Agrawal’s TechCrunch essay entitled “Why Groupon Is Poised For Collapse”. It’s a great example of overstretch and dubious logic, with a couple of moments of brilliance and genuine insight thrown in at the same time. Groupon, of course, being in its quiet period, can’t react. Except, it just can’t help itself, and has put up a whiny post, supposedly authored by the company cat, about how unfair the whole situation is.After reading the post in question, 'whiny' is probably the first adjective that will come to your mind as well. It's easy to understand how Mason, Lefkofsky and company might feel wounded. A few months ago, between the investor buzz and love letters from the Wall Street Journal, the people at Groupon could easily assume that they had a world full of friends. Now everyone's pointing out problems in your business model and asking why your marketing techniques seem twenty years out of date.
The fact is that when Groupon made the decision to go public, it invited exactly this kind of attention — both before the IPO and forever more. When Groupon was private, no one really knew anything about its financials, and CEO Andrew Mason could happily declare that he’d much rather talk about building miniature dollhouses. Once it’s public, however, he’ll have a fiduciary responsibility to his shareholders, and will have to answer such questions at length. Will that make him happier than answering such questions with a death-ray stare? I doubt it, to be honest. Revenues and business models and profits and forecasts are serious things, and you can’t kid around with shareholders in the same way you can with journalists.
In other words, Mason will have to go from saying nothing, which can be fun, to saying something, which almost certainly won’t be. Rather than moan about his inability to say anything in the quiet period, he should enjoy it while it lasts. From now on in, the boring financial questions are going to be unavoidable — from analysts, from journalists, from shareholders, even probably from merchants and customers who wonder whether Groupon’s profitability is a sign that they’re being ripped off.
If your BS detector isn't going off, you need to check your battery
Take, for example, their positive review of this Economist piece on IBM's hundredth birthday. Here's the key observation from the Economist:
IBM’s secret is that it is built around an idea that transcends any particular product or technology. Its strategy is to package technology for use by businesses. At first this meant making punch-card tabulators, but IBM moved on to magnetic-tape systems, mainframes, PCs, and most recently services and consulting. Building a company around an idea, rather than a specific technology, makes it easier to adapt when industry “platform shifts” occur.The article, which goes on to argue that Apple, Amazon and Facebook live up to this while Dell, Cisco and Microsoft do not, is a nearly perfect example of perhaps the most popular genre of business writing, the secret of success story.
True, IBM’s longevity is also due, in part, to dumb luck. It almost came unstuck early on because its bosses were hesitant to abandon punch cards. And it had a near-death experience in 1993 before Lou Gerstner realised that the best way to package technology for use by businesses was to focus on services. An elegant organising idea is no use if a company cannot come up with good products or services, or if it has clueless bosses. But on the basis of this simple formula—that a company should focus on an idea, rather than a technology—which of today’s young tech giants look best placed to live to 100?
Here's the template:
1. Come up with a positive-sounding platitude like "successful businesses are obsessed with quality" or "a company should focus on an idea, rather than a technology";
2. Make a short list of companies that are currently hot. You may also choose to make a list of not-hot companies for the purpose of contrast. It is not necessary for the hot companies actually to be more successful;
3. Claim that your platitude explains the success or failure of these companies. This will usually entail stepping over some rather deep holes. For example, the Economist explains IBM's 100 year run with an insight* the company's management supposedly had in 1993. (If you have the nagging feeling that you've heard this basic argument before, you're probably right).
In real life, it's next to impossible to find a secret-to-success that is both widely applicable and pretty. This isn't to say that you can't learn something useful by studying successful companies, but those lessons tend to be complicated and difficult to apply in other settings. When you do find one that is simple and general, it's usual something like 'you can overcharge customers if you keep your pricing opaque.'
Somehow, though, "you can fool some of the people enough of the time" doesn't have the inspirational quality for a good business slogan.
* If at all possible, work a visionary CEO's brilliant insight into the story somewhere.
Some confusion with the concept of future tense
Even as it issued the disclaimer that its report was not intended to be predictive, McKinsey linked to the report with the following teaser in its new statement: "The shift away from employer-provided health insurance will be vastly greater than expected and will make sense for many companies and lower-income workers alike."
Weekend Blogging -- Free TV edition
ThisTV has caught on to the fact that the most interesting films are often on the far ends of the spectrum and has responded with a wonderful mixture of art house and grind house. Among the former, you can see films like Persona, the Music Lovers and Paths of Glory. Among the latter you'll find American International quickies and action pictures with titles like Pray for Death. You can even find films that fit into both categories like Corman's Poe films or Milius' Dillinger.
Sunday, June 19, 2011
"Just when I thought I was out... they pull me back in." -- another Groupon post (merchant relations edition)
For the latest nail in this coffin, Felix Salmon sends us to this extraordinary post by Benjamin Edelman and Paul Kominers:
Voucher services typically seek to cast themselves as mere marketing vendors that are not responsible for the conduct of the corresponding merchants. For example, Groupon’s Terms of Sale claim that “The Merchant, not Groupon, is the seller of the Voucher and the goods and services and is solely responsible for redeeming any Voucher you purchase.” On this view, a voucher service avoids liability for merchants’ shortfalls.
But a voucher service is the merchant of record for the charge to the customer’s credit card. As the entity officially responsible for charging the consumer, the voucher service thus faces increased responsibility to see that the consumer receives what was promised. Furthermore, the voucher service, not the merchant, writes the promotional text touting the merchant’s offering. As Rakesh Agrawal points out, Groupon’s financial disclosures even count the entirety of the consumer’s purchase price as revenue to Groupon. In this context, a consumer naturally looks to a voucher service for assistance if a merchant fails to perform. We think it is probably an unfair and deceptive practice, under the FTC Act and state equivalents, for voucher vendors to attempt to disclaim liability in such circumstances.
More generally, we are struck by Groupon’s attempts to push all responsibility to merchants. On every relevant question — discounting alcohol, honoring expiration dates, providing cashback — Groupon’s historic contract and current Merchant Terms of Service claim merchants are responsible. In our view, this approach invites confusion and non-compliance. Voucher services are far better positioned than merchants to determine what the legal system requires: Voucher services can research regulations centrally, once for each state in which they operate, then notify affiliated merchants of applicable requirements. In contrast, Groupon’s current approach asks each individual merchant to conduct its own research. If merchants actually conducted such research, it would be duplicative and potentially wasteful — thousands of small businesses re-researching the same questions. But in fact merchants typically ignore the questions, rationally concluding that these questions are too difficult for them to address on their own. Thus, by pushing merchants do to the work individually, voucher services virtually assure that the work is not done at all.
Importantly, the legal and regulatory questions flagged in this article are questions that arise distinctively in the context of discount vouchers: a merchant would never confront such questions were it not for discount vouchers. Having created the transactions giving rise to this regulatory complexity, we think discount voucher services should be expected to achieve compliance.
Wednesday, June 15, 2011
So much for maintaining a light tone over the weekend
Americans are living longer, but not in every corner of the country. A new study shows that in hundreds of U.S. counties — mostly in the South — life expectancy has fallen.
The researchers believe problems like smoking and obesity are partly to blame.
"There are enormous variations within the country" said Dr. Christopher Murray, a University of Washington researcher. He's a study author and an editor of the online journal, Population Health Metrics, which released the study Wednesday.
Overall, life expectancy in the U.S. is at an all-time high. The Centers for Disease Control and Prevention recently estimated that a baby born in 2009 could expect to live 78 years and 2 months.
The CDC doesn't calculate estimates by county; Murray's research covers 2000 through 2007 when U.S. life expectancy grew a year to nearly 78.
A federal expert in these kinds of statistics said Murray's methods were sound, but the findings aren't terribly surprising.
The U.S. estimate actually dropped from 2004 to 2005, noted Bob Anderson of the CDC's National Center for Health Statistics. Given that downward blip — and the fact that statistics fluctuate more when you're dealing with smaller populations __ it's not unexpected to see some declines at the local level, he said.
The study found that life expectancy for women fell significantly in 702 of the nation's more than 3,100 counties. The largest declines — by nearly 2 years — were in Mississippi's Madison County, near Jackson, and the adjacent Hughes and Okfuskee counties in eastern Oklahoma.
"How To Party Your Way Into a Multi-Million Dollar Facebook Job" -- the sad state of business journalism
If you want Facebook to spend millions of dollars hiring you, it helps to be a talented engineer, as the New York Times today [18 May 2011] suggests. But it also helps to carouse with Facebook honchos, invite them to your dad's Mediterranean party palace, and get them introduced to your father's venture capital pals, like Sam Lessin did.
Lessin is the poster boy for today's Times story on Facebook "talent acquisitions." Facebook spent several million dollars to buy Lessin's drop.io, only to shut it down and put Lessin to work on internal projects. To the Times, Lessin is an example of how "the best talent" fetches tons of money these days. "Engineers are worth half a million to one million," a Facebook executive told the paper.
We'll let you in on a few things the Times left out: Lessin is not an engineer, but a Harvard social studies major and a former Bain consultant. His file-sharing startup drop.io was an also-ran competitor to the much more popular Dropbox, and was funded by a chum from Lessin's very rich childhood. Lessin's wealthy investment banker dad provided Facebook founder Mark Zuckerberg crucial access to venture capitalists in Facebook's early days. And Lessin had made a habit of wining and dining with Facebook executives for years before he finally scored a deal, including at a famous party he threw at his father's vacation home in Cyprus with girlfriend and Wall Street Journal tech reporter Jessica Vascellaro. (Lessin is well connected in media, too.) . . .
To get the full impact, you have to read the original New York Times piece by Miguel Helft. It's an almost perfect example modern business reporting, gushing and wide-eyed, eager to repeat conventional narratives about the next big thing, and showing no interest in digging for the truth.
Let's start with the credibility of the story. While computer science may well be the top deck of the Titanic in this economy, has the industry really been driven to cannibalization by the dearth of talented people? There are certainly plenty of people in related fields with overlapping skill sets who are looking for work and there's no sign that the companies like Facebook are making a big push to mine these rich pools of labor. Nor have I seen any extraordinary efforts to go beyond the standard recruiting practices in comp sci departments.
How about self-interest? From a PR standpoint, this is the kind of story these companies want told. It depicts the people behind these companies as strong and decisive, the kind of leaders you'd want when you expect to encounter a large number of Gordian Knots. When the NYT quotes Zuckerberg saying “Someone who is exceptional in their role is not just a little better than someone who is pretty good. They are 100 times better,” they are helping him build a do-what-it-takes-to-be-the-best image.
The dude-throws-awesome-parties criteria for hiring tends to undermine that image, as does the quid pro quo aspect of Facebook's deals with Lessin's father.
Of course, there's more at stake here than corporate vanity. Tech companies have spent a great deal of time and money trying to persuade Congress that the country must increase the number of H-1Bs we issue in order to have a viable Tech industry. Without getting into the merits of the case (for that you can check out my reply to Noah Smith on the subject), this article proves once again that one easily impressed NYT reporter is worth any number of highly paid K Street lobbyists.
The New York Times is still, for many people, the paper. I've argued before that I didn't feel the paper deserved its reputation, that you can find better journalism and better newspapers out there, but there's no denying that the paper does have a tremendous brand. People believe things they read in the New York Times. It would be nice if the paper looked at this as an obligation to live up to rather than laurels to rest on.
Tuesday, June 14, 2011
More bad statistics from Freakonomics
Freakonomics Radio is not helping Marketplace's average.
Dubner: Let me let you hear from a different economist, his name is Bruce Sacerdote, he's at Dartmouth. He wanted to know how much this kind of activist-parenting -- if you want to call it that -- actually pays off. And one way to measure this, especially if you're talking about educational achievement -- which is what parents probably care about the most -- is to look at adoption studies, where you can actually measure the impact that a family, that the parents, will have on a kid.
Ryssdal: So what's his thesis, that kids adopted into, I guess, high-education homes will be more likely to go to college, is that the deal?
Dubner: Exactly right. If parents are so important, then parents can take an adopted kid who might otherwise not have gone to college, and that kid will become college material. So Sacerdote sliced and diced a lot of good data and he did find parental influence.
Bruce Sacerdote: But it's not quite as big as I expected to find.
Ryssdal: All right, so quantify for me: how big is not so big?
Dubner: If you're a child who's adopted into a high-education family -- that is where the parents both went to college -- you are about 16 percentage points more likely to go to college than a kid who gets adopted into a low-education family. So that sounds pretty good, OK?
Until you compare that to the rate for biological kids from high-education families, who are about 75 percentage points more likely to go to college than biological kids from low-education families. So on the one hand, this is a little dispiriting for parents. We don't seem to have as much influence as we might think. On the other hand, in a weird way, it kind of takes some of the pressure off, right? At least it did for Bruce Sacerdote.
Sacerdote: This notion that genes are really important and that kids are hard-wired to do certain things, I think understanding that did help me relax and not worry so much that I was going "screw them up" in some terrible way.
One of the concerns I have about sending an economist to do a statistician's job is that economists, by training, have a tendency to make things simpler.* This works reasonably well in economics where the problems are generally too complex to tackle without making some simplifying assumptions about linearity and additive effects and independence and where (perhaps more importantly) the economists have a pretty good idea of which corners can be safely cut. Unfortunately, when they move outside of familiar ground, you often get something like this.
Unless something important has been left out (always a possibility when something is presented to a general audience), Sacerdote has made the common but serious mistake of treating a select group as a random sample. People who choose to adopt are not a random sample of parents, particularly given the difficulties in the process, and those who do self-select have to go a difficult approval process.
Think for a moment about how unrepresentative the resulting sample is. Now consider traits that are likely to be correlated to both parents' and children's education levels and which the process is likely to select for or select out. Here are a few that come immediately to mind:
Career level (high income may not be a requirement but financial stability is and these days that represents a significant career accomplishment);
Encouragement and support**;
Educational expectations for their children (I can't imagine that all that many prospective parents who don't give a damn about their kid's education make it to the end of the process).
The genetic component of this question is confounded with all these things and more. As a result, (at least as presented here) Sacerdote actually succeeds in proving the opposite of his conclusion. His example doesn't show what, if any, role genetics plays in how likely a kid is to go to college. That 75 could be mainly the result of poverty (which is highly correlated to parent's education levels). I'm not saying that it is but there is no way of ruling out that and other explanations.
What Sacerdote has shown is that even when you control not only for genetics but also for most obvious environmental factors, you still got a 16 point bump just from having adoptive parents who both went to college. Under any circumstances this would be a substantial improvement but in the context of this natural experiment it's huge.
At the risk of over-sharpening the blade, we have here a natural experiment that shows nothing about the role of genetics in academic achievement. What it does show is that even when you greatly reduce most of the obvious parenting-related environmental sources of variability, you still get a substantial effect from the remaining factors. A highly respected researcher then uses this example to argue publicly that genetic factors far outweigh the impact of parenting.
We really need to do something about Freakonomics.
(I'm also not comfortable with the way Sacerdote gives a percentage difference rather than the ranges but that's a subject for another post)
* There are, of course exceptions. Most notably Nate Silver.
** I'm pretty sure in most low education adoptive homes you won't see this relationship:
Hart and Risley also found that, in the first four years after birth, the average child from a professional family receives 560,000 more instances of encouraging feedback than discouraging feedback; a working- class child receives merely 100,000 more encouragements than discouragements; a welfare child receives 125,000 more discouragements than encouragements.
Is Megan McArdle abandoning the Chicago School?
People really underweight the role that norms play in sustaining a modern economy. I suspect that if the "default option" folks got their way and people started regarding default as a commercial decision with no more moral weight than changing cable vendors, the advocates of this position would be unpleasantly surprised to find that the only thing less fun than being young and burdened with student loans is being young and completely unable to access any form of credit at all.My first corporate job was building models to decide who got unsecured loans and, based on that experience, I'm pretty sure she's wrong in the particulars. A change in attitudes would affect default rates somewhat but there are a number of powerful external incentives here that would limit the impact of a new set of norms.
I agree, however, with McArdle's larger point, which is strange, since I don't believe that McArdle agrees with it herself.
As noted before, freshwater economics is essentially a reductionist approach that relies heavily on a number of simplifying assumptions. If you claim that social norming plays a significant role in the economy, then you have to allow for local optima and plateaus and collective action problems and all sorts of behavior that is not rational in the economic sense and which will not average out when you look at things in the aggregate. That way lies madness and perhaps even Keynesianism.
It's possible that McArdle is starting to question what she learned at the feet of her Booth School professors, but I think it's more likely that she's willing to engage in a bit of intellectual inconsistency to reach her desired conclusion.
Some reflections on student loan debt
1) It is very hard to settle student loan debt and the lenders have ruthless policies to singificnatly increase what you owe
2) There is a problem with a social norm that makes default a trivial event
That being said, I must admit that the elephant in the room is probably the increasingly draconian state of US debt law. The stories of police raids over student loans (actually not owed by the person being put in handcuffs) may be missing important facts but the general picture is not pleasant.
But, at a more prosaic level, the inability to fail has a lot of serious issues. I have little sympathy for Elie Mystal , per se, but I do see a broad culture of making things worse when one experiences adverse life events. If one should end up unemployed they may well have no health insurance, limited access to bankruptcy for large debts (student loans), and have issues with basic necessities.
I am all for civic virtue and personal responsibility. These are both things that I wish we had more of. But I worry that we are creating a culture of cascading failures where one thing going wrong can set off a series of disasters. In a world with long term unemployment, is this really a good direction to be going?
What fun is it firing people if you don't get to humiliate them too?
Rhode Island does, however, offer a bright side for educators and job security. If you can make it past principal, you seem to be safe. Take Frances Gallo and Victor Capellán, two administrators overseeing a school district so small it only includes one high school, one middle school and a few elementary schools. Despite this high administrator to school ratio, even the most basic aspects of administration (such as scheduling substitutes) are routinely screwed up.
Gallo and Capellán admit the schools that they are responsible for are terrible, but they don't hold themselves responsible. If you can't guess who they do blame, you haven't following the education reform movement.
Monday, June 13, 2011
Some products simply demand a targeted approach
Even the most die-hard users admit to having bought some turkeys. Kris Kendall, a marketing manager in Freemont, Calif., has been labeled a "Groupon pimp" by her buddies, since she buys several offers every week and is constantly emailing her friends about terrific deals. But the occasional missteps have cost her serious money.
... "I've also let a few expire because I just never had the mojo to go -- namely, a pole-dancing workout class which I bought for a bunch of my friends, and then we never actually went."
Sunday, June 12, 2011
Weekend Gaming -- in memoriam/elegant games
Toy Designer Ned Strongin Dies at 91
Strongin was known for designing and licensing several award-winning toys, including Connect Four, Weebles, Giggle Wiggle and the Wiggly Giggly ball. Connect Four, his most commercially successful invention, remains one of Hasbro's best-selling games.If you're not familiar with the Connect Four, here's the introduction from Wikipedia:
"Ned was regarded in the toy and game industry as one of the three post-World War II 'fathers' of external toy and game inventions, along with Marvin Glass and Eddy Goldfarb," said Phil Orbanes, president of Winning Moves Games. Orbanes worked with Strongin to license Giggle Wiggle to Hasbro Games.
Connect Four (also known as Four Up, Plot Four, Find Four, Four in a Row, and Four in a Line) is a two-player game in which the players first choose a color and then take turns dropping their colored discs from the top into a seven-column, six-row vertically-suspended grid. The pieces fall straight down, occupying the next available space within the column. The object of the game is to connect four of one's own discs of the same color next to each other vertically, horizontally, or diagonally before one's opponent can do so.It's an entertaining and genuinely challenging game and it illustrates the most fascinating attributes of what I've called elegant games: a simple rule or condition (in this case, that each column must be filled from the bottom up) can greatly increase the complexity and subtlety of a game.
A Groupon postscript
After that, it gets even more interesting.
FORTUNE -- "Lets start having fun... lets get funky... let's announce everything... let's be WILDLY positive in our forecasts... lets take this thing to the extreme... if we get wacked [sic] on the ride down-who gives a shit... THE TIME TO GET RADICAL IS NOW... WE HAVE NOTHING TO LOSE..."This is a quote from the dot-com era. It's pretty much what you'd expect a novice executive to say back then, when it was all about money and not at all about creating something good. It was written in an email by the co-founder of a company called Starbelly.com, which labeled itself a B2B provider -- back when people greeted that phrase with a straight face.
In early 2000, Starbelly sold itself to another company called Ha-Lo Industries for $240 million, much of which went to the author of those words, a man named Eric Lefkofsky. Not long after that transaction, Ha-Lo declared bankruptcy. Shareholders and others blamed the Starbelly deal, and a series of lawsuits ensued.
More on McKinsey
The predictable fallout led Democrats, and several reporters, to press McKinsey for the survey itself -- a request McKinsey has declined on the grounds that the material is proprietary.
But multiple sources both within and outside the firm tell TPM the survey was not conducted using McKinsey's typical, meticulous methodology. Indeed, the article the firm published was not intended to give the subject matter the same authoritative treatment as more thorough studies on the same topic -- particularly those conducted by numerous think tanks, and the Congressional Budget Office, which came to the opposite conclusion. And that's created a clamor within the firm at high levels to set the record straight.
"This particular survey wasn't designed in away that would allow it to be peer review published or cited academically," said one source familiar with the controversy.
All sources were granted anonymity, in order to be able to speak candidly about the controversy.
Reached for comment today, a McKinsey spokesperson once again declined to release the survey materials, or to comment beyond saying that, for the moment, McKinsey will let the study speak for itself. However, McKinsey notes that the survey is only one indicator of employers' potential future actions -- that the conclusions remain uncertain and employers' future decisions will ultimately depend on numerous variables. The three authors of the report were not immediately available for comment.
Another keyed-in source says McKinsey is unlikely to release the survey materials because "it would be damaging to them."
Both sources disagree with the results of the survey, which was devised by consultants without particular expertise in this area, not by the firm's health experts.
A third source speculates that the firm may have reached its outlying conclusion by basing its questions on the firm's own advice to clients on how best to arbitrage the new reforms. Specifically, under the law, employers could devise their benefits packages in ways that makes them unappealing to lower-income employees, who would then have to enter the exchanges. Though TPM could not confirm this, the conclusion is supported by a disclosure within the McKinsey study itself.
Saturday, June 11, 2011
Reporting on methods
As someone who does research, this really bothers me. It should bother you too. Look, anybody can say what they like on a topic. They can put out a glossy report. They can claim they did a “survey” to make it sound scientifically rigorous. They can talk to the media all about it. They can stand behind their good name and reputation, if they have one. But when what they’re saying runs counter to previous experience and other credible estimates, they’d better have a good explanation.
But, McKinsey has no explanation. None. They’re stonewalling. You know what would happen to me if I tried that? Suppose I sent my new results to a journal, results that were very different from that of others, and said, “Trust me. They’re good.” Well, my paper would be laughed out of the editorial office.
And that’s as it should be. That would not be research. That would be the opposite of research. That would be indistinguishable from making things up. Well, anybody can make things up. The difference between making things up and actually doing some sound, rigorous work is the difference between fiction and reality.
It is no surprise that I, also, see the methods as being critical. Some extremely misleading associations can occur depending on how the sample was developed and how the survery was conducted. Of special concern:
"Our survey shows significantly more interest in alternatives to ESI [employer sponsored insurance] than other sources do, for several reasons," the report says. "Interest in these alternatives rises with increasing awareness of reform, and our survey educated respondents about its implications for their companies and employers before they were asked about post-2014 strategies. The propensity of employers to make big changes to ESI increases with awareness largely because shifting away will be economically rational not only for many of them but also for their lower-income employees, given the law's incentives.
So the survey educated employers on how it would be economically rational to not give health insurance and then 70% of them still decided to do so!! This is the precise opposite of the headline -- that employers will tr to keep covering employees even if it is not an optimal strategy. Probably because they want to do right by their employees.
Why is this key issue not getting a lot more attention in the interpretation of the report? And why is McKinesy's reputation not being brought into question given that they released such a report?
update: The fallout continues.
Thursday, June 9, 2011
"We've made huge advances in what they're called"*
US. American Express launched an integrated marketing campaign today showcasing the possibilities that exist within its Membership Rewards programme, with a focus on demonstrating how points are a ‘social currency’, that when redeemed help cardmembers connect to the people, experiences and things they love.* For the source of the quote, click here.
The real problem with Groupon
But as serious as these things might be, they are fixable. A management shake-up, a few smart people in the right places. I've seen business lines with bigger problems turned around and by companies with fewer resources than Groupon.
Problems with business plans are different and when you look at the bare bones core of Groupon's business plan, stripped of all the hype and obfuscation and needless complexity, Groupon is based on bringing together customers and merchants and convincing both to do something they normally hate doing: respectively, paying up front and taking a loss.
A Groupon offer is not a coupon; it's what we would normally think of as a gift card, a really crappy gift card at an exceptionally good price. How crappy? It can be used at only at a specific business (sometimes only at one location). You have to buy it on their schedule. Many if not most have expiration dates. Some even have blackouts on holidays. Finally, it can only be used one-per-visit with any unused balance being sacrificed.
People purchase these terrible cards partly because of the considerable (and non-sustainable) level of advertising and buzz; mainly, though, they buy them because the discount, both nominal and effective, is huge. (Felix Salmon has argued that the effective discount can be much smaller, but based on the offers I've received and reports from other users, it's generally easy to keep a purchase close to the offer amount.)
In other words, Groupon has huge revenue because, even when you take into account breakage, its merchant partners are offering huge discounts and certainly, in some cases, taking a loss.
What do merchants get in exchange for these discounts? The stated reason is to bring in new customers, but if that's your goal, Groupon is a terrible choice. Its customer list is enormous but of poor quality and the company hasn't even bothered to gather the data needed to do the most basic of targeting. There are any number of options using the internet and/or traditional media that will bring in a much higher percentage of the right kind of customers.
But merchants keep coming to Groupon despite its mediocre list and the fat slice it takes out of every deal (from Wikipedia):
As of 2010[update], it is difficult for local merchants to get Groupon interested in agreeing to a particular deal. According to the Wall Street Journal, seven of every eight possible deals suggested by merchants were dismissed by Groupon.
Just to be clear, merchants spend time and effort putting together offers that will probably be rejected and, if they're not, will probably bring in a lot of customers they don't want. They do this because Groupon has successfully branded itself as the next big thing.
This is not something the company stumbled into. Groupon has aggressively pursued fast growth, generated ubiquitous buzz and has done its damnedest to portray itself as part of the social media movement. The 'social' aspect of Groupon's business has always been trivial to the point of cosmetic but it plays a large, even dominant role in the public image of the company.
In one sense, Groupon's strategy has worked very well. After all, the people who started the company will almost certainly walk away with a great deal of money. Eventually, though, the company will have to make the transition to former next big thing and since being the current next big thing is an essential part of the company's model, that transition is not going to be pretty.
Wednesday, June 8, 2011
(Probably) OE's penultimate Groupon
Yet for Mr. Lefkofsky, Groupon wasn't a one-off in the lottery of high-tech wealth. It was an extension of a lifetime of starting and selling companies-- sometimes with mixed results. The process has led him to a set of guiding business principles: Enter big, fast-growing markets, change course when things aren't working and use data as your guide.Just for future reference, a lot of fast moving, information-based companies start collecting data on customers as soon as they launch.
"In our current environment, business and customers are changing so much faster than in the old days," he said in an interview. "You need access to information to figure out what to do next."
(I'm still having trouble believing this is a new feature.)
How exactly do you cold call a CT scan?
Here's one example among a million. The other day I went to the IPO announcement of a company that does some fairly state-of-the-art medical stuff. The company was spun off from a public institute a few years back to exploit this technology, but it's been unable to establish significant revenue or market share, or to get within shouting distance of breaking even. Meanwhile, competitors with similar technologies have gobbled up most of the market share, and one is already quite profitable. The company said it planned to raise some tens of millions of dollars with the share issue, many times its current annual expenditures and about a third of its overall market cap. And what would it do with this money? It was going to use half of it to finance a marketing drive, targeting key decisionmakers at American health-care providers and health insurers, and doctors.
Why hadn't this company been able to generate significant revenues? Were its technologies inferior? No, said an independent molecular biologist I talked to. Its product was certainly as good as the competition's. Moreover, it had actually gone to the trouble of getting its technology approved by the FDA, which the competition hadn't. (In this sub-sector FDA approval isn't yet mandatory.) But it hadn't marketed itself well. It hadn't established the relationships with providers and insurers that would ensure that its product was the one they selected. Doing so would require a marketing budget of tens of millions of dollars, in a sub-sector where the entire annual market is a few hundred million dollars.
Just think about this for a minute. A medical technology company is going public to generate the money it needs to advertise its products to hospital directors and insurance-company reimbursement officers. This entails significant extra expenditures for marketing, the new stocks issued to fund the marketing will ultimately have to pay dividends, banks will have to be paid to supervise the IPO that was needed to generate the funds to finance the marketing campaign (presumably charging the industry-cartel standard 7%)...and all this will have to be paid for by driving up the price the company charges to deliver its technologies. But beyond the added expense, why would anyone think that a system in which marketing plays such a large role is likely to be more effective, to lead to better treatment, than the kind of process of expert review that governs grant awards at NIH or publishing decisions at peer-reviewed journals? Why do we think that a system in which ads for Claritin are all over the subways will generate better overall health results than one where a national review board determines whether Claritin delivers treatment outcomes for some populations sufficiently superior to justify its added expense over similar generics? What do we expect from a system in which, as ProPublica reports today, body imaging companies hire telemarketers to sell random people CT scans over the phone?
Canadian Health Care
The universal access point is important. When people talk about fear and uncertainty (for business) as being a concern, I think it could also be an issue at the individual level. After all, the fear of an unexpected dread disease appears to be a major concern for most people. How much better would it be if we could eliminate these concerns by breaking the link between health coverage and employment while controlling costs in an effective manner?
The most startling thing in Dr. Carroll's graphs was that physicians are beginning to emigrate to Canada (on net). So even the medical care specialists (with requisite wage controls) seems to like the system.
Cap'n Crunch -- the Movie (you know they'll get around to it eventually)
Constantine von Hoffman summarizes the latest crop of films based on consumer goods:
Universal is set to make Ouija with the single-named director McG (Terminator Salvation) in line to join TRON: Legacy writers Edward Kitsis and Adam Horowitz — not the guy from the Beastie Boys, who’s Adam Horovitz, with a “v”. And this is all under the watchful eye of Michael Bay’sThe back story here (which is much more interesting than any of these movies are likely to be) involves innovation, demographic shifts, winnowing processes, business psychology and a truly nasty collective action problem, but the upshot is a situation where name recognition of any kind is given far more weight by studio executives than can be justified by modelling or common sense.
- production company.
- Ridley Scott is developing Monopoly. Yes, the Ridley Scott who made Aliens and Gladiator. Lord, I wish I could make up stuff this good.
- Jonathan Aibel and Glenn Berger, who co-wrote that timeless classic, Kung-Fu Panda 2, have said they are working on the Candy Land script. No, a gun was not being held to their heads when they said this. What a strange question.
On the bright side, a Cap'n Cruch movie should produce a nice check for the estate of Jay Ward.
Tuesday, June 7, 2011
Uncertainty and default -- another "if you don't post it someone else will"
Stan Collender and James Kwak are also on the case.
Personally, I’m not very convinced of the “business confidence” theory of the weak recovery.
I would myself lay much more emphasis on economic factors like: (i) the continuing destruction of American consumer wealth as housing prices deflate; (ii) the burden of rising oil prices; (iii) the collective decision of American consumers to increase their saving by 6 points of personal income – a laudable decision, but one that subtracts a lot of demand from the economy.
But if I were a believer in the business confidence theory, here’s the counter-question I’d put to Michael Barone:
Which is more likely to subtract from business confidence: a lame speech by the president – or a highly credible and sustained threat by the majority party in the House of Representatives to force a default on the debts, contracts, and other obligations of the United States?
Adverse selection, misaligned interests and Groupon
As you might have guessed, we here at OE are starting to get a bad feeling about Groupon's business plan, particularly given that the company's success relies on targeting customers but the company has badly mismanaged the collection of the most rudimentary of customer data.
Another challenge the company faces is adverse selection. In order to be sustainable, Groupon has got to connect merchants with the right kind of customers. If the people who reply to offers are mainly bargain hunters, merchants will realize that these huge discounts aren't worth the money and the whole system will collapse.
You would expect Groupon to try to target 'future regulars' with pitches like "Looking for an excuse to try something new?"
Instead we get this:
This is, admittedly, a small thing, but it shows, at the very least, an inattention to detail and, at worst, a willingness to prioritize getting another email address over protecting the interests of Groupon's merchant partners.
The people behind Groupon have proven extraordinarily adept at running up big numbers and generating hype, but they have shown remarkably little interest in setting up a sustainable, profitable company. Their targeting strategies would have been considered somewhat primitive a decade ago. Their attitude toward customer data is nonchalant (see here for examples of both). They've used a carpet bombing approach to advertising (including the inevitable Super Bowl ad) which generates large email lists but seldom produces high quality ones.
All of this makes me wonder if these people are more focused on the stock price in 2012 than in the solvency of the company in 2020.
Sawdust in the transmission
Which gave me an excuse to dredge up another favorite bit of Southern pop culture...
[if you have trouble viewing this on your browser -- and thanks to Blogger you probably will -- you might want to view or download it at Internet Archive]
Trivia bonus points: Griffith has said this was his favorite episode.
p.s. This is a public domain copy that doesn't include rights to the theme. You might want to mute the first thirty seconds or so.
Monday, June 6, 2011
Yeah, n = 1, but based on this sample, I can't see Groupon doing any targeting at all
There was just one problem. In order to reverse engineer a model there has to be a model and I couldn't a trace of one. There was no indication that Groupon was using any information about me when they sent me an email. Here's an amusing but not unrepresentative example:
$125 for 24-Karat-Gold Specialty Facial and Chocolate Foot Scrub at Chocolat Day Spa in Arcadia ($260 Value)
107 S 1st Ave.
Arcadia, California 91006
Of course, it's possible that I might give something like that as a gift (though I honestly don't think I know any women who would want a chocolate foot scrub -- the whole thing sounds disturbingly like a front for a fetish website), but when you have offers that the email recipient would never consider buying as anything but a gift, You can't really call it targeted marketing.
For further evidence that Groupon is not that sophisticated (or even that serious) about targeted marketing, check out the following:
The scary word here is "New." Did it really take over two years to add a feature that should have been fully operational the day the first website launched? Has Groupon actually neglected to gather rudimentary data on tens of millions of customers?
This might not be as bad as it looks (I don't see how it could be worse). Perhaps there's a level of sophistication that I'm missing. Unfortunately, as mentioned before, people who are betting on Groupon are assuming that the company will successfully pull off some extraordinarily difficult tricks with data. The apparent inability to manage simple tasks like gathering customer level demographics does not make these bets look all that smart.
p.s. Felix Salmon seems to mean something completely different by 'targeting' (though it's possible his definition is broad enough to include what I'm talking about here).
Sunday, June 5, 2011
What's the best investment you can ever make? (another Groupon post)
Just think about it for a moment. The returns are fantastic. There's almost no risk. The minimum investment is trivial. Your money is only tied up for a few days. You get your picture taken holding a big check. What's not to love?
Now, I can hear some of you negative types out there complaining about the difficulty of distinguishing between winning tickets and losing tickets (which are notoriously bad investments), but people routinely ignore these concerns when looking at the business plans behind IPOs. Why should you have a lower investing standard when dealing with your broker than you have at the 7 Eleven?
Take promises of targeted marketing. While it's true that almost all marketing is targeted to some extent and a few companies have been able to take that targeting to a relatively high level,* identifying customers who have a high likelihood (rather than a slightly higher likelihood) of buying something remains an extraordinarily challenging business problem. Most of the proposals you'll read that rely on solving that problem fall into the winning lottery ticket category.
Even with the recent explosion of consumer level data, the vast majority of plans to use targeted marketing run into one or both of the following problems: the lift provided by the selection method won't be large enough; the list produced won't be large enough.
Let's take the Groupon example. As pointed out here by Kaiser Fung, the merchants want new customers who are likely to become regulars. How would you go about targeting this segment? You might try matching the offer with the demographics of website the customers came in through, for instance, high end restaurants for people who came to Groupon through a New Yorker ad, but you'll still get lots responders who are already regular customers and more than a few bargain hunters (yes, even from the New Yorker). Or you could make offers only to people who have been identified as new to the area and are on the mailing lists of similar businesses in their old town, but I can tell you from experience, the number of names you'll get probably won't be large enough to bother with.
And Groupon has to thread an extraordinarily fine needle here. In most business situations, we might have a few customers who end up costing us a little money (for example, someone who just gets the loss leader at the drive-thru), but we're probably talking about fairly trivial amounts. In these cases, it's usually enough to build a model that distinguishes between responders and non-responder and fortunately, response is generally a quick and easy variable to measure reliably.
Groupon has to worry about non-responders (who are still associated with some costs), and about bargain hunters who use the offer then never come back (who cost Groupon's partners a substantial amount), and about regulars who use an offer for a visit they would have made anyway (who represent a double loss).
Separating all this chaff from the customers you want would be daunting even with the best of data and you will not have good data. How do I know? Because I've been there. I've dealt with third party data and I've written the hundreds of lines of SAS code needed to produce clean, usable data-sets. And I was only dealing with data from four or five sources, not trying to tease out a badly defined target variable from data collected from thousands of merchants. (remember, we're trying to identify customers who made their first visit using a Groupon offer and have since returned on their own dime.)
On top of all this, we're talking about a targeted marketing strategy that would have to work with everything from family pizza parlors to upscale wine bars, from pricey spas to summer camps, from teeth whitening to Scotchguarding (all of which have been recently offered by the company).
It's possible that Groupon will get around these problems but, until then, you might be better off with a scratcher-based portfolio.
* Of course, some people have proven pretty good at picking lottery tickets too.
More on Groupon
Our merchant arrangements are generally structured such that we collect cash up front when our customers purchase Groupons and make payments to our merchants at a subsequent date. In North America, we typically pay our merchants in installments within sixty days after the Groupon is sold. In most of our International markets, merchants are not paid until the customer redeems the Groupon.
Now you match this up with this, admittedly anecdotal example:
A good mate who owns a restaurant and did one of these deals after said it was outright amazing - many people would come in and spend EXACTLY the amount of the coupon. They didn't want to go 50c under and heaven forbid they went 50c over and have to pay more at full price
Even worse, you seem to have to more effects. One is a priming effect. New customers assume your $30 entree is worth $15. That is poison. The second is that merchants have begun to do things like "Groupon lines" (rational from their point of view to focus on the full-paying customers first) that reduce the value of the service.
So the business model involves a slow reimbursement to the merchant (waiting for money is death in a small business where cash flow issues can be fatal), enormous discounts (typically 75% off, with the last 25% coming in slowly), and "bargain hunters" who are unlikely to become regular customers.
As a final point, consider:
Perhaps Groupon management thinks it is creating a sustainable Prisoner’s Dilemma, one that ultimately destroys value for the local merchant ecosystem but benefits Groupon. In other words, Groupon could grow so big that local merchants have to use it, even though it ultimately hurts them. In game theory terms, Groupon creates an equilibrium point at “All Local Merchants Defect,” and then, having forced merchants into this value-destroying equilibrium, takes a cut for having rigged the game. Obviously, Groupon couldn’t share this thinking publicly. They would just continue to use the attract-loyal-new-customers argument even though it no longer makes any sense for a ginormous Groupon.
This may sound cynical. But if this is Groupon’s game plan, it isn’t cynical. It’s naïve. Most local merchants simply don’t have enough value in their collective ecosystem to share anything remotely like this much value with Groupon. This isn’t a stable equilibrium, it’s a suicidal one. The local merchants will have to stop using Groupon en masse not long after they first start experimenting with it.
The only way this works is if process quadruple for restaurants (because everyone uses a groupon). Maybe more than quadruple because you replace cash in hand with a 60 day payment. Why would restaurants not break this equilibrium and offer 60% off if you show up without a groupon? Immediate cash in hand, much higher profits, the customer pays less and they don't have to buy a groupon in advance.
And if you don't get this type of prisoner's dilemma, then it is hard to see where the sustainable value is going to be in this business model when your clients will eventually hate you.
Friday, June 3, 2011
Salmon/Fung cage match -- Did Salmon use a representative example?
Fung (who shares my high opinion of Salmon's acumen) has a good take-down of Salmon's analysis. You should probably read the whole thing but there's one particular aspect that strikes me as requiring additional attention.
Of course, that $7.50 doesn't take into account the cost to the restaurant of preparing and serving the meal (which would further help Fung's case), but putting that aside, how likely are customers to overshoot by a factor of three?
Let's start with [Salmon's] neighborhood restaurant example:
At Giorgio's, for instance, diners paid $15 for their Groupon -- which gave them $30 of food. But dinner for two at Giorgio's, with some kind of alcohol, can easily run to $100 or more. So even after knocking $22.50 off the bill (remember that Giorgio's kept $7.50 of the proceeds of Groupon), the restaurant would often still make money.
This is a bit complicated. We can trace how the cash flows. For Groupon, diners pay them $15, and they keep half of that, $7.50. For the diners, they paid Groupon $15 (now worth $30 spending), and so they pay Giorgio's $70; in other words, they paid $85 out of pocket for a meal worth $100 without Groupon. Giorgio's take in $70 from the diners plus $7.50 coming from Groupon for a meal worth $100....
So, I don't think the Groupon model is the kind of slam dunk Felix seems to think it is. Only if certain conditions are met will the merchants gain anything from Groupon:
- the value of the coupon has to be a fraction of the total spending at the merchant; in this example, the diners spent more than 3 times the face value of the coupon. What if the diners spend exactly $30? Then Giorgio’s loses $22.50 on each regular customer and earns $7.50 on each new customer, meaning that every 3 new customers pay for each regular’s discount. Not very attractive numbers at all.
Looking at the offers currently on Groupon, I see three restaurants, Beto's Grill ($20 for $10), Stefano's Pizzeria ($20 for $10) and Henry's Hat ($35 for $15). Of the three, I'm only familiar with Henry's Hat (a game themed bar that, last time I was there, had Kruzno in its library), but, based on the information online, it would be fairly easy for two people to keep the tab down to close to the amount of the Groupon offer in all three.
Obviously, there are plenty of places in LA where you should plan on paying big money for your dinner, but I haven't noticed those places on Groupon. Instead I've seen a lot of moderately priced spots, and I doubt you've got a lot of couples running up a $60 dollar tab on three buck a slice pizza.
Apparently, journalists who work for Yahoo don't have access to the internet
In Little Rock, Arkansas, for instance, the 2010-2011 yearbook at Russellville Middle School lists the "5 worst people of all time." Students listed former President George W. Bush and Bush's vice president, Dick Cheney, right after Adolph Hitler, Osama Bin Laden and Charles Manson.
Superintendent Randall Williams asked the yearbook printing company to cover the list with pieces of tape, but many parents aren't mollified.
"I think it's just hard to explain, and I've talked to the sponsor and she is very very very upset about it. That she didn't pay any attention to that particular part of that particular page," he told the local Fox affiliate. "I think she maybe just scanned the whole page and went on."
Friday afternoon data dump
There is nothing so distinctly middle-brow as the fear of being middle-brow -- yet another TNR edition.
In this excellent NPR piece, advocates claim this test can identify psychopaths but the researcher who developed the test isn't comfortable with the way it has been used. Lots of interesting stat questions here.
I always feel nervous when I disagree with Felix Salmon's analysis of a business model but Kaiser Fung made me feel a little better about this one.
Also from Gelman: If I can find the time, I'd like like to delve into this story about perennial nut-case Satoshi Kanazawa; I'd also like to join in the discussion of this Tyler Cowen post.
I wonder if this TNR writer knows that this practice goes back at least to Hearst (and was the basis for the Hitler allusion in Citizen Kane).
More reasons from Ebert to hate 3D.*
When a rocket scientist I know (this is LA, they're all over the place) heard an interview with James Surowiecki about the Wisdom of Crowds, he commented "wow, they've just discovered standard error." Perhaps now people are starting to realize some of the limitations. There's just not a lot of there there.
*update: Ebert has another post up on the search for the next big thing. His thoughts on the changing economics of the industry are particularly relevant.
Time will tell. What depresses me is that mainstream Hollywood seems to be experiencing a crisis of confidence. For decades there was the faith that if you released a good movie, people might very well buy tickets to it. The traditional pattern was to open it slowly, hope for good reviews, and then "roll it out" more widely.
The economics of television advertising put an end to the theory that an audience might find a movie. Now the movies must find an audience. Big new releases open everywhere at once, and everybody knows they will be available before long on DVD or On Demand . Theaters offer 3D as something you can't get at home (apart from "3D television," about which I am unpersuaded). Now we get rocking and rolling D-Box seats.
RIP: Joel Rosenberg
It is a poorer world without him.