Reporting on methods

Austin Frakt has the definitive take on the new McKinsey report:

As someone who does research, this really bothers me. It should bother you too. Look, anybody can say what they like on a topic. They can put out a glossy report. They can claim they did a “survey” to make it sound scientifically rigorous. They can talk to the media all about it. They can stand behind their good name and reputation, if they have one. But when what they’re saying runs counter to previous experience and other credible estimates, they’d better have a good explanation.

But, McKinsey has no explanation. None. They’re stonewalling. You know what would happen to me if I tried that? Suppose I sent my new results to a journal, results that were very different from that of others, and said, “Trust me. They’re good.” Well, my paper would be laughed out of the editorial office.

And that’s as it should be. That would not be research. That would be the opposite of research. That would be indistinguishable from making things up. Well, anybody can make things up. The difference between making things up and actually doing some sound, rigorous work is the difference between fiction and reality.

It is no surprise that I, also, see the methods as being critical. Some extremely misleading associations can occur depending on how the sample was developed and how the survery was conducted. Of special concern:

"Our survey shows significantly more interest in alternatives to ESI [employer sponsored insurance] than other sources do, for several reasons," the report says. "Interest in these alternatives rises with increasing awareness of reform, and our survey educated respondents about its implications for their companies and employers before they were asked about post-2014 strategies. The propensity of employers to make big changes to ESI increases with awareness largely because shifting away will be economically rational not only for many of them but also for their lower-income employees, given the law's incentives.

So the survey educated employers on how it would be economically rational to not give health insurance and then 70% of them still decided to do so!! This is the precise opposite of the headline -- that employers will tr to keep covering employees even if it is not an optimal strategy. Probably because they want to do right by their employees.

Why is this key issue not getting a lot more attention in the interpretation of the report? And why is McKinesy's reputation not being brought into question given that they released such a report?

update: The fallout continues.

"We've made huge advances in what they're called"*

I have a feeling we're going to get awfully tired of the word 'social.'

US. American Express launched an integrated marketing campaign today showcasing the possibilities that exist within its Membership Rewards programme, with a focus on demonstrating how points are a ‘social currency’, that when redeemed help cardmembers connect to the people, experiences and things they love.

* For the source of the quote, click here.

The real problem with Groupon

If you've been reading this blog over the past few days you've probably picked up on the fact that neither if us is a big fan of Groupon as an investment. We've talked about issues with priming and adverse selection, the company's oddly primitive approach to targeting and its curious indifference to customer data. We didn't even get around to the difficulties with running a business based on so many merchant partners.

But as serious as these things might be, they are fixable. A management shake-up, a few smart people in the right places. I've seen business lines with bigger problems turned around and by companies with fewer resources than Groupon.

Problems with business plans are different and when you look at the bare bones core of Groupon's business plan, stripped of all the hype and obfuscation and needless complexity, Groupon is based on bringing together customers and merchants and convincing both to do something they normally hate doing: respectively, paying up front and taking a loss.

A Groupon offer is not a coupon; it's what we would normally think of as a gift card, a really crappy gift card at an exceptionally good price. How crappy? It can be used at only at a specific business (sometimes only at one location). You have to buy it on their schedule. Many if not most have expiration dates. Some even have blackouts on holidays. Finally, it can only be used one-per-visit with any unused balance being sacrificed.

People purchase these terrible cards partly because of the considerable (and non-sustainable) level of advertising and buzz; mainly, though, they buy them because the discount, both nominal and effective, is huge. (Felix Salmon has argued that the effective discount can be much smaller, but based on the offers I've received and reports from other users, it's generally easy to keep a purchase close to the offer amount.)

In other words, Groupon has huge revenue because, even when you take into account breakage, its merchant partners are offering huge discounts and certainly, in some cases, taking a loss.

What do merchants get in exchange for these discounts? The stated reason is to bring in new customers, but if that's your goal, Groupon is a terrible choice. Its customer list is enormous but of poor quality and the company hasn't even bothered to gather the data needed to do the most basic of targeting. There are any number of options using the internet and/or traditional media that will bring in a much higher percentage of the right kind of customers.

But merchants keep coming to Groupon despite its mediocre list and the fat slice it takes out of every deal (from Wikipedia):

As of 2010^[update], it is difficult for local merchants to get Groupon interested in agreeing to a particular deal. According to the Wall Street Journal, seven of every eight possible deals suggested by merchants were dismissed by Groupon.

Just to be clear, merchants spend time and effort putting together offers that will probably be rejected and, if they're not, will probably bring in a lot of customers they don't want. They do this because Groupon has successfully branded itself as the next big thing.

This is not something the company stumbled into. Groupon has aggressively pursued fast growth, generated ubiquitous buzz and has done its damnedest to portray itself as part of the social media movement. The 'social' aspect of Groupon's business has always been trivial to the point of cosmetic but it plays a large, even dominant role in the public image of the company.

In one sense, Groupon's strategy has worked very well. After all, the people who started the company will almost certainly walk away with a great deal of money. Eventually, though, the company will have to make the transition to former next big thing and since being the current next big thing is an essential part of the company's model, that transition is not going to be pretty.

(Probably) OE's penultimate Groupon

From the the easily impressed Wall Street Journal:

Yet for Mr. Lefkofsky, Groupon wasn't a one-off in the lottery of high-tech wealth. It was an extension of a lifetime of starting and selling companies-- sometimes with mixed results. The process has led him to a set of guiding business principles: Enter big, fast-growing markets, change course when things aren't working and use data as your guide.

"In our current environment, business and customers are changing so much faster than in the old days," he said in an interview. "You need access to information to figure out what to do next."

Just for future reference, a lot of fast moving, information-based companies start collecting data on customers as soon as they launch.




(I'm still having trouble believing this is a new feature.)

How exactly do you cold call a CT scan?

Via Chait, M.S. at the Economist has some sharp thoughts on the way markets handle health care.

Here's one example among a million. The other day I went to the IPO announcement of a company that does some fairly state-of-the-art medical stuff. The company was spun off from a public institute a few years back to exploit this technology, but it's been unable to establish significant revenue or market share, or to get within shouting distance of breaking even. Meanwhile, competitors with similar technologies have gobbled up most of the market share, and one is already quite profitable. The company said it planned to raise some tens of millions of dollars with the share issue, many times its current annual expenditures and about a third of its overall market cap. And what would it do with this money? It was going to use half of it to finance a marketing drive, targeting key decisionmakers at American health-care providers and health insurers, and doctors.

Why hadn't this company been able to generate significant revenues? Were its technologies inferior? No, said an independent molecular biologist I talked to. Its product was certainly as good as the competition's. Moreover, it had actually gone to the trouble of getting its technology approved by the FDA, which the competition hadn't. (In this sub-sector FDA approval isn't yet mandatory.) But it hadn't marketed itself well. It hadn't established the relationships with providers and insurers that would ensure that its product was the one they selected. Doing so would require a marketing budget of tens of millions of dollars, in a sub-sector where the entire annual market is a few hundred million dollars.

Just think about this for a minute. A medical technology company is going public to generate the money it needs to advertise its products to hospital directors and insurance-company reimbursement officers. This entails significant extra expenditures for marketing, the new stocks issued to fund the marketing will ultimately have to pay dividends, banks will have to be paid to supervise the IPO that was needed to generate the funds to finance the marketing campaign (presumably charging the industry-cartel standard 7%)...and all this will have to be paid for by driving up the price the company charges to deliver its technologies. But beyond the added expense, why would anyone think that a system in which marketing plays such a large role is likely to be more effective, to lead to better treatment, than the kind of process of expert review that governs grant awards at NIH or publishing decisions at peer-reviewed journals? Why do we think that a system in which ads for Claritin are all over the subways will generate better overall health results than one where a national review board determines whether Claritin delivers treatment outcomes for some populations sufficiently superior to justify its added expense over similar generics? What do we expect from a system in which, as ProPublica reports today, body imaging companies hire telemarketers to sell random people CT scans over the phone?

Canadian Health Care

Aaron Carroll provides support for the Canadian Health Care system after comments by Paul Krugman. It is a pretty devastating critique (quality seems slightly higher) made even more so when you consider cost and universal access.

The universal access point is important. When people talk about fear and uncertainty (for business) as being a concern, I think it could also be an issue at the individual level. After all, the fear of an unexpected dread disease appears to be a major concern for most people. How much better would it be if we could eliminate these concerns by breaking the link between health coverage and employment while controlling costs in an effective manner?

The most startling thing in Dr. Carroll's graphs was that physicians are beginning to emigrate to Canada (on net). So even the medical care specialists (with requisite wage controls) seems to like the system.

Cap'n Crunch -- the Movie (you know they'll get around to it eventually)

In the progression of movies-based-on-old-TV-shows to movies-based-on-theme-park-rides to movies-based-on-board-games (via About.com), this would seem to be the next logical step.

Constantine von Hoffman summarizes the latest crop of films based on consumer goods:

Universal is set to make Ouija with the single-named director McG (Terminator Salvation) in line to join TRON: Legacy writers Edward Kitsis and Adam Horowitz — not the guy from the Beastie Boys, who’s Adam Horovitz, with a “v”. And this is all under the watchful eye of Michael Bay’s

• production company.
  
• Ridley Scott is developing Monopoly. Yes, the Ridley Scott who made Aliens and Gladiator. Lord, I wish I could make up stuff this good.
• Jonathan Aibel and Glenn Berger, who co-wrote that timeless classic, Kung-Fu Panda 2, have said they are working on the Candy Land script. No, a gun was not being held to their heads when they said this. What a strange question.

The back story here (which is much more interesting than any of these movies are likely to be) involves innovation, demographic shifts, winnowing processes, business psychology and a truly nasty collective action problem, but the upshot is a situation where name recognition of any kind is given far more weight by studio executives than can be justified by modelling or common sense.

On the bright side, a Cap'n Cruch movie should produce a nice check for the estate of Jay Ward.

If you recognize this picture, you don't have to give your age

Uncertainty and default -- another "if you don't post it someone else will"

I'd been meaning to post something about the uncertainty caused by Dodd-Frank and Clinton Era taxes compared with the uncertainty cause by the debt limit showdown, but David Frum beat me to it with a very good post:

Personally, I’m not very convinced of the “business confidence” theory of the weak recovery.

I would myself lay much more emphasis on economic factors like: (i) the continuing destruction of American consumer wealth as housing prices deflate; (ii) the burden of rising oil prices; (iii) the collective decision of American consumers to increase their saving by 6 points of personal income – a laudable decision, but one that subtracts a lot of demand from the economy.

But if I were a believer in the business confidence theory, here’s the counter-question I’d put to Michael Barone:

Which is more likely to subtract from business confidence: a lame speech by the president – or a highly credible and sustained threat by the majority party in the House of Representatives to force a default on the debts, contracts, and other obligations of the United States?

Stan Collender and James Kwak are also on the case.

Adverse selection, misaligned interests and Groupon

[Sorry about the all-Groupn-all-the-time blogging. We've got some more varied posts in the queue.]

As you might have guessed, we here at OE are starting to get a bad feeling about Groupon's business plan, particularly given that the company's success relies on targeting customers but the company has badly mismanaged the collection of the most rudimentary of customer data.

Another challenge the company faces is adverse selection. In order to be sustainable, Groupon has got to connect merchants with the right kind of customers. If the people who reply to offers are mainly bargain hunters, merchants will realize that these huge discounts aren't worth the money and the whole system will collapse.

You would expect Groupon to try to target 'future regulars' with pitches like "Looking for an excuse to try something new?"

Instead we get this:



This is, admittedly, a small thing, but it shows, at the very least, an inattention to detail and, at worst, a willingness to prioritize getting another email address over protecting the interests of Groupon's merchant partners.

The people behind Groupon have proven extraordinarily adept at running up big numbers and generating hype, but they have shown remarkably little interest in setting up a sustainable, profitable company. Their targeting strategies would have been considered somewhat primitive a decade ago. Their attitude toward customer data is nonchalant (see here for examples of both). They've used a carpet bombing approach to advertising (including the inevitable Super Bowl ad) which generates large email lists but seldom produces high quality ones.

All of this makes me wonder if these people are more focused on the stock price in 2012 than in the solvency of the company in 2020.

Sawdust in the transmission

I was thinking about companies designed to look good long enough for the investors to get safely past the IPO and the sawdust analogy hit me.

Which gave me an excuse to dredge up another favorite bit of Southern pop culture...

[if you have trouble viewing this on your browser -- and thanks to Blogger you probably will -- you might want to view or download it at Internet Archive]



Trivia bonus points: Griffith has said this was his favorite episode.

p.s. This is a public domain copy that doesn't include rights to the theme. You might want to mute the first thirty seconds or so.

Yeah, n = 1, but based on this sample, I can't see Groupon doing any targeting at all

After finishing the last post, I got to thinking about the emails I'd gotten from Groupon and I decided to go through them and try some back-of-the-envelope reverse engineering on their targeting methods. Obviously, I just had one account to look at but that's often enough to let me make an educated guess at things like what sort of data they might be looking at.

There was just one problem. In order to reverse engineer a model there has to be a model and I couldn't a trace of one. There was no indication that Groupon was using any information about me when they sent me an email. Here's an amusing but not unrepresentative example:

$125 for 24-Karat-Gold Specialty Facial and Chocolate Foot Scrub at Chocolat Day Spa in Arcadia ($260 Value)
$125 Value $260 Discount 52% You Save $135

The Company Chocolat Day Spa

Locations 107 S 1st Ave. Arcadia, California 91006 Get Directions

Of course, it's possible that I might give something like that as a gift (though I honestly don't think I know any women who would want a chocolate foot scrub -- the whole thing sounds disturbingly like a front for a fetish website), but when you have offers that the email recipient would never consider buying as anything but a gift, You can't really call it targeted marketing.

For further evidence that Groupon is not that sophisticated (or even that serious) about targeted marketing, check out the following:



The scary word here is "New." Did it really take over two years to add a feature that should have been fully operational the day the first website launched? Has Groupon actually neglected to gather rudimentary data on tens of millions of customers?

This might not be as bad as it looks (I don't see how it could be worse). Perhaps there's a level of sophistication that I'm missing. Unfortunately, as mentioned before, people who are betting on Groupon are assuming that the company will successfully pull off some extraordinarily difficult tricks with data. The apparent inability to manage simple tasks like gathering customer level demographics does not make these bets look all that smart.

p.s. Felix Salmon seems to mean something completely different by 'targeting' (though it's possible his definition is broad enough to include what I'm talking about here).

What's the best investment you can ever make? (another Groupon post)

Answer: a winning lottery ticket.

Just think about it for a moment. The returns are fantastic. There's almost no risk. The minimum investment is trivial. Your money is only tied up for a few days. You get your picture taken holding a big check. What's not to love?

Now, I can hear some of you negative types out there complaining about the difficulty of distinguishing between winning tickets and losing tickets (which are notoriously bad investments), but people routinely ignore these concerns when looking at the business plans behind IPOs. Why should you have a lower investing standard when dealing with your broker than you have at the 7 Eleven?

Take promises of targeted marketing. While it's true that almost all marketing is targeted to some extent and a few companies have been able to take that targeting to a relatively high level,* identifying customers who have a high likelihood (rather than a slightly higher likelihood) of buying something remains an extraordinarily challenging business problem. Most of the proposals you'll read that rely on solving that problem fall into the winning lottery ticket category.

Even with the recent explosion of consumer level data, the vast majority of plans to use targeted marketing run into one or both of the following problems: the lift provided by the selection method won't be large enough; the list produced won't be large enough.

Let's take the Groupon example. As pointed out here by Kaiser Fung, the merchants want new customers who are likely to become regulars. How would you go about targeting this segment? You might try matching the offer with the demographics of website the customers came in through, for instance, high end restaurants for people who came to Groupon through a New Yorker ad, but you'll still get lots responders who are already regular customers and more than a few bargain hunters (yes, even from the New Yorker). Or you could make offers only to people who have been identified as new to the area and are on the mailing lists of similar businesses in their old town, but I can tell you from experience, the number of names you'll get probably won't be large enough to bother with.

And Groupon has to thread an extraordinarily fine needle here. In most business situations, we might have a few customers who end up costing us a little money (for example, someone who just gets the loss leader at the drive-thru), but we're probably talking about fairly trivial amounts. In these cases, it's usually enough to build a model that distinguishes between responders and non-responder and fortunately, response is generally a quick and easy variable to measure reliably.

Groupon has to worry about non-responders (who are still associated with some costs), and about bargain hunters who use the offer then never come back (who cost Groupon's partners a substantial amount), and about regulars who use an offer for a visit they would have made anyway (who represent a double loss).

Separating all this chaff from the customers you want would be daunting even with the best of data and you will not have good data. How do I know? Because I've been there. I've dealt with third party data and I've written the hundreds of lines of SAS code needed to produce clean, usable data-sets. And I was only dealing with data from four or five sources, not trying to tease out a badly defined target variable from data collected from thousands of merchants. (remember, we're trying to identify customers who made their first visit using a Groupon offer and have since returned on their own dime.)

On top of all this, we're talking about a targeted marketing strategy that would have to work with everything from family pizza parlors to upscale wine bars, from pricey spas to summer camps, from teeth whitening to Scotchguarding (all of which have been recently offered by the company).

It's possible that Groupon will get around these problems but, until then, you might be better off with a scratcher-based portfolio.

* Of course, some people have proven pretty good at picking lottery tickets too.

More on Groupon

As a follow-up to Mark, the comment thread to this post is worth reading. One clever reader noted, in the S-1 groupon just filed:

Our merchant arrangements are generally structured such that we collect cash up front when our customers purchase Groupons and make payments to our merchants at a subsequent date. In North America, we typically pay our merchants in installments within sixty days after the Groupon is sold. In most of our International markets, merchants are not paid until the customer redeems the Groupon.

Now you match this up with this, admittedly anecdotal example:

A good mate who owns a restaurant and did one of these deals after said it was outright amazing - many people would come in and spend EXACTLY the amount of the coupon. They didn't want to go 50c under and heaven forbid they went 50c over and have to pay more at full price

Even worse, you seem to have to more effects. One is a priming effect. New customers assume your $30 entree is worth $15. That is poison. The second is that merchants have begun to do things like "Groupon lines" (rational from their point of view to focus on the full-paying customers first) that reduce the value of the service.

So the business model involves a slow reimbursement to the merchant (waiting for money is death in a small business where cash flow issues can be fatal), enormous discounts (typically 75% off, with the last 25% coming in slowly), and "bargain hunters" who are unlikely to become regular customers.

As a final point, consider:

Perhaps Groupon management thinks it is creating a sustainable Prisoner’s Dilemma, one that ultimately destroys value for the local merchant ecosystem but benefits Groupon. In other words, Groupon could grow so big that local merchants have to use it, even though it ultimately hurts them. In game theory terms, Groupon creates an equilibrium point at “All Local Merchants Defect,” and then, having forced merchants into this value-destroying equilibrium, takes a cut for having rigged the game. Obviously, Groupon couldn’t share this thinking publicly. They would just continue to use the attract-loyal-new-customers argument even though it no longer makes any sense for a ginormous Groupon.

This may sound cynical. But if this is Groupon’s game plan, it isn’t cynical. It’s naïve. Most local merchants simply don’t have enough value in their collective ecosystem to share anything remotely like this much value with Groupon. This isn’t a stable equilibrium, it’s a suicidal one. The local merchants will have to stop using Groupon en masse not long after they first start experimenting with it.

The only way this works is if process quadruple for restaurants (because everyone uses a groupon). Maybe more than quadruple because you replace cash in hand with a 60 day payment. Why would restaurants not break this equilibrium and offer 60% off if you show up without a groupon? Immediate cash in hand, much higher profits, the customer pays less and they don't have to buy a groupon in advance.

And if you don't get this type of prisoner's dilemma, then it is hard to see where the sustainable value is going to be in this business model when your clients will eventually hate you.

Salmon/Fung cage match -- Did Salmon use a representative example?

Felix Salmon is one of the sharpest business and finance bloggers out there but I've never quite shared his enthusiasm for the Groupon business model. Disagreeing with Salmon on business matters makes me a little nervous, so I feel a bit better to have Kaiser Fung on my side.

Fung (who shares my high opinion of Salmon's acumen) has a good take-down of Salmon's analysis. You should probably read the whole thing but there's one particular aspect that strikes me as requiring additional attention.

Here's Fung:

Let's start with [Salmon's] neighborhood restaurant example:

At Giorgio's, for instance, diners paid $15 for their Groupon -- which gave them $30 of food. But dinner for two at Giorgio's, with some kind of alcohol, can easily run to $100 or more. So even after knocking $22.50 off the bill (remember that Giorgio's kept $7.50 of the proceeds of Groupon), the restaurant would often still make money.

This is a bit complicated. We can trace how the cash flows. For Groupon, diners pay them $15, and they keep half of that, $7.50. For the diners, they paid Groupon $15 (now worth $30 spending), and so they pay Giorgio's $70; in other words, they paid $85 out of pocket for a meal worth $100 without Groupon. Giorgio's take in $70 from the diners plus $7.50 coming from Groupon for a meal worth $100.

...

So, I don't think the Groupon model is the kind of slam dunk Felix seems to think it is. Only if certain conditions are met will the merchants gain anything from Groupon:

• the value of the coupon has to be a fraction of the total spending at the merchant; in this example, the diners spent more than 3 times the face value of the coupon. What if the diners spend exactly $30? Then Giorgio’s loses $22.50 on each regular customer and earns $7.50 on each new customer, meaning that every 3 new customers pay for each regular’s discount. Not very attractive numbers at all.

Of course, that $7.50 doesn't take into account the cost to the restaurant of preparing and serving the meal (which would further help Fung's case), but putting that aside, how likely are customers to overshoot by a factor of three?

Looking at the offers currently on Groupon, I see three restaurants, Beto's Grill ($20 for $10), Stefano's Pizzeria ($20 for $10) and Henry's Hat ($35 for $15). Of the three, I'm only familiar with Henry's Hat (a game themed bar that, last time I was there, had Kruzno in its library), but, based on the information online, it would be fairly easy for two people to keep the tab down to close to the amount of the Groupon offer in all three.

Obviously, there are plenty of places in LA where you should plan on paying big money for your dinner, but I haven't noticed those places on Groupon. Instead I've seen a lot of moderately priced spots, and I doubt you've got a lot of couples running up a $60 dollar tab on three buck a slice pizza.