Marketplace on data mining and targeted marketing

Two stat relevant segments on today's show, one on Obama email marketing and another on mining online data. Good stuff.

Tsunamis and Scientologists

Since this last post seems to have gotten some interest, here are some more thoughts (or at least some previous thoughts spelled out in greater detail).

If you remember this started with a humor piece about the horrors of living in LA. It's an old and pretty much mined out genre. It was already tired when Harlan Ellison wrote the definitive rebuttal ("Face Down in Gloria Swanson's Swimming Pool") thirty-four years ago and time has not made it any fresher. Ellison could make a meal out of this latest piece, picking through  the literary flaws or the dubious cultural observations but I have to admit, it wasn't those that really caught my attention; it was the sampling methodology. 

Of course, we don't expect writers to base their work on random samples but we can hope for representative ones and one way to accomplish that is by sampling broadly. Here we have any incredibly narrow sample, both geographically and culturally. When the author tries to generalize from the sample, he (not entirely sure of the gender)  ends up applying traits to the entire area that are only found in a tiny portion of either Hollywood or Venice Beach, specifically the most touristy parts (Not that there's anything wrong with being touristy, but it should make you nervous when an expert on a region doesn't seem to be familiar with anything you can't see from a tour bus).

Take the Tsunami signs. When you walk down to the beach in LA you almost always really do walk down. This is a rocky coast and most of the beach towns are fairly elevated (more than New York, for instance). The notable exception is Venice (which actually has canals, by the way). Venice is the only neighborhood I know of that's close to sea level. Thus it's the only place I've seen Tsunami warning signs.

Then there's the fear of Scientology the author claims all Angelenos have. In the tiny area of Hollywood the author is focusing on, signs of the church and its founder are everywhere. Literally within a two mile radius, you have multiple buildings, displays, festivals, people offering free personality tests, even an L. Ron Hubbard drive. Outside of that radius, virtually nothing and the subject of Scientology seldom comes up.

For reason that aren't entirely coincidental, the Groundlings, Second City LA and the Upright Citizen's Brigade are also in that same two mile radius, hence the author's belief that all Angelenos are fans of improv. 

It's like this through the entire piece. The author latches onto some characteristic found only in four or so square miles of LA and assumes it holds for the remaining four thousand. The result is that old story, if you take your data from an unrepresentative subpopulation and try to generalize, you're likely to come out looking like an idiot.

Now if I could just figure out how a link to this got into Felix Salmon's blog.

Horribly pretentious person seeks out horribly pretentious people so she can be horrified at their pretentions

There's a certain type of humorous writing (and you have no idea how badly I wanted to use quotation marks there) where an out-of-towner moves to a city/state/region, seeks out the most stereotypical aspects and complains about them ("The trouble with Houston is, if you don't like cowboy hats and line dancing..."). These pieces are also sometimes accompanied with impressive-sounding sociological terms, in which case they qualify as serious journalism.

The LA subgenre invariably focuses on a tiny sliver of the more than four thousand square miles of LA County (when people talk about LA, they generally mean the county). It's a tiny strip that runs through (but doesn't encompass) Silver Lake, Hollywood, Beverly Hills, and Santa Monica. When you read one of these pieces bitching about cliche LA, you can be reasonably sure the writer didn't make it to Watts or East LA (if you want to read someone who actually did explore the area, check out Dana Goldstein).

Perhaps the worst example inexplicably come recommended by Counterparties. In it the writer, with a couple of notable exceptions, picks things that are so close they are actually in walking distance of each other, largely in a neighborhood that most Angelenos tend to avoid, leaving it to the tourists. As a result, she spends most of her time complaining about things that are only of concern to people who live in a tiny neighborhood dominated by Scientology buildings, improv theatres and tourist attractions. The writer is the only resident of LA I have ever encountered who actually talks about the stars on the sidewalk.

The only non-Hollywood area she refers to is Venice, specifically the boardwalk (once again, more of a tourist spot than a place for locals) and the Tsunami warning signs. She goes on about these at length in a way that makes me wonder if she has actually seen any of the coast other than Venice and Santa Monica (Venice is exceptionally low lying. Most of LA's coast is not).

Finally, as for the writer's depiction of LA being a town full of waiters calling themselves actors and directors, my experience here has actually been the opposite. Most people I meet are sensitive to the cliche and tend to overcompensate. I've never had a conversation with an aspiring entertainment type who overstated his or her resume. I have, however, had acquaintances I've known for a while before I happened to look them up on IMDB or Wikipedia and discovered they had done something pretty impressive. If the writer of this piece keeps running into people trying to impress her with exaggerated accomplishments, perhaps it's because they think she's shallow enough to be impressed by them.

Walmart and Unions

Matt Yglesias points out that Megan McArdle is actually making a pretty solid pro-Union arguement for Walmart (while trying to argue the opposite).  Consider:

This is a great case study in rhetorical strategies. But the analysis is admirably clear. Wal-Mart's profit margins, though by no means enormous, are larger than those of its main competitors. Given the weak national labor market, Wal-Mart has no reason to cough up extra money to its workforce. But a strong labor union could coerce them into coughing up higher pay and bringing their margins in line with Costco and Macy's. As a result, each Wal-Mart employee might get a bit less than $3,000 more a year. Whether that's "life-changing" or not is an interesting question, but since we're talking about low-wage workers here, I think the intuitions of highly paid professionals may be a bit off. It seems very plausible that the marginal hedonic value of a thousand bucks or three to Wal-Mart's workforce would be very large.

How large?  The average wage for a Walmart employee is tricky as many are not full time and the mean hourly wage is a subject of debate.  However, this is a high end estimate (both in hourly wages and in the assumption of full time employment):

The average Walmart "associate," Wake Up Walmart reports, makes $11.75 an hour. That's $20,744 per year. Those wages are slightly below the national average for retail employees, which is $12.04 an hour. They also produce annual earnings that, in a one-earner household, are below the $22,000 poverty line.
 

So a $3,000/year hike in wages would, conservatively, be a 15% increase in pay.  That would leave prices unchanged and reset the margins at Walmart to that of Macy's (hardly a disaster in the making to have margins at this level).  If the average employee makes more like $14,000 (another plausible estimate) then the pay rise is even steeper.  At these pay levels, a few thousand a year can be a life changing event, given the number of fixed expenses in life.   

The other argument Megan advances, that Amazon is coming, is admittedly more subtle.  But here is the thing, Matt is absolutely correct that Amazon is unstoppable so long as its current business model is permitted to exist.  It has no margin . . . at all, and is thus a bet on the future of e-commerce.  Now the United States has been aggressively subsidizing e-commerce in a number of ways, ranging from failing to collect sales taxes for online items, providing a reliable postal service everywhere to make the shipping model work, and Wall Street being willing to pour cash into the company despite no profits. 

But that issue doesn't affect Walmart more than any other retailer.  Nobody is immune to this problem and it remains to be seen how this will all shake out. 

Profile of a businessman I admire

I was working on a longer post about what a well run business looks like and I remembered the following Malcolm Gladwell piece for the New Yorker. When you read the first paragraph, you'll think the title of this post is a joke; when you get to the end, you'll know it's not.

Hostess and salaries

Mark pointed out that, in my Hostess and wages post, I only put in the previous wage cuts and not the new one that management was proposing.  The relevant passage:

Again, the 14-year Hostess bakery veteran: “Remember how I said I made $48,000 in 2005 and $34,000 last year? I would make $25,000 in five years if I took their offer. It will be hard to replace the job I had, but it will be easy to replace the job they were trying to give me.”

One thing that is easy to overlook in the discussion of did Unions show inflexibility is that a 50% wage cut over less than 10 years is an amazing drop in nominal wage (and the initial drop from 30,000 to 19,000 employees).  Mark's piece has pointed out that management was not doing any of the obvious moves to try save the business.  While the actual pattern of executive compensation is slightly more complicated than early reports indicated, it is pretty clear that huge (long term) wage cuts were not envisioned in the executive suite.

In some ways this links to a broader trend of firms being reluctant to actually pay workers.  This is linked to a reluctance to train workers, as well, but that only makes sense.  But here is where the whole story gets surreal to me.  Employees are assets as well as expenses.  We all seem to get that when we pay CEOs a lot of money -- we argue that it is not just an expense but also an investment in talent.  It is true that the modern world of corporate America has been very good at breaking down trust (few employees an informal promise at all anymore).

That is part of the issue with Hostess.  Not only were they trying to drop wages but the pattern of action on the part of the management team was to focus entirely on the easy target of salaries and ignore the hard targets of actually using information to reform the company into a more competitive shape.  Now maybe that was impossible (it seems unlikely but who knows).  But then we'd expect all of the bakers to be going under.

Nor is it impossible for firms to make highly creative moves to try and survive.  Notice how McDonald's and Starbucks (two companies that I consider well run) have quietly tried to sneak into each other's space.  Suddenly there is an egg sandwich at Starbucks (why go to two drive-thrus in the morning?).  And there are Latte's at McDonalds (why pay $5 for a Latte if you can't tell the difference).  Both firms are tougher, as a result, and -- as a pleasant side effect -- customers are better off.

But I want to go back to Mark's piece -- if you want to give management credit for a well run business strategy then you should be suspicious when failure is entirely blamed on the employees (who, t the union level, typically do not craft strategy).  

For those of you who are still working their way through the leftovers

Thanksgiving, 1905 from the incomparable Windsor McCay

from Mippyville.

update: the hash was very good, thanks for asking

The real reason for almost all big business failures

The commonly voiced view of business, red in tooth and claw, a Darwinian landscape where only the fit survive may hold for small to mid-sized operations, but with big business, survival is surprisingly easy. As a rule it is only the remarkably unfit that die. When you see a big company go under you can generally find a history of incompetent management and stupid or shortsighted decisions.

Though as narrative-obsessed as their political brethren, business journalists tend to be notably averse to stories built around the question "what happens when you give the keys to an idiot?" That's a shame because some of the most entertaining accounts start with that premise. It's understandable though. Business journalists, once again like their political brethren, tend to have overly cozy relationships with their subjects and, (compared to reporters thirty or forty) tend to sympathize more with management than labor.

The decline of professional standards also plays a part. A surprising number of "news" stories are actually press releases in only slightly rewritten form and PR department go to great lengths to avoid statements that make their bosses look like morons.

Given these factors and the uncomfortable cognitive dissonance that journalists often feel when forced to report that a major corporation's management is incompetent, they will almost always pass over the obvious narrative and opt instead for one of these three standards:

1. the company was made obsolete by new technology;

2. it was doomed by changing markets;

3. it just couldn't get good help.

Of course, in theory, these things can doom a company, in practice though, when you hear one of these explanations given for a collapse, you will almost always find that managerial malpractice played at least as large a role. For example, ebooks had much less to do with Borders demise than did an ill-conceived expansion into the ridiculously over-served British market.*

Here the LA Times invaluable Michael Hiltzik takes apart the standard narratives (numbers 2 and 3) of the troubles at Hostess (via Thoma, as usual)

Let's get a few things clear. Hostess didn't fail for any of the reasons you've been fed. It didn't fail because Americans demanded more healthful food than its Twinkies and Ho-Hos snack cakes. It didn't fail because its unions wanted it to die.

It failed because the people that ran it had no idea what they were doing. Every other excuse is just an attempt by the guilty to blame someone else.

...

Hostess management's efforts to blame union intransigence for the company's collapse persisted right through to the Thanksgiving eve press release announcing Hostess' liquidation, when it cited a nationwide strike by bakery workers that "crippled its operations."

That overlooks the years of union givebacks and management bad faith. Example: Just before declaring bankruptcy for the second time in eight years Jan. 11, Hostess trebled the compensation of then-Chief Executive Brian Driscoll and raised other executives' pay up to twofold. At the same time, the company was demanding lower wages from workers and stiffing employee pension funds of $8 million a month in payment obligations.

Hostess management hasn't been able entirely to erase the paper trail pointing to its own derelictions. Consider a 163-page affidavit filed as part of the second bankruptcy petition.

There Driscoll outlined a "Turnaround Plan" to get the firm back on its feet. The steps included closing outmoded plants and improving the efficiency of those that remain; upgrading the company's "aging vehicle fleet" and merging its distribution warehouses for efficiency; installing software at the warehouses to allow it to track inventory; and closing unprofitable retail stores. It also proposed to restore its advertising budget and establish an R&D program to develop new products to "maintain existing customers and attract new ones."

None of these steps, Driscoll attested, required consultation with the unions. That raises the following question: You mean to tell me that as of January 2012, Hostess still hadn't gotten around to any of this?

The company had known for a decade or more that its market was changing, but had done nothing to modernize its product line or distribution system. Its trucks were breaking down. It was keeping unprofitable stores open and having trouble figuring out how to move inventory to customers and when. It had cut back advertising and marketing to the point where it was barely communicating with customers. It had gotten hundreds of millions of dollars in concessions from its unions, and spent none of it on these essential improvements.

The true recent history of Hostess can be excavated from piles of public filings from its two bankruptcy cases. To start with, the company has had six CEOs in the last 10 years, which is not exactly a precondition for consistent and effective corporate strategizing.

... 

Hostess first entered bankruptcy in 2004, when it was known as Interstate Bakeries. During its five years in Chapter 11, the firm obtained concessions from its unions worth $110 million a year. The unions accepted layoffs that brought the workforce down to about 19,000 from more than 30,000. There were cuts in wages, pension and health benefits. The Teamsters committed to negotiations over changes in antiquated work rules. The givebacks helped reduce Hostess' labor costs to the point where they were roughly equal to or even lower than some of its major competitors'.

But the firm emerged from bankruptcy with more debt than when it went in — in with $575 million, out with $774 million, all secured by company assets. That's pretty much the opposite of what's supposed to happen in bankruptcy. By the end, there was barely a spare distributor cap in the motor pool that wasn't mortgaged to the private equity firms and hedge funds holding the notes (and also appointing management).

... 

The post-bankruptcy leadership never executed a growth strategy. It failed to introduce a significant new product or acquire a single new brand. It lagged on bakery automation and product R&D, while rivals such as Bimbo Bakeries USA built research facilities and hired food scientists to keep their product lines fresh. At the time of the 2004 bankruptcy, Hostess was three times the size of Bimbo. Today it's less than half Bimbo's size. (Bimbo, which has been acquiring bakeries such as Sara Lee and Entenmann's right and left, might well end up with Hostess' brands.)

UPDATE * I meant to include a link to this growth fetish post that gives my take on what the people at Borders might have been thinking.

"Lance Armstrong And The Business Of Doping"

This Planet Money story is the first (and will probably be the last) account of the Armstrong scandal that has held my interest. It also changed my what's-the-big-deal attitude.

To have one $8 billion dollar writedown, Ms. Whitman, may be regarded as a misfortune. To have two looks like carelessness.

Ryan McCarthy in Counterparties:

HP has become the place where synergies go to die. In its earnings release today, HP said it has taken an $8.8 billion writedown, caused by what it calls “serious accounting improprieties” at Autonomy, a software company it acquired for $11 billion in August 2011. This, as David Benoit notes, is HP’s second acquisition-related $8 billion writedown of the year.

HP is furiously pointing fingers: in a statement, the company said there was a “willful effort on behalf of certain former Autonomy employees to inflate the underlying financial metrics”. And CEO Meg Whitman told analysts that Deloitte had signed off on Autonomy’s accounts, with KPMG signing off on Deloitte. Still, only $5 billion of this quarter’s write down came from Autonomy’s accounting; the rest came from those pesky “headwinds against anticipated synergies and marketplace performance”.

There’s a strong case that HP should have smelled a rat. Bryce Elder points to “a decade’s worth of research questioning Autonomy’s revenue recognition, organic growth and seemingly flexible definition of a contract sale”. Then there’s the scathing statement which came about a month after the HP sale, in which Oracle revealed that it too had looked at Autonomy, but considered the company’s $6 billion market value to be “extremely over-priced”. Lynch, for his part, says that he was “ambushed” by the allegations.

I love the phrase "flexible definition."

Introducing "You Do the Math"

As Joseph mentioned earlier, I have a new blog up called You Do the Math focusing on ideas for teaching primary and secondary math. There's going to be a fair amount of cross-posting with this blog but I wanted a second outlet because:

1. I have a decade's worth of tips, exercises, lesson plans, test ideas, games, and puzzles for math teachers, much of which wouldn't be of much interest to most of the readers of this site.

2. I couldn't see the typical fifth grade teacher slogging through posts on the economics of health care and the  potential for selection effects in polling and the rest of the analytic esoterica you'll find here.

3. I had some ideas I wanted to play around with. Since so many of the teaching posts aren't time sensitive, I'm using the scheduling function to keep a steady stream of content flowing. Many of the posts will be short and or recycled but I've already scheduled at least one a week for the next five months. When I get to six months I plan to go back to the beginning and start putting up a second post for each week. It will be interesting to so what the effect is on traffic, page rank, etc.

4. Pauline Kael talked about the way writing a regular column helped her discipline and organize her thinking about movies. I believe a blog can serve much the same function in other areas.

I've already got one up that talks about using codes to introduce numberlines and tables and another that shows how you can estimate pi by seeing how many times randomly selected points fall within a given radius and I have about two dozen more posts sitting in the queue. If this sort of thing sounds interesting to you, check it out and if you know of anyone who might have use for this, please pass it on.

Thanks,
Mark

Blog list updated

The blog roll has had a fall cleaning.  I removed a few blogs that were really interesting but which had not been updated for months or years.   I added two new blogs: Mark Palko's new math teaching blog (which may occasionally cross post) and Mark Thoma's Economist's View (for which it is past time we added it).

Happy Thanksgiving, all!  

Toys for Tots -- reprinted, slightly revised

A good Christmas can do a lot to take the edge off of a bad year both for children and their parents (and a lot of families are having a bad year). It's the season to pick up a few toys, drop them by the fire station and make some people feel good about themselves during what can be one of the toughest times of the year.

If you're new to the Toys-for-Tots concept, here are the rules I normally use when shopping:

The gifts should be nice enough to sit alone under a tree. The child who gets nothing else should still feel that he or she had a special Christmas. A large stuffed animal, a big metal truck, a large can of Legos with enough pieces to keep up with an active imagination. You can get any of these for around twenty or thirty bucks at Wal-Mart or Costco;*

Shop smart. The better the deals the more toys can go in your cart;

No batteries. (I'm a strong believer in kid power);**

Speaking of kid power, it's impossible to be sedentary while playing with a basketball;

No toys that need lots of accessories;

For games, you're generally better off going with a classic;

No movie or TV show tie-ins. (This one's kind of a personal quirk and I will make some exceptions like Sesame Street);

Look for something durable. These will have to last;

For smaller children, you really can't beat Fisher Price and PlaySkool. Both companies have mastered the art of coming up with cleverly designed toys that children love and that will stand up to generations of energetic and creative play.

*I previously used Target here, but their selection has been dropping over the past few years and it's gotten more difficult to find toys that meet my criteria.

** I'd like to soften this position just bit. It's okay for a toy to use batteries, just not to need them. Fisher Price and PlaySkool have both gotten into the habit of adding lights and sounds to classic toys, but when the batteries die, the toys live on, still powered by the energy of children at play.

Bitter Harvest

Just in time for Thanksgiving.

From Marketplace:

Michael Hogan, CEO of A.D. Makepeace, a large cranberry grower based in Wareham, says for now cranberries are still a viable crop in Massachusetts. But climate change is making it much tougher to grow there.

"We're having warmer springs, we're having higher incidences of pests and fungus and we're having warmer falls when we need to have cooler nights," Hogan says.

Those changing conditions are costing growers like Makepeace money. The company has to use more water to irrigate in the hotter summers, and to cover the berries in spring and fall to protect them from frosts.

They're also spending more on fuel to run irrigation pumps, and have invested heavily in technology to monitor the bogs more closely. It's also meant more fungicides and fruit rot.

"Because of the percentage of rot that was delivered to Ocean Spray, they paid us $2 a barrel less," Hogan says. "We delivered 370,000 barrels last year. So you're talking about millions of dollars."

Cranberries also need a certain number of "chilling hours" to ensure that deep red color. Glen Reid, assistant manager of cranberry operations for A.D. Makepeace, says that's a problem as well.

"The berries actually need cold nights to color up," Reid says. "Last year we had a big problem with coloring up the berries. We had a lot more white berries."

Some growers in the Northeast have already moved some cranberry production to chillier climates, like eastern Canada. Ocean Spray has started growing cranberries on a new farm in the province of New Brunswick.

Makepeace's Hogan is even considering Chile, which apparently has a favorable climate for growing the North American berry.

I know we've been over this before, but it gets harder and harder to see how a cost/benefit analysis of climate change don't support immediate implementation of a carbon tax.or something similar.

Economics: a new definition?

Is it fair to quote the definition of economics from the blurb for a book?  If so, consider this definition in the blurb for Emily Oster's new book:

When Oster was expecting her first child, she felt powerless to make the right decisions for her pregnancy. How doctors think and what patients need are two very different things. So Oster drew on her own experience and went in search of the real facts about pregnancy using an economist’s tools. Economics is not just a study of finance. It’s the science of determining value and making informed decisions. To make a good decision, you need to understand the information available to you and to know what it means to you as an individual.

So, when applied to a medical topic (like pregnancy) how does this differ from evidence based medicine?  Should I be calling myself an economist?

None of this mean that Emily shouldn't write this book. My own read on the alcohol and pregnancy angle is that the current advice does seem to be based on an excess of caution.  But it seems odd to argue that being an economist is the key piece here.  Jumping fields is fine and can often lead to amazing insights, but maybe we should call this shifting of fields what it is rather than expanding the definition of economics to make it less meaningful?