The big PR push (and credulous coverage) of Netflix's House of Cards has got me thinking about the necessary conditions for successfully launching an original scripted series. Before we get to the business case for Netflix, here are some thoughts on the current cable landscape.
When it comes to producing original content, the big players tend to fall into three groups (with one notable exceptions):
HBO
Showtime
USA
FX
TNT
Nick
Cartoon
AMC
Let's start slicing. Nick and Cartoon are certainly successful (SpongeBob is, by some standards, the most popular cable show ever) but animation and children's programming play by a very different set of rules. Production costs are extremely low and high audience turnover means that you can keep running the same few episodes for years. Furthermore, since children's programming works best in dedicated blocks, it's difficult for general interest channels to fully capitalize on this hugely profitable market. (Might throw Disney in here as well.)
Then there are the pay channels. Once again a very special business model. From a revenue standpoint, someone who watches five hours a month of Showtime counts the same as someone who watches fifty. What matters is loyalty. If you can bring in people with movies and specials, then get them sufficiently caught up in, say, Dexter, you don't have to worry about Starz stealing them away with better movies and specials. Better yet, if you can get fans of an older show like Dexter also hooked on a new show like Homeland, you can keep those monthly checks coming in for a long time.
This model does not work if your revenue comes from advertising. Having one show that a viewer never misses isn't nearly as valuable as having a bunch of shows that a viewer watches occasionally. This makes it extremely difficult for an expensive original cable show to pay for itself (particularly with a thirteen episode season). In this environment, viable shows have to add synergistic value. This brings us to the USA, FX and TNT, big flagship channels of ginormous media companies. These channels have the deep pockets, extensive synergies and long time frames that make it easier to make the business case for a Burn Notice or a Justified.
Which leaves AMC as the exception, the channel that would seem to suggest that anybody with good enough shows can make a go of original programming, but before an executive at AE points at AMC when greenlighting their next original drama, there are a couple of cautionary notes to keep in mind:
1. It's dangerous using outliers as examples
AMC got extraordinarily lucky twice, first in finding executives who could spot good shows and second in having good shows cross their desks. This is not something you can plan on in advance. In fact, it's something most companies fail at when they try.
2. (and this is the big one) It's not entirely clear that AMC is actually making money on this
It would take a good financial analyst and a lot of access to answer this definitively, but certainly looks like the network loses serious money on Mad Men and has been forced to make big and risky budget cuts to the show that is making them money, the ratings hit Walking Dead. Keep in mind, AMC has been both good and lucky. The odds against duplicating its success are slight and that success may not be all that, you know, successful.
These days, every cable channel seems to have an original scripted series in the works. From the executives' standpoint, this is easy to understand; these shows are the best way for a network to get prestige and media attention and who wouldn't want to be the VP who greenlit the next Breaking Bad? From the standpoint of the investor, though, if the channel isn't a TNT or an HBO or a Cartoon Channel, you might prefer another strategy.
Comments, observations and thoughts from two bloggers on applied statistics, higher education and epidemiology. Joseph is an associate professor. Mark is a professional statistician and former math teacher.
Sunday, February 17, 2013
Saturday, February 16, 2013
Weekend cinema buff alert
Hulu has temporarily lifted the paywall around the Criterion Collection. If you're a movie lover, you should definitely drop by.
p.s. but be prepared for LOTS of commercials.
p.s. but be prepared for LOTS of commercials.
The best tech writing you'll read this week just might be an article about wheelbarrows
I may may not always show it but I'm a great fan of technology (it's tough love, but love nonetheless). What I am not a fan of is the way that we discuss technology. Many (maybe most) journalists on this beat are so besotted with the gee-whiz aspects that they have little time to think seriously about why are certain technologies successful, what demands they makes on infrastructure, and how they answer existing needs and create new ones. (Search on 'ddulites' for more on this).
Having journalists report on the subject of their infatuation is generally a bad idea, but in this case the damage goes beyond the inevitable annoying prose. Technology is important and when we can't discuss it intelligently we make bad decisions that end up holding progress back.
Look at Groupon. One of the most exciting and promising areas of research today is the study of social networks, but partly because business reporters did such a poor job covering the subject, this crappy gift-card company was able to convince investors that its creaky business model was 'social.' (and no, requiring minimum orders for a deal to go through wouldn't have qualified even if they hadn't set the threshold meaninglessly low). The money that went to Groupon was diverted from other investments, including businesses that actually used the properties and analyses of social networks in their business plan.
Perhaps it's not surprising that the best thought-out, most clear-eyed popular writing on technology I've seen recently would focus on the least glamorous type of technology imaginable (via DeLong).
Wednesday, February 13, 2013
More fun with charts
Daniel Kuehn and Joseph previously discussed this post by Megan McArdle entitled "Department of Awful Statistics: Income Inequality Edition." They both make good points, but I'd like to approach this from the angle of appropriate visualization. McArdle supports her thesis that the middle class is neither disappearing nor getting poorer with charts derived from census table H-17 which you can and really should download here (the best way to keep us all honest is to play along at home).
The trouble is they're bad graphs.
To the extent that statistics includes data visualization, this is definitely bad statistics. When trying to depict trends and relationships, you generally want to get as much of the pertinent information as possible into the same graph. You don't want to force the reader to jump around the page trying to estimate slopes and compare magnitudes, nor do you want to take a few snapshots when you can easily picture all the data.
There are lots of acceptable ways of laying out the data table H-17, but I'm going just going to go with the simplest (partly because I like simple and partly because I'm doing this on Openoffice). As with McArdle's graphs, the numbers are inflation-adjusted.
I'm not that comfortable with this data (for reasons I'll get to in a minute), but this does look fairly consistent with the hollowing out of the middle class with 35K-75K (the top two lines) dropping more or less steadily for decades. Also check out the more than fourfold increase of people making more than 150K,
The two main things that make me uncomfortable with the data are the start point (with falls close to at least a couple of inflection points) and, on a related note, the failure to account for the baby boom which was at the bottom of its earning power forty years ago and should be close to the maximum now.
As far as I can tell income distribution is not broken down by age in these tables (though I suspect the data are available on request). We can, however, answer the related question of what median income looks like when we control for age and extended over a longer interval. (Download table P8 from here)
You can see why I was nervous about starting in 1967.
The question of income inequality and what's happening to the middle class is a complicated one and is probably best addressed by people who know what they're talking about, but if you are going to try to argue one side of the case graphically, you should at least take the time to use appropriate graphs.
p.s. I picked 35-44 because it seemed like a good representative mid-career interval and because, since I wasn't comparing different age groups, an uncluttered one-line graph seemed sufficient. If you prefer, here's the multi-range version (though I don't know if it adds much information).
The trouble is they're bad graphs.
To the extent that statistics includes data visualization, this is definitely bad statistics. When trying to depict trends and relationships, you generally want to get as much of the pertinent information as possible into the same graph. You don't want to force the reader to jump around the page trying to estimate slopes and compare magnitudes, nor do you want to take a few snapshots when you can easily picture all the data.
There are lots of acceptable ways of laying out the data table H-17, but I'm going just going to go with the simplest (partly because I like simple and partly because I'm doing this on Openoffice). As with McArdle's graphs, the numbers are inflation-adjusted.
I'm not that comfortable with this data (for reasons I'll get to in a minute), but this does look fairly consistent with the hollowing out of the middle class with 35K-75K (the top two lines) dropping more or less steadily for decades. Also check out the more than fourfold increase of people making more than 150K,
The two main things that make me uncomfortable with the data are the start point (with falls close to at least a couple of inflection points) and, on a related note, the failure to account for the baby boom which was at the bottom of its earning power forty years ago and should be close to the maximum now.
As far as I can tell income distribution is not broken down by age in these tables (though I suspect the data are available on request). We can, however, answer the related question of what median income looks like when we control for age and extended over a longer interval. (Download table P8 from here)
You can see why I was nervous about starting in 1967.
The question of income inequality and what's happening to the middle class is a complicated one and is probably best addressed by people who know what they're talking about, but if you are going to try to argue one side of the case graphically, you should at least take the time to use appropriate graphs.
p.s. I picked 35-44 because it seemed like a good representative mid-career interval and because, since I wasn't comparing different age groups, an uncluttered one-line graph seemed sufficient. If you prefer, here's the multi-range version (though I don't know if it adds much information).
Tuesday, February 12, 2013
Thinking about failure and collective amnesia.
No. Not in the sad my-life-adds-up-to-nothing way, but more in the case study sense. I've been noticing how often optimistic analyses of proposed products and business models echo the same arguments used over the years for various underperforming enterprises and catastrophic failures, invariably without a flicker of recognition.
No doubt, this is partly due to a general lack of long-term memory in the pundit class, but the problem seem particularly acute when it comes to failure. There are exceptions like this well-thought-out analogy by Josh Marshall or this piece of historical context for Zucker's Leno debacle from Kliph Nesteroff, but as a rule, most journalists don't pay nearly enough attention to these counterexamples (which makes it all the more difficult to avoid repeating mistakes).
I'll try to add some more entries and drill down into some of the specifics, but in the meantime, here's a short list of some potentially useful examples ideas that seemed (and in some cases, actually were) good ideas at the time.
The aforementioned mentioned attempt to make Jerry Lewis king of the talk shows.
A late Eighties format that doubled the resolution of video tapes while being completely compatible with standard VHS.
An attempt to break the Seventies DC/Marvel duopoly.
An attempt to break the Coke/Pepsi duopoly (perhaps breaking duopolies deserves a subcategory).
Adios, Amiga.
No doubt, this is partly due to a general lack of long-term memory in the pundit class, but the problem seem particularly acute when it comes to failure. There are exceptions like this well-thought-out analogy by Josh Marshall or this piece of historical context for Zucker's Leno debacle from Kliph Nesteroff, but as a rule, most journalists don't pay nearly enough attention to these counterexamples (which makes it all the more difficult to avoid repeating mistakes).
I'll try to add some more entries and drill down into some of the specifics, but in the meantime, here's a short list of some potentially useful examples ideas that seemed (and in some cases, actually were) good ideas at the time.
The aforementioned mentioned attempt to make Jerry Lewis king of the talk shows.
A late Eighties format that doubled the resolution of video tapes while being completely compatible with standard VHS.
An attempt to break the Seventies DC/Marvel duopoly.
An attempt to break the Coke/Pepsi duopoly (perhaps breaking duopolies deserves a subcategory).
Adios, Amiga.
Friday, February 8, 2013
Fun with charts
This post by Daniel Kuehn is worth reading, although all of the action is in the comments.
I think he is right on about the denomintor problem in interpreting her graphs. It's also a very good example of when a point can be correct and yet not explain all of the differences (her comments about rounding). However, the labeled buldge seems to be a lesser sin than variable bracket sizes on a density plot.
As for the change argument, it is fine to use a chart to explain something and then talk about the expected changes to the distribution. Where I am less happy is that there are changes going on in the United States all of the time (aging of the population, propensity to form a new household) that are going to influence the shape of this curve. It is possible to imagine the curve shifting exactly as Jon Evans suggested, and the reasons having to do with factors that have nothigng to do with inequality.
But standardized curves have their own issues . . . So even Magan's use of the curve to show the shifts over time doesn't address the null conditional on the changes in the underlying population. This may even help her argument, I am not sure, but certainly I would rather graph density plots in equal sized segments just for reader clarity
Still, a worthwhile argument to follow and it is useful insofar as it improves understanding of what the plots do and do not mean.
I think he is right on about the denomintor problem in interpreting her graphs. It's also a very good example of when a point can be correct and yet not explain all of the differences (her comments about rounding). However, the labeled buldge seems to be a lesser sin than variable bracket sizes on a density plot.
As for the change argument, it is fine to use a chart to explain something and then talk about the expected changes to the distribution. Where I am less happy is that there are changes going on in the United States all of the time (aging of the population, propensity to form a new household) that are going to influence the shape of this curve. It is possible to imagine the curve shifting exactly as Jon Evans suggested, and the reasons having to do with factors that have nothigng to do with inequality.
But standardized curves have their own issues . . . So even Magan's use of the curve to show the shifts over time doesn't address the null conditional on the changes in the underlying population. This may even help her argument, I am not sure, but certainly I would rather graph density plots in equal sized segments just for reader clarity
Still, a worthwhile argument to follow and it is useful insofar as it improves understanding of what the plots do and do not mean.
Wednesday, February 6, 2013
Annals of bad analogies
From here:
Think about it this way. Say your elderly mother had to be hospitalized for life-threatening cancer. The best doctor in the region is at Sacred Heart, a Catholic, private hospital. Could you ever imagine saying this? “Well, I don’t think our taxpayer dollars should subsidize this private institution that has religious roots, so we’re going to take her to County General, where she’ll get inferior care. ’Cause that’s just the right thing to do!”An analogy is the weakest form of argument, because it presumes similarities between cases. In this case we are equating a one time event (cancer treatment) with a long term process (educating people). There is also a difference in that cancer outcomes are much easier to measure (due to the fast time between diagnosis and resolution) than an educational process. So "better" is much easier to evaluate. Finally, it ignores magnitudes. What is "better" and by how much. Is it a matter of preference (Starbucks coffee is better than McDonald's coffee) or an objective metric?
No. You’d want to make sure that your tax dollars got your mom the best care. Period. Our approach should be no different for our children. Their lives are at stake when we’re talking about the quality of education they are receiving. The quality of care standard should certainly be no lower.
But this whole thing dodges the main question-- why is the County General hospital not competitive with the Sacred Heart hospital? Is that not the more interesting question? Is it because the County General can't turn patients away and so gets the sickest of the sick?
These points matter.
Monday, February 4, 2013
Imbalance in the infrastructure debate
Joseph's previous post builds on this thread from Mark Thoma. Each is worth reading but I think both understate the extraordinary asymmetry between the pro and con in the should-we-spend-on-infrastructure debate (distinct from the where-to-spend debate). Consider the following statements:
1. We need to repair and upgrade the country's infrastructure in the relatively near future (let's say a decade)
So far as I can tell, almost no one is willing to stand up and argue against this point, which is strange because, though I don't happen to agree with them, there are reasonable arguments to be made here and, once this point has been conceded, the remaining ground is extraordinarily difficult to defend.
2. The economy is not operating at full capacity
We've already stipulated that we need to build these things which means we've also agreed to tolerate at least some crowding out at some point in the future. You simply can't have one without the other; you can only seek to minimize the effect. A crowding-out argument for delaying pretty much has to assume that there will be more slack in the economy far enough in the future to make waiting worth it (but not so far to extend past our decade window). I've heard lots of people making crowding-out arguments but none making the necessary corollary. (even if you believe that crowding out is unaffected by economic conditions, you still don't have any argument for waiting)
3. Borrowing costs for the federal government are historically low.
As a general rule, repairs don't get cheaper the longer you put them off. This tends to put the burden of proof on those arguing for a delay. If we were living in a period of historically high borrowing costs, you could argue that rates were likely to head back down if we waited. There are reasonable cost-based arguments against infrastructure spending, but only in the spend/don't spend context, not spend now/spend later.
The infrastructure debate is another example of how the public discourse has entered a phase reminiscent of Carroll's Tortoise/Achilles tale, where showing the premise is true and showing the premise leads to a conclusion is not sufficient to make the other side accept that conclusion. Of course, it's not an exact analogy. Carroll was making a point about the limits of logical system. What we're seeing here is more probably a demonstration of people's willingness to ignore the rules of argument when those rules lead to an uncomfortable policy position.
1. We need to repair and upgrade the country's infrastructure in the relatively near future (let's say a decade)
So far as I can tell, almost no one is willing to stand up and argue against this point, which is strange because, though I don't happen to agree with them, there are reasonable arguments to be made here and, once this point has been conceded, the remaining ground is extraordinarily difficult to defend.
2. The economy is not operating at full capacity
We've already stipulated that we need to build these things which means we've also agreed to tolerate at least some crowding out at some point in the future. You simply can't have one without the other; you can only seek to minimize the effect. A crowding-out argument for delaying pretty much has to assume that there will be more slack in the economy far enough in the future to make waiting worth it (but not so far to extend past our decade window). I've heard lots of people making crowding-out arguments but none making the necessary corollary. (even if you believe that crowding out is unaffected by economic conditions, you still don't have any argument for waiting)
3. Borrowing costs for the federal government are historically low.
As a general rule, repairs don't get cheaper the longer you put them off. This tends to put the burden of proof on those arguing for a delay. If we were living in a period of historically high borrowing costs, you could argue that rates were likely to head back down if we waited. There are reasonable cost-based arguments against infrastructure spending, but only in the spend/don't spend context, not spend now/spend later.
The infrastructure debate is another example of how the public discourse has entered a phase reminiscent of Carroll's Tortoise/Achilles tale, where showing the premise is true and showing the premise leads to a conclusion is not sufficient to make the other side accept that conclusion. Of course, it's not an exact analogy. Carroll was making a point about the limits of logical system. What we're seeing here is more probably a demonstration of people's willingness to ignore the rules of argument when those rules lead to an uncomfortable policy position.
Friday, February 1, 2013
Sometimes you do it because it is a good idea anyway
Mark Thoma looks back five years to reports of skepticism about infrastructure development. Part of this is that maybe we just need to relax our rules on project timelines a wee bit. But another piece of it is that the worst mistake we can have to to end up competing with the private sector for labor to make cool things that we will end up needing. As disasters go, this one is rather mild.
Thursday, January 31, 2013
Shamisen heroes and free TV
I had been working on another piece about over the air television when I happened to surf across this video on one of the many Asian-themed channels you can get with an antenna in LA and it struck me as an appropriate accompaniment for a quick note about over the air TV.
With a pair of rabbit ears I pick up programming in at least a half dozen languages. That's an indication of the diversity of the medium and its importance to some underserved segments of the population, but it also represents a real competitive weakness. Terrestrial television suffers from a crippling lack of attention. Both Journalists who cover both media and personal finance are almost completely oblivious to this innovative, totally free source of programming.
If you're a medium trying to get the attention of the mainstream media, having a large part of your viewership consist of recent immigrants is not going to help.
I'll leave you with something a bit more traditional from the Yoshida brothers, though still, well... Hell, just watch it.
Wednesday, January 30, 2013
Unintended consequences
I normally agree with pro-immigration stands, but this one strikes me as likely to do something different than expected:
Nor am I sure we want even more price support for housing in the United States (mortgage deductions already support prices). And I doubt that, in world with sublets, that this would really help keep people local over and above the what we already have in place.
Plus, the rich enforcement mechanisms we have for work VISAs are already pretty scary, without adding this new level
While I think the region-based visa would be a positive step by itself, there is an additional twist I would recommend adding to the policy: require the purchase of a home from the visa recipient. This would be similar to the EB-5 program, which gives green cards to rich foreigners who invest in the U.S. This would allow non-rich immigrants to make an investment in the region sponsoring their visa. Not only does this increase the political popularity of the program and provide a way to transfer some of the gains of immigration to the native born population, but it also serves as an enforcement mechanism. Workers are less likely to leave the region their visa ties them too if they have made a large investment in that area which they cannot sell for the length of their visa.I love the idea of a regional (i.e. state level) work VISA. But the house thing is a terrible idea. First of all, how do you get a loan? If the requirement is "cash on the barrelhead" then we are only opening the market up to wealthy immigrants. High skilled people just starting out are squeezed out of the market. Or you get an asset, with little money down bought by people with a weak understanding of the local market who have the ability to flee overseas if things collapse.
Nor am I sure we want even more price support for housing in the United States (mortgage deductions already support prices). And I doubt that, in world with sublets, that this would really help keep people local over and above the what we already have in place.
Plus, the rich enforcement mechanisms we have for work VISAs are already pretty scary, without adding this new level
Friday, January 25, 2013
Paul Krugman on progress
Paul Krugman gets skeptical:
I have been seeing a lot of comments on automated cars lately. Of the West Coast Stat bloggers, I think I can be fairly described as the technological optimist. But even I worry that there could be some rather unexpected consequences to such a change, espeically if you have both humans and automated cars operating at the same time on the same roads.
By and large, I’m in the camp of those disillusioned about technology — mainly, I think, because the future isn’t what it used to be. A case in point is Herman Kahn’s The Year 2000, a 1967 exercise in forecasting that offered a convenient list of “very likely” technological developments. When 2000 actually did roll around, the striking thing was how over-optimistic the list was: Kahn foresaw most things that actually did happen, but also many things that didn’t (and still haven’t). And economic growth fell far short of his expectations.It is often the case that Krugman has a relatively unique take on things. Still, he is slowly coming around to having some upside views on one innovation:
But driverless cars break the pattern: even Kahn’s list of “less likely” possibilities only mentioned automated highways, not city streets, which is where we will apparently be in the quite near future.
I have been seeing a lot of comments on automated cars lately. Of the West Coast Stat bloggers, I think I can be fairly described as the technological optimist. But even I worry that there could be some rather unexpected consequences to such a change, espeically if you have both humans and automated cars operating at the same time on the same roads.
Thursday, January 24, 2013
A different take on grades and money from home -- part 1 Hypothesis Shopping
New study claims more money from home is associated with lower grades.
I came across this in a post from Andrew Gelman who had some sharp criticisms for the analysis that lead to this conclusion (criticisms seconded by Joseph in these previous posts). I clicked through the links with a slightly bloodthirsty attitude, already toying with ideas for posts mocking Dr. Hamilton, the hapless researcher. As I followed through, though, I felt less and less like mocking and more inclined to a shaded, even positive view, particularly given my previously noted feelings about looking at research in context.
Not that I disagree with Andrew and Joseph's criticisms -- the survivor bias is really difficult to get past -- but on some big how-we-do-science questions, I see a lot to like here, starting with where Hamilton started (and where she didn't).
Back in 2004, Hamilton and a group of other researchers spent a year studying a group of mostly first year students at large university, then continued to track their progress and interview them for the next five years. One of the things they observed was that the students who didn't have to work or struggle financially due to checks from home tended to take school less seriously, study less and get worse grades. Hamilton decided to follow this up by seeing if this pattern held nationally.
The resulting analysis was not well done, but I like the general approach: use case studies, participant observation, interviews and similar techniques to study your subjects extensively, form your hypotheses based on those observations, see how they hold up when tested against more general data. (or, put more broadly, actually devote time and effort into coming up with your hypotheses.)
That may seem like damning with faint praise, but I think a lot about where hypotheses come from and I give quite a bit of credit to researchers who put the work in to do it right, particularly in an age of hypothesis shopping.
Hypothesis shopping is one of our two leading sources of bad studies. The other is the epicyclism. The first entails running through endless unlikely relationships until one turns up significant (sometimes mistaken for data mining by people who know nothing about data mining). The second entails coming up with increasingly convoluted hypotheses to fit the data, often to preserve an ideological position. Sadly there's a nontrivial overlap between the two.
Hypothesis shopping has always been around but only recently has it become what you might call "cost effective" due to huge advances in the availability of data and the power of computers. Today, anyone with a large data set and a late-model laptop can crank through thousands of possible relationships looking for something with a good p-value. Just start with a couple dozen potential dependent variables, a few hundred independent variables, and some reasonable sounding transformations and interactions. You are pretty much certain to find something at least as impressive and "significant" as the the findings you'll routinely see in Slate or the New York Times.
The results of these processes are often absurd enough to be obvious to everyone (with the exception of the aforementioned publications), but for every watching-sports_before-conception-makes-you-more-likely-to-have-a-boy (I really hope that's a made up example), there's a provocative but not easily dismissed story with huge policy implications. It would be useful to know if we're talking about an effect that has been observed in other contexts or if it's just the green jelly bean.
Knowing the provenance of a hypothesis doesn't protect us from bad research but it does, to a large degree, inoculate us against certain kinds of bad research.
I came across this in a post from Andrew Gelman who had some sharp criticisms for the analysis that lead to this conclusion (criticisms seconded by Joseph in these previous posts). I clicked through the links with a slightly bloodthirsty attitude, already toying with ideas for posts mocking Dr. Hamilton, the hapless researcher. As I followed through, though, I felt less and less like mocking and more inclined to a shaded, even positive view, particularly given my previously noted feelings about looking at research in context.
Not that I disagree with Andrew and Joseph's criticisms -- the survivor bias is really difficult to get past -- but on some big how-we-do-science questions, I see a lot to like here, starting with where Hamilton started (and where she didn't).
Back in 2004, Hamilton and a group of other researchers spent a year studying a group of mostly first year students at large university, then continued to track their progress and interview them for the next five years. One of the things they observed was that the students who didn't have to work or struggle financially due to checks from home tended to take school less seriously, study less and get worse grades. Hamilton decided to follow this up by seeing if this pattern held nationally.
The resulting analysis was not well done, but I like the general approach: use case studies, participant observation, interviews and similar techniques to study your subjects extensively, form your hypotheses based on those observations, see how they hold up when tested against more general data. (or, put more broadly, actually devote time and effort into coming up with your hypotheses.)
That may seem like damning with faint praise, but I think a lot about where hypotheses come from and I give quite a bit of credit to researchers who put the work in to do it right, particularly in an age of hypothesis shopping.
Hypothesis shopping is one of our two leading sources of bad studies. The other is the epicyclism. The first entails running through endless unlikely relationships until one turns up significant (sometimes mistaken for data mining by people who know nothing about data mining). The second entails coming up with increasingly convoluted hypotheses to fit the data, often to preserve an ideological position. Sadly there's a nontrivial overlap between the two.
Hypothesis shopping has always been around but only recently has it become what you might call "cost effective" due to huge advances in the availability of data and the power of computers. Today, anyone with a large data set and a late-model laptop can crank through thousands of possible relationships looking for something with a good p-value. Just start with a couple dozen potential dependent variables, a few hundred independent variables, and some reasonable sounding transformations and interactions. You are pretty much certain to find something at least as impressive and "significant" as the the findings you'll routinely see in Slate or the New York Times.
The results of these processes are often absurd enough to be obvious to everyone (with the exception of the aforementioned publications), but for every watching-sports_before-conception-makes-you-more-likely-to-have-a-boy (I really hope that's a made up example), there's a provocative but not easily dismissed story with huge policy implications. It would be useful to know if we're talking about an effect that has been observed in other contexts or if it's just the green jelly bean.
Knowing the provenance of a hypothesis doesn't protect us from bad research but it does, to a large degree, inoculate us against certain kinds of bad research.
Wednesday, January 23, 2013
Survivor bias
There has been some discussion about meony from parents leading to a lower GPA from both Andrew Gelman and I. One of the comments at Andrew Gelman's site got me thinking:
When I was in the corporate world I never was asked for my transcripts (my degree, all of the time but my transcripts never). Having 2 years of college and then dropping out leads to worse life outcomes than having a degree, so far as I can tell.
Or put it another way, what would you prefer:
But even more interesting, the authors comments support my intuition precisely -- parental funding keeps marginal students in school. From a causal perspective this is way more interesting than the headline effect of giving dropping grades and is way more intuitive as well.
” The higher graduation rate of students whose parents paid their way is not surprising, she said, since many students leave college for financial reasons. (…)What is the actual target of inference here? Is it GPA or is it graduation?
Oddly, a lot of the parents who contributed the most money didn’t get the best returns on their investment (…) Their students were more likely to stay and graduate, but their G.P.A.’s were mediocre at best, and some I didn’t see study even once.”
When I was in the corporate world I never was asked for my transcripts (my degree, all of the time but my transcripts never). Having 2 years of college and then dropping out leads to worse life outcomes than having a degree, so far as I can tell.
Or put it another way, what would you prefer:
- A child who got high marks but did not complete their program?
- A child who got low marks but earned a degree?
But even more interesting, the authors comments support my intuition precisely -- parental funding keeps marginal students in school. From a causal perspective this is way more interesting than the headline effect of giving dropping grades and is way more intuitive as well.
Health Care spending
I often worry that comparison between the United States and Canada understate the room we have to improve health care costs. But Matt Yglesias points to a health care chart showing spending per capita by the government in the United States and Canada:
This is the chart that I think ought to dominate the conversation about public sector health care spending in the United States and yet is curiously ignored. The data show government health care spending per capita in the United States and Canada. The United States spends more. And that's not more per person who gets government health insurance, it's more per resident. And yet Canada covers all its citizens and we don't. That should be considered shocking stuff, and yet I rarely hear it mentioned.
Even odder is that the most recent time I heard it mentioned was Valerie Ramey talking at the American Economics Association conference in San Diego and her conclusion was that this showed U.S. health care needs free market reforms. The more straightforward interpretation, I would think, is that the U.S. needs to make its system more like Canada's. It's important to note that the example here is Canada. Not some radically different society. Not some far-off distant land. And the gap is actually growing.
In 2010, the Canadian government was spending roughly ~$3,000 per capita and the US government was spending ~$4,000 per capita. Not per beneficiary, but per capita!!
Now Canada is not some sort of dystopia without private medicine. Emergency services are fully covered but it was common when I was last there to go to a private clinic for health care like an X-ray to avoid the queue in the public options. But this is still catastrophic coverage for all citizens, which is an impressive feat.
At the end of the post, Matt talks a little about the less innovation counter-argument. Closely related is the MD shortage argument. Now there are two responses. One, is to note (as Matt does) that we want health care costs to go down in the United States and that this will have bad effects as well as good effects.
But the second is more compelling. We could just invest money in medical research and in awarding prizes for drug discovery. It would make innovation costs transparent and separate it out from rent-seeking and administrative costs (which would be lower in the single-payer insurance or out of pocket expenses world).
As for educating MDs, we can steal a leaf from Canada and subsidize education a wee bit more and lower wages will still have no trouble attracting people to a high status and still relatively high pay profession. No solution is perfect, but why is this one not being debated?
Subscribe to:
Posts (Atom)