Wednesday, May 2, 2012

Outcome assessment

This idea from Dean Dad is only going to work with an extremely objective metric.  It works fine for sales or time in a race, but I think that the assessment of soft skills is sufficently difficult that this is likelty to cause more trouble than it is worth.  Consider:
With the overall allocation flat or shrinking, every college is pretty much up against it already. None of them has much margin for error.  That’s especially true given how fixed most of the costs are, and how old most of the buildings are.  (Gotta love 70’s architecture!) 
Now they’re being told that it’s not enough that they do well; they must do better than their counterparts.  If Northern State’s “score” -- however defined -- moves up five points, but Southern State’s moves up ten, then Northern State takes a cut. 
If the colleges had large endowments, this could be a spur to entrepreneurialism.  If they were gambling with money they could afford to lose, then this could be just the kick in the pants they needed to start trying more ambitious things.  The prospect of the big win can justify the risky play.
But when the colleges are running on empty at the outset, the prospect of any meaningful loss is simply intolerable.  Instead of spurring innovation, this will heighten the already-strong culture of loss aversion.  Taking a flyer on a strategy that would take years to pay off isn’t an option when the years in between could require layoffs.
Worse, any kind of statewide collaboration -- exactly the sort of thing that would “move the needle” on educational attainment, workforce development, or any social good you care to name -- would be entirely out of the question.  Why would I share my breakthrough innovation with Nearby State, when it would erode my competitive advantage? 
And just how long, exactly, do you think it would take before the quick fix of grade inflation starts to look attractive?
 It is the quick fix of grade inflation that I consider to be the most important threat.  Even with externally administered objective tests we are creating massive incentives to find any way to improve on the tests possible.  Some of these approaches will align with the goals of the program (e.g. improving classroom instruction).  But other goals will be to load the deck in any way possible (inflating grades, changing admisison standards, shifting learning to the test-based goals) in order to try and avoid these crushing cuts.  After all, even small amounts of advantage will add up to more resources in the future.  And I am becoming more and more convinced that a lot of outcomes are driven by resources as much as anything else. 

I am not saying no version of this approach can work, but it is always concerning when outcomes are not hard but rather soft.

1 comment:

  1. If you think this is bad, you should examine more closely what happens at the elementary and high school levels.

    ReplyDelete