What do grades measure?

[I wrote this in the middle of the big SAT thread and I thought I had posted it weeks ago but it appears that I never got around to it. So better late than never...]

As discussed before,  many of the calls for getting rid of the SAT use the argument that high school grades are a better indicator of college success so we don't need the SAT. There's a modeling fallacy here (also as previously discussed), but putting that aside, the suggestion that we should rely almost entirely on grades as a measure of academic accomplishment (not to be confused with measures of character and personal achievement) raises the question of what  exactly do grades measure? Put another way, what factors do we expect to be highly correlated with grade-point average?

First off, let's think about where grades come from. In most classes grades come almost entirely from tests, homework, in-class activities and writing assignments. In some cases there is an unavoidable subjective element in the evaluation of this work. With this in mind, think about what attributes and personality traits would correlate with higher performance.

Various forms of intelligence would doubtless factor in. This is an extraordinarily complex topic, but, in general, it's safe to say that school tends to easier if you're smarter.

The ability to memorize would possibly be an even bigger factor in many (perhaps most) courses. Closely related to this attribute and in some cases indistinguishable from it by many metrics would be the tolerance for the act of rote memorization. Lots of people with excellent memories find the act of sitting and going over the same facts again and again extremely unpleasant. Put another way, this is one of the many areas where hard work can compensate for a lack of aptitude.

This second attribute overlaps with the next major related categories: discipline, patience and focus. A great deal of academic success depends on the ability to spend large amounts of time going over material that is neither interesting nor challenging. (This can lead to the paradoxical but not uncommon result of high aptitude leading to boredom leading to poor performance in the area of that aptitude.) I'd argue these factors are often the dominant drivers of GPA.

Because of the unavoidable element of subjectivity, the halo effect and likability can also improve grade point averages.

Between the level of material covered and the need to fashion lessons and tests to serve large numbers of students, grades often tend to favor conventional thinkers over more original ones. As students progress through college, the emphasis tends to shift to more divergent learning but at least in high school, the student who thinks differently will often be penalized.

Personal stability and home life can also be a major factor, particularly in areas like homework and other out-of-class assignments.

Finally, there is the support network: quality of instruction; availability of tutors and homework assistance; libraries and learning centers; computers with good reliable Internet access; family members who have both the time and the ability to help explain assignments.

From an analytic standpoint, it would be nice if we had separate metrics for each of these aspects. As it is, we really can't distinguish between the the student with exceptionally good grasp of material in the a student who worked hard or who had a lot of help..

This is not a call for reforming all our grading system. Though there is certainly room for improvement, it is far from the most pressing matter we face and, more importantly, badly thought-out changes (and badly thought-out has been the reform norm lately) can do far more harm than good.

What we do need to do with this or any other ranking system is try to understand its drivers and limitations and to take steps to minimize the damage caused by mistakes (because mistakes will happen).