Ten years ago, the NYT et al. was railing against the unfairness of the SAT and ACT. Now, they're basically saying what we were saying then.

Thursday, May 15, 2014

The SAT probably is unfair to the disadvantaged but not for the reasons you've been hearing

This is another one of those posts that I started weeks ago as part of the big SAT thread then didn't get around to posting. One of the big questions was the fairness of the test. I had alluded to the problem but usually as a side question. Part of the reason I didn't spend more time on the question was because of my specialty. Though I did originally certify to teach math and English, I've been focused almost entirely on the former for a number of years, and almost all of the prep work I've done with student has been on the math side.

If I had been working with the verbal part, I would have had to address some uncomfortable questions. The verbal SAT is a good, well-designed and informative test but there are inescapable concerns about its fairness. It is very difficult to design a verbal reasoning test that is not culturally biased. Language and culture are so intertwined that it is almost impossible to even discuss one with out considering the other. Cultural biases are not nearly as much of a concern on the mathematics side of the test.

Still, even in the reasonably objective and unambiguous world of mathematics, there are any number of ways in which background can give an unfair advantage. These include (but are by no means limited to) enrichment activities, role models, high-achieving schools and community culture, support and tutoring, and expensive prep classes.

This last item has become one of if not the central element in the SAT/fairness discussion. All stories on the subject seem to be contractually obligated to talk about expensive test prep courses and yet, as far as I can tell, they all frame the issue in a way that makes the criticisms completely invalid.

There are two fallacies in this standard line of argument: The first is based on confusion over absolute versus relative values; the second is based on a common but profoundly wrong concept of the test itself.

As an absolute statement it is true that if prep courses do any good at all then the ability to pay for them will provide an unfair advantage. The question on the table now, though, is not absolute. The people who are arguing for the elimination of the SAT are also arguing, sometimes implicitly, often explicitly, for grades to take a much larger role in the college selection process to take up the slack. This leads to a very different question: does having money give one a greater advantage on the SAT then it does on GPA?

Private tutoring centers are a huge national industry, and if you send your child to one for any length of time, the cost will probably be far greater than what you would've spent on an SAT prep course. We could, of course, have a long discussion about the intrinsic value of what is taught in one versus the other, but from an economic fairness standpoint, all we care about is the cost and the effectiveness in improving the given metric.

A valid argument here would start with a comparison of the ways that privilege can provide an unfair advantage on the SAT versus GPA, but what we've gotten so far is the pseudo-argument: A is worse than B because A is bad ("French Fries are so fatty; I think I'll have onion rings instead."). As far as I can tell, none of the many stories describing the potential impact of prep courses even mention the existence of the private tutoring industry.

The other fallacy here is the very wrong but very common belief that the SAT is some kind of mysterious black box, the secrets of which can only be revealed by one of the illuminated. I've already been through this at some length but just to reiterate, because of the stability of the tests and the large number of previous editions that the College Board has published, the SAT is one of the most transparent exams you're ever likely to take.

At least on the math side (which is the area I have some experience with), this transparency, along with the nature of the questions, makes the test surprisingly easy to teach and to teach yourself. In the latter case, it goes like this:

Take one of the old tests (don't worry about the time limit);

Check your answers;

Read the explanations for the ones you got wrong;

If you don't understand the explanations for some of those problems, take them to a teacher or administrator and ask for help (as a former teacher, I can tell you that educators love to see this kind of initiative and will go to great lengths to encourage it). There are also free after-school programs that would be glad to help (I volunteer at one of them);

Repeat the process. After you start breaking fifty or sixty percent, work on reducing your time.

If it's this easy, why does anyone bother with a pricey prep course? Well, for one thing, it's not that easy. We are talking about a tremendous amount of work and self-discipline. The courses provide structure and external discipline, not to mention a large dose of motivation and reassurance to counteract the test's foreboding reputation (a problem greatly compounded by journalists' tendency to talk about the exam in dark and mysterious terms).

To sum up, there is tremendous unfairness in our education system. The SAT is sometimes part of that unfairness, but neither for the reasons or to the extent you often hear.