Tuesday, October 2, 2012

Jaime Escalante and the full factorial

Just so there isn't any confusion following my previous post, having followed education from lots of angles for a long time (including stints teaching in Watts and the Delta), I have no doubt that Jaime Escalante was the real thing, a genuinely great teacher with exceptional technique and a profound understanding of both the cognitive and emotional aspects of learning.

I brought up the fact that Escalante wasn't able to duplicate his results, not because he was overrated (I honestly don't believe he was), but because the results of even the best teachers are affected by a number of factors and interactions. Escalante was a great teacher in the right school and community with the right administrator at the right time. That was part of why he accomplished so much at a school that most teachers would have struggled with.

The idea that one teacher might do better in school A than in school B while another teacher might do better in B may not seem like that radical a notion but it has big and potentially troubling implications.

Consider three of the factors that might interact with the teacher effect:

Level (remedial, average, advanced);

Class size (small, medium and large);

Administrator (for the sake of the discussion, we'll limit this to two -- A, who keeps a high profile and is liked and respected by the kids and B, who doesn't and isn't).

Both common sense and anecdotal data should alert us to the potential for first, second, even third order interaction here.

As a personal example, my preferred approach to teaching secondary math classes (particularly when students came in below grade level) was to reserve some time at the end for kids to work individually on worksheets and homework while I went from desk to desk to make sure that each student understood the lesson and was doing the problems correctly. Every student got some personal attention and none got left behind.  (By comparison, my college teaching style was mostly lecture/Q&A-based and worked about as well for two hundred students as it did for twenty.)

For a teacher with a style that relied ono one-to-one interaction to help struggling students, you might not see a first order interaction with level and teacher effect (as long as you kept the size small), or with size and effect (as long as you kept the level advanced), but the combination of large and remedial would severely limit the effectiveness of this approach. The administrator and school culture also play a role here -- it's easier to spend time with the kids who are falling behind if the rest of the class is quietly doing its work.

These interactions seem reasonable enough, certainly not the sort of thing you can rule out, but in a real sense, proposals for test based teacher evaluation routinely do just that. Most evaluation periods, by necessity, cover a tiny range of data: one school; one administrator; one subject; one level; one class size. The result is a ridiculously narrow picture of a teacher's performance. If there's a serious potential for interaction, their conclusions can't possibly be valid.

Even if we ignore the potential for interaction, three or four years of confounded, nested data is an awfully thin basis for decisions about bonuses, promotions and dismissals. If we allow for the obvious possibility that some teachers work better with certain students and under certain administrators and in certain environments (as Escalante did), we will either need a fractional (perhaps even full) factorial approach which will require huge samples or we will have to have a sophisticated understanding of just how teaching styles, learning styles and management styles interact not only with each other but with subject matter, school and class structures, adolescent psychology, group dynamics, cultural differences and government policy.

Good luck

No comments:

Post a Comment