Bring red flags, lots of red flags – Part VI: Campbell's Law et al.

The familiar concerns with factors that undermine and distort educational metrics appear to have crossed the ocean.

I asked two experts in statistics — Nat Malkus, from the American Enterprise Institute, and Bryan Graham, from the University of California, Berkeley — to help me evaluate the findings. “This is good evidence of positive effects,” says Malkus. Both pointed out that the study’s results are complicated by Bridge’s high dropout rate: While a third of public-school students dropped out, nearly half of Bridge students left during the study and were unable to take the final assessment. ‘‘The high attrition rate should give one pause,’’ Malkus says, ‘‘when considering the full effect of the program.’’ Graham, co-editor of The Review of Economics and Statistics, says that ‘‘organizations are under a lot of pressure to do these studies and ‘prove’ their program works. Reasonable and informed people could look at the information in that report and come to widely different conclusions about the effect of Bridge on academic achievement as they measure it. It’s information, just not especially actionable information.”

Another area of achievement that Bridge trumpets is the success of its students on the eighth-grade K.C.P.E. test. In 2015, according to Bridge, 63 percent of Bridge students who had been there for at least two years passed, compared with 49 percent of Kenyan students nationwide. But it’s unclear whether Bridge’s approach will be sustainable as the company grows. Former Bridge employees told me that in preparation for the 2015 exam, those on track to get a lower score were asked to repeat a year. The rest were taken to a residential cram school and prepped for the test by teachers who flew in from the United States.