4. Campbell’s Law

• This law states that “the more any quantitative social indicator is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” In this chapter, Daniel tells stories from the auto industry, the health care system, and the Soviet Union. The Volkswagen scandal resulted from the fact the industry was only looking at emissions while cars were idling. The engineers gamed the software to produce good numbers under these circumstances. In healthcare, the incentives suggest that doctors should not operate on patients who might not survive. The old Soviet Union metrics were also gamed heavily. In Schools, the high testing pressure has lead to cheating, excluding students who will generate bad scores and the lowering of standards. The main point here is that whenever we do something, we need to look at the negative side effects. Be sure to read my summary of Young Zhao’s paper on the topic: What works may hurt: Side effects in education.

5. Score Inflation

• The National Assessment of Educational Progress (NAEP) is a set of tests widely considered to be the gold standard for evaluating educational trends. Over the years the scores on these tests have shown little if any improvement. Score trends in some states, however, have shown steady growth. The suspicion is that some states are dumbing down the exams so that scores go up. This would seem to happen to avoid political risk for those people in a position to do just this. Another trick is to create new and more difficult tests and let the public know that they should expect lower scores at first. As teachers get used to the new tests, scores got up and the leaders look good. There is no credible study in the US that has not found grade inflation. Test preparation is more pervasive in poor schools as is test inflation. There is suspicion that some teachers focus on kids on the pass/fail borderline.

6. Cheating

• Here Daniel covers known cheating scandals and explains the most common ways that cheating occurs. They involve teachers erasing wrong answers and replacing them with correct answers, giving students the questions ahead of time, giving students help during the test, and not giving tests to students who they know will do poorly. Cheating can be spotted by scanners that can read erasures and when scores in a school or a class increase a great deal. The people in the best position to carry out investigations of possible cheating are often the same people who have the most to lose if cheating is uncovered. While we know cheating goes on and is promoted by the high-stakes nature of the tests, we don’t know the extent to which it takes place.

Share this: