What works may hurt: Side effects in education by Yong Zhao tells about an important lesson that education needs to borrow from medicine. That is the study of side effects. Educational research tends to focus only on proving the effectiveness of practices and policies in pursuit of what works. It has generally ignored the potential harms. This article presents evidence that shows side effects are inseparable from effects. Journal of Educational Change
Download the pdf here. ISSN 1389-2843, J Educ Change, DOI 10.1007/s10833-016-9294-4

Introduction

• Medical research is a field education should emulate. Education researchers have been urged to adopt randomized controlled trials (RCT), a more ‘‘scientific’’ research method believed to have resulted in the advances in medicine. As a result, the RCT is now the gold standard in educational research. The What Works Clearinghouse as of 2015 accepts only studies using RCT as meeting its Group Design Standards without Reservations. The difference is that in education there is less effort to weigh the risks against their effectiveness. In medicine, even after a drug is approved, research on side effects continues.

What Are Side Effects?

• Side effect is defined as ‘‘an unwanted or unexpected result or condition that comes along with the desired effects of something.” In medicine side effects are expected and looked for. Studying and reporting side effects in trials has saved lives. Once side effects are known, effort is placed on finding treatments that are as effective with fewer side effects. In education, however, it is extremely rare to find a study that evaluates both the effectiveness and adverse effects of a product, teaching method, or policy in education. Don’t expect to see warnings like ‘‘this program will raise your students’ test scores in reading, but may make them hate reading forever” on any education product. The only people looking for negative effects in education are those that disagree with a product or policy.

Direct instruction: Instruction that stifles creativity

• Despite the vast amount of research, there is no general agreement whether direct instruction (DI) is an effective approach. Rather than continuing the argument between supporters and detractors of direct instruction, a more rational and productive approach would be for both sides to acknowledge that DI, like all medical products has effects and side effects. With direct or traditional teaching, students tend to do slightly better on achievement tests, but they do slightly worse on tests of abstract thinking, such as creativity and problem solving. When children are shown exactly how to do something, they are less likely to explore and come up with novel solutions. Students who receive instruction first tend to produce only the correct solutions they were told. It is possible for students to show high performance on memory tasks or carrying out problem-solving procedures without a commensurable understanding of what it is that they are doing. As educators we need both effective ways to transmit knowledge and foster creativity. Thus DI has its place. Its side effects, however, need to be minimized.

The best or the worst: The conflicting evidence of performance

• Due to their results on international tests, East Asian education systems have become the object of idolization and a source of ideas for improving education. These systems, however, have somehow made a large number of students lose confidence and interest in math, science, and reading, while helping them achieve excellence in testing. Yong notes that this evidence is still preliminary, but there is a negative correlation between test scores and confidence. The same trend is observed for the United States. If indeed the policies and practices that raise test scores also hurt confidence and attitude, we must carefully weigh the risks against the benefits. Do we care more about test scores or confidence and attitude?

When risks outweigh benefits: Test-based accountability

• America could have avoided the significant damages caused by test-based accountability if side effects had been taken seriously. High stakes testing has been associated with the distortion of instruction, turning teaching into test preparation, cheating, preventing some students from taking the tests, and narrowing of the curriculum among others. States and districts have manipulated drop out rates and misrepresented test results, and both teachers and students have been demoralized. All of this harm has not resulted in closing achievement gaps or improving achievement.

A call to study side effects

• There is no regulation that asks developers of education interventions to study and disclose potential side effects when providing evidence for their effectiveness. The focus, therefore is exclusively on marshaling evidence to show benefits and effects. Consumers only have information of what works, without knowledge of the potential costs. The negative effects of educational products, when occasionally discovered, are not considered an inherent quality of the product or policy. The collateral damages of NCLB could have been anticipated based on Campbell’s Law, which states: ‘‘The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.’’ Reported side effects are often brushed aside as lacking objectivity, scientific rigor, or motivated by ideology.

Recommendations

• 1. Research organizations and academic journals can require research articles to include both main effects and side effects.
• 2. Federal clearing houses such as What Works should include information about the negative effects of educational approaches, methods, products, or policies.
• 3. Education researchers, policy makers, and product developers should voluntarily study side effects and disclose such information.
• 4. Consumers of educational research, policy, and products should ask for information about both effects and side effects.
• 5. Program evaluation should include investigating both effects and side effects.
• 6. Reports of side effects after the implementation of interventions should be considered seriously, instead of discarding them as unintended consequences, improper implementation, or simply complaints by unhappy parents, students, or teachers. It is the responsibility of the policy and product developers’ to investigate and respond to such reports.

Yong Zhao

• Yong Zhao is a Foundation Distinguished Professor in the School of Education at the University of Kansas. He is also a professorial fellow at the Mitchell Institute for Health and Education Policy, Victoria University in Australia. He previously served as the Presidential Chair and Director of the Institute for Global and Online Education in the College of Education, University of Oregon, where he was also a Professor in the Department of Educational Measurement, Policy, and Leadership. His works focus on the implications of globalization and technology on education. He has published over 100 articles and 30 books, including Counting What Counts: Reframing Education Outcomes(2016), Never Send a Human to Do a Machine’s Job: Correcting Top 5 Ed Tech Mistakes (2015), Who’s Afraid of the Big Bad Dragon: Why China has the Best (and Worst) Education System in the World (2014), Catching Up or Leading the Way: American Education in the Age of Globalization (2009)and World Class Learners: Educating Creative and Entrepreneurial Students (2012). He is a recipient of the Early Career Award from the American Educational Research Association. He is an elected fellow of the International Academy for Education. Check out his website and follow him on Twitter @YougZhaoUO.

Share this: