2. Value-Added Models (VAMS) and the Human Factor

• VAMs are statistical tools used to measure the purportedly causal relationships between having been instructed by teacher A and achieving a specific growth over a certain time while controlling for various student background characteristics. Gains are measured between current and previous scores, which allows students to act as their own controls. If VAMs are used in your state, check to see which student factors are controlled for such as race, ethnicity, poverty, and English proficiency. VAMs have a norm referenced feel as by definition about 50% of teachers are typically categorized as adding value and the other 50% are categorized as subtracting it. There are also ceiling and floor effects in play as some students have nowhere to go but down or up thanks to mean regression effects.
• Audrey makes a compelling case here that VAMs are unreliable, invalid, biased, unfair, and fraught with measurement errors. They are therefore used inappropriately to make consequential decisions while unintended consequences go unrecognized. Meanwhile, private companies sell their VAM models for profit. The SAS product known as Education Value-Added Assessment System or EVAAS is mentioned prominently here. Audrey uses four case studies of teachers terminated due in part to poor VAM scores from Huston, TX to make her points. Students are not randomly assigned to classes and teachers feel that their scores are largely due to the nature of the students they teach. It is also common for teachers to score high one year and low the next even though they aren’t doing anything different. VAM scores also lack transparency as teachers by in large don’t have any idea about how they work. Even if they do understand it, by the time it arrives the students have moved to the next grade.

3. A VAMoramic View of the Nation

• Audrey has done a fairly exhaustive study of the people who have done research in this field. Her experience shows that 95% of the researchers protest the use of any of the available models. Unfortunately, 44 states and D.C. as of this writing use some form of VAM to evaluate teachers. She also found that many of the remaining 5% have some conflicts of interest as far as profiting from sale of products or the financial interests of their schools. The bulk of this chapter gives the history, model specifics, model use, and model claims and realities for the most widely used models. If your state uses one, it is probably detailed here. In all cases, empirical evidence shows that the models have serious questions regarding reliability, validity, and bias, and should therefore not be used in any high-stakes decision making process.

Share this: