Sunday, March 15, 2009

"You Can't Measure Teacher Performance"

When I bring up the issue of results orientation, I consistently hear one major objection: "You can't measure teacher performance. There are too many factors from the students' home lives and previous education. Thus standardized test scores cannot be used." Let me explain why I do not think this is a valid objection.

First, those of us who work in non-education industries are very familiar with being evaluated. As an engineer, each year I work on various projects that are almost impossible to compare with other engineers' work. How do they evaluate engineers? Well, my manager has to look at what he knows about the design, comparing it to previous years, and weighing effects of external factors. In addition, he has to figure out how the difficulty of my designs this year compared with those I worked on in previous years, and those other engineers are working on. It's not an exact science, and can sometimes feel unfair-- but in order for our company to continuously improve and ensure success, the evaluation must be done.

How does this relate to teaching? I think to start with, evaluating teaching has a huge advantage over evaluating engineering: a large sample size. If you look at a particular class and track, such as 9th-grade college-prep algebra, each year you typically have several dozen students taking the class, spread across several classrooms. In addition, those students tend to be roughly comparable to a similar group from the same school the year before, and to similar groups in neigboring schools. If we assume that there are a large number of students impacted by an unfair home life, let's throw out the bottom 20%, for example, of scores when looking at test scores. Of those remaining, we can further reduce the influence of outliers by using medians rather than averages, or similar techniques.

In this case, I think looking at the large set of test scores, and comparing to similar groups of students, should be quite informative. It should be supplemented with classroom observations, student opinion surveys, and other available evaluation instruments. While no evaluation system can ever be perfect, I think this would still be likely to be much more accurate than systems used routinely by successful corporations. Overall, if I have a child in the school, my bottom line is how much he or she really learned in the class. If we can measure that to a decent level of approximation, and reward and learn from the most consistently successful teachers, I think that will be a recipe for increased success.

2 comments:

  1. Eric, I am an electrical engineer (no longer practicing)and now a secondary high school math teacher for the past 9 years. I find that increasingly, more students enter my math class without the proper mastery of essential skills for success. For example, if I were to give a 5th grade test to my algebra 2 classes on operations and understanding of ratios in all its forms (arithmetic with fractions, decimals, and percents), well over 75% would achieve a score below 50%. With this kind of input, I would argue that it would be quite unfair to measure what these students have learned in an Algebra 2 class.

    ReplyDelete
  2. Hi "Math Teacher"-- Glad to see people are still interested in what I wrote, even though I lost!

    As for your point: I agree, it would be unfair to compare those students to well-prepared students.

    *But*-- I think it would be fair to compare them to a class of similarly-prepared students taught Algebra 2 by another teacher.

    In other words, with careful work, assuming you're not the only teacher stuck with such a group (and I can assure you you're not!), fair measures can be prepared.

    ReplyDelete