How can we fairly incorporate student achievement data into teacher assessments?
March 22, 2011
StudentsFirst Interviews Dartmouth Professor Douglas Staiger About Value-Added Assessments.
Professor Staiger is a co-author of a Brookings Institution paper titled “Evaluating Teachers: The Important Role of Value-Added,” and has conducted extensive research on teacher effectiveness in affiliation with the National Bureau of Economic Research. Professor Staiger previously served as a faculty member at Stanford and Harvard Universities, and his research is featured many in leading journals.
StudentsFirst: For those new to education policy, what is a value-added assessment?
Professor Staiger: Value-add is a measure of how much a teacher’s students did on their end-of-year test scores compared to what is expected. A teacher is said to have positive “value-added” when the students in his or her class outperform students who had similar starting points – similar prior achievement, similar backgrounds, and similar classmates.
StudentsFirst: How do value-added assessments work?
Professor Staiger: The key part of value-added is coming up with an estimate of what is “expected” for each child. Value-added isn’t magic. We just set an average expectation based on what we know about each child and classroom at the beginning of each year.
Typically, that might include how well the child has done on prior tests. Some tests make adjustments for demographics, such as socioeconomic status or other categories. Finally, some tests may look at the overall classroom environment, raising expectations when you have a group of high-performing students together, and lowering the baseline when you have a cluster of children together who have previously performed poorly.
The value-added assessments combine these elements, compare them to the average performance of other students who had the same characteristics, and produce an “expected” performance as the average for one year for kids in that classroom. For any individual teacher, the value-added assessment measures whether students of that teacher did better or worse than expected.
StudentsFirst: That sounds complicated. How much of a value-added assessment depends upon how it is constructed?
Professor Staiger: The core element is simple: how did the student do prior to having this specific teacher? The question is what else you consider when constructing your baseline expectation. How much weight do you give to socio-economic status? How much weight do you give to the classroom as a whole?
These issues are not problems, however – they are tools. These are important design decisions that schools have to make anyway whether or not you use value-added assessments.
For example, one key question is how much weight you give to “the classroom as a whole.” If you give a lot of weight to it (and not just the individual student), then you lower the baseline for teachers in high-poverty schools – and you thereby make their results look better.
But this is an important conceptual issue and policy design question. Do you want to make it relatively more attractive for teachers to want to teach in high-poverty schools? That’s something you can do with the way you design your school system’s individual value-added baseline.
StudentsFirst: How are value-added assessments used today?
Professor Staiger: The purpose of these measures is to improve education, and there are three ways to do that.
First, we can use them for professional development by identifying top-performing teachers, so that we can learn what they are doing and spread their practices. We can also use it to find professionals who seem to be performing below expectations, and try to target professional development for those teachers. In other words, this first category is about using value-added assessments to help current teachers get better.
Second, we can use value-added assessments as part of a system for categorizing professionals for staffing purposes such as placements, promotions, tenure, and layoffs. We can think of this as using the tool to help us selectively retain and deploy the best teachers.
Third, it’s possible to use value-added assessments to help make decisions in areas such as merit pay. This can either create incentives for better performance, or possibly help the teaching profession better attract those college graduates who prefer incentive-based compensation schemes.
StudentsFirst: Are value-added assessments reliable measures of teacher performance?
Professor Staiger: Reliability is related to stability over time. In other words, does performance tend to be similar year-after-year for the same teachers? If the value-added assessment jumps around without correlation, you might be concerned that it’s not a useful indicator of teacher performance.
In the case of value-add, the year-on-year correlation for individual teachers is about .3, or 30 percent. So, is that a lot of reliability, or not?
Well, it depends on your perspective. If you take the perspective of a student or a parent, then you have to take the data very seriously.
For example, imagine that you are the parent of a student entering a new high school. Imagine further that you’re given a choice of teachers, and you have the option of seeing their value-added assessment scores. If you choose teachers with higher value-added assessment scores, then on average your student will do significantly better on their tests over the course of their education.
If you were that parent, you would want to use the value-added assessment to make your decision. And that’s the decision that school systems face when choosing to use value-added assessments to make decisions about retaining and deploying teachers.
StudentsFirst: How much does a value-added assessment depend upon the individual students that an individual teacher is assigned by random chance?
Professor Staiger: We have conducted controlled research on this specific question. Random assignments of students within schools have demonstrated that value-added assessments are still a reliable indicator of individual teacher performance, by the standards I described a moment ago.
StudentsFirst: So, should schools use value-added assessments as their sole tool for assessing teachers?
Professor Staiger: No. A smart evaluation system would collect and use other sources of information about teacher performance. Value-added assessments only measure performance on tests. There are other types of student performance we care about, such as citizenship and love of learning. Also, even for tests, a 30 percent year-on-year correlation is not perfect. For this reason, we should collect and use well-structured observations by principals or master teachers to supplement our assessments of teachers.
StudentsFirst: If we expand the use of value-added assessments, does that mean more standardized testing?
Professor Staiger: Well, we don’t have to use the same standardized tests we use today. We can and should look for better tests. But, we do need to improve our testing regime. Currently, only 25 percent of teachers are in tested grades and subjects. So, if we want value-added assessments to be anchors for teacher evaluations, we need to expand testing to more grades and more subjects.
StudentsFirst: Does that create a risk of “teaching to the test”?
Professor Staiger: The problem of “teaching to the test” occurs when you put very high stakes on a single measure. In a smart system, you should not make decisions dependent upon a single measure. Instead, you should use value-added assessments as part of a broader professional evaluation. That makes it much harder to “teach to the test” and distort behavior, since evaluators will be able to more easily identify efforts to manipulate this one measure.
StudentsFirst: So are you defending the use of value-added assessments?
Professor Staiger: I’m not wedded to them. We should continually be looking for new, better ways of evaluating teachers. We may find instruments that may be equally useful or even better. But we cannot ignore this data. The analysis of value-added assessments reveals enormous performance gaps that need to be explored.
StudentsFirst: Ideally, what role should value-added assessments play in public education?
Professor Staiger: Ideally it should be a component of teacher evaluations, including professional development and retention. However, it should only be one part of the system. The ideal professional development system would include hard information as well as qualitative information to help everyone make the right decisions.
