VANDERBILT (US) — Rewarding teachers with incentive pay, in the absence of any other support programs, does not raise student test scores.
Paying teachers bonuses based on their performance has been a controversial issue nationwide since the 1950s, but until now the concept has never been scientifically researched.
“We tested the most basic and foundational question related to performance incentives—does bonus pay alone improve student outcomes?—and we found that it does not,” says Matthew Springer, assistant professor of leadership, policy, and organizations at Vanderbilt University.
“These findings should raise the level of the debate to test more nuanced solutions, many of which are being implemented now across the country, to reform teacher compensation and improve student achievement.”
The Project on Incentives in Teaching, called the POINT Experiment, took place over the 2007 – 2009 school years with participation by mathematics teachers in grades 5 through 8 in Metropolitan Nashville Public Schools.
Nearly 300 teachers, approximately 70 percent of all middle-school math teachers in Nashville’s public schools, volunteered to participate. The complete study, including setup and analysis, began in 2005 and ended in 2010.
POINT tested no other types of incentives or systems of support for the teachers, such as professional development or guidance on instructional practices—many of which have evolved over the five years since POINT began.
“We designed POINT in this manner not because we believed that an incentive system of this type is the most effective way to improve teaching performance, but because the idea of rewarding teachers on the basis of students’ test scores has gained such currency,” Springer explains.
“We sought a clean test of the basic proposition: If teachers know they will be rewarded for an increase in their students’ test scores, will test scores go up? We found that the answer to that question is no. That by no means implies that some other incentive plan would not be successful.”
Here’s how the POINT experiment worked:
- Following a year of detailed project design by a multi-disciplinary team from Vanderbilt and RAND, all middle-school math teachers in Nashville were invited to volunteer for the experiment. Approximately 70 percent of all middle-school math teachers in Nashville’s public schools stepped forward to participate.
- Approximately half of the nearly 300 volunteers were randomly assigned to a “treatment” group, in which they were eligible for bonuses of up to $15,000 per year on the basis of their students’ test-score gains on the Tennessee Comprehensive Assessment Program (TCAP).
- The other half were assigned to a “control” group not eligible for these bonuses. Teachers were evaluated based on an historical performance benchmark for MNPS teachers, not on competition with one another. All teachers in the treatment group had the chance to earn bonuses. (The names of participating teachers—and which group they were in—have been kept confidential by the research team.)
- The annual bonus amounts were $5,000, $10,000 or $15,000. Over the course of the experiment, POINT paid out more than $1.27 million in bonuses. Overall, 33.6 percent of the original group received bonuses, with the average bonus being approximately $10,000.
Teacher attrition occurred during the experiment. About half of the 296 teachers who initially volunteered remained through the end of the third year. The teachers who left the study either left the school system, moved to other grades, or stopped teaching mathematics. Only one participating teacher specifically asked to be removed from the experiment.
While there was no overall effect on student achievement across the entire treatment group, the researchers found a significant benefit for fifth graders in Year 2 and Year 3 of the experiment: fifth graders taught by teachers who earned bonuses did show gains in test scores.
However, the effect did not carry over to sixth grade when students were tested the following year. Springer says this finding raises questions about what is different about fifth grade and what factors—student development, curriculum, teaching, and classroom structure—may have played a role.
He also notes that implementation of POINT went smoothly, with no complaints from teachers about the calculation of bonuses, the payment of awards, bonuses they did or did not receive, or the fairness of the process.
This in itself is a significant finding, Springer says, because historically, teacher associations have opposed performance or merit pay plans, particularly if the pay plan awarded teachers solely on their individual value-added score.
“We believe there is an important lesson here: teachers are more likely to cooperate with a performance pay plan if its purpose is to determine whether the policy is a sound idea, than with plans being forced on them in the absence of such evidence and in the face of their skepticism and misgivings.”
More news from Vanderbilt University: www.vanderbilt.edu/news