Claim tying higher value-added scores for teachers to lifelong student success is unjustified, according to new review

Do higher “value-added” scores for teachers lead to greater economic success for their students later in life?

Contrary to the conclusions in a recent well-publicized paper by economists Raj Chetty, John Friedman, & Jonah Rockoff, a review produced by National Education Policy Center (NEPC) finds that claim is unjustified.

“Despite widespread references to this study in policy circles, the shortcomings and shaky extrapolations make this report misleading and unreliable for determining educational policy,” Professor Moshe Adler of the Department of Urban Planning at Columbia University concluded in his review. His review raised at least nine different concerns, including the improper use of prior research and the failure to report important results when those results contradicted the authors’ conclusions.

The claims now being questioned made their way into President Obama’s State of the Union message two years ago and also surfaced in a ruling by a California judge, in the Vergara case, who found that certain due process protections for teachers violated that state’s constitution.

Adler and the Harry Van Arsdale, Jr. Center for Labor Studies at Empire State College,
SUNY reviewed the Chetty-Friedman-Rockoff claims for the Think Twice think tank review project. The National Education Policy Center (NEPC) produced the review with funding from
the Great Lakes Center for Education Research and Practice.

Adler’s review covers Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates and Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood. The National Bureau of Economic
Research (NBER) published both papers as working papers.

In his review Adler found five problems that invalidate the paper’s main claim:

  • The paper fails to mention the existence of a crucial conflicting result that was reported in an earlier version of the same paper.
  • The paper claims that there was insufficient data to investigate whether teacher value-added has an effect on income at any age after the age of 28.
    This claim is untrue.
  • The method used to calculate the effect of teacher value-added on income at age 28 is biased and misleading
  • The paper makes assumptions that inflate its main result, but these assumptions are contradicted by the evidence
  • The studies that the paper cites in support of its methodology don’t actually provide that support.

In conclusion, Adler writes, “The two papers under review here use questionable techniques to reach conclusions that are not supported by the data. These
problems rend the papers of no value in guiding educational policy.”

Find this Think Twice Review on the Great Lakes Center website:

http://www.greatlakescenter.org

Find both reports on the National Bureau of Economic Research website:

http://www.nber.org/papers/w19423
; http://www.nber.org/papers/w19424

Think Twice, a project of the National Education Policy Center (NEPC), provides the public, policymakers and the press with timely,
academically sound reviews of selected publication. The project is made possible with funding from the Great Lakes Center for Education Research and Practice.

The review can also be found on the NEPC website:

http://nepc.colorado.edu