Battelle Blasts Papers decision

From our mailbag, Battelle for Kids condemns the Plain Dealer and NPR's decision to publish teacher's value-added scores, calling it "the poster child for name, blame, and shame and the antithesis of our approach to using value-added data"

To: All SOAR districts
From: Jim Mahoney and Bobby Moore
Date: June 17, 2013

Yesterday, a three-part series on value-added was launched by The Cleveland Plain Dealer and State Impact Ohio. It includes both articles and radio segments specific to value-added analysis as a measure of teacher effectiveness. Highlighted in the articles is a link to a database of teacher ratings, hosted by The Plain Dealer and the State Impact Ohio partnership.

Currently, Ohio laws governing the release of teacher records would apply to teacher value-added results. Thus, teacher level value-added information is subject to public records requests through ODE. Through The Plain Dealer and State Impact Ohio database, the general public can now access a teacher's overall composite rating derived from two years of his/her results in grades 4-8 math and reading. These data reflect information for less than 1/3 of the math and reading, grades 4-8 teachers in Ohio.

Battelle for Kids was not aware these ratings would be published in this way, at this time.

While Battelle for Kids does support the use of value-added information for school improvement and as one of several components of a multi-measures evaluation system, value-added should NOT be used in isolation to draw conclusions about a teacher's effectiveness.

Multiple data points over time from multiple perspectives are crucial because teaching and learning and the evaluation of teaching and learning are complex.

Therefore, we are NOT supportive of these ratings being publically available and discourage promoting the use of this public database.

Talking points and articles, to support your local conversations, are available on the Ohio Student Progress Portal.

http://cts.vresp.com/c/?BattelleForKids/f43a0e1b46/fb8aa9ca4e/313346eb88/sflang=en

Obviously, this is the poster child for name, blame, and shame and the antithesis of our approach to using value-added data.

Please call if you have any questions.

Thank you for all you do for Ohio's students!

-Jim and Bobby

The weakest "linkage"

Many changes are starting to ripple down to the classroom level as Ohio moves forward with its efforts to implement corporate education reform. One of those changes is the creation and increasing use of teacher level value add reports. We provided some basic background on value add here if you need a refresher.

One of the most important steps in producing these complex reports for each teacher is to know which teacher taught which student, in which subject, and for how long. We need to know this for every student and every teacher. It's a process called "linkage". Without this linkage teachers could not be credited with the instruction they provided to each student.

By 2013, it will not be just RttT districts and Battelle for Kids’ projects that will require this linkage to occur, but all school districts must “implement a classroom-level value-added program (HB 153; Section 3319.112(A)(7)).

These teacher-level value-added reports will be used to determine teacher effectiveness and will be a significant factor in teacher evaluations. So it is clear that being able to accurately link student to teacher per subject is going to be critical if this system has any hope of working fairly.

If one imagines common scenarios such as students moving, teachers getting sick and having a sub, and one multiplies that by over 120,000 teachers in Ohio and almost 2 million students - the opportunity for linkage error is simply massive, only surpassed by the sheer magnitude of the administrative effort needed to keep this whole enterprise from unravelling.

Battelle for Kids has spent part of the summer providing some training and webinars on this issue.

In spring 2010, more than 125,000 rosters were verified by educators in South Carolina, Texas, Ohio and Oklahoma. Recent analyses of linkage results from schools across the country yield alarming results, including

Everytime a student moves, someone will have to go into a computer system and remove them from each teacher's roster, and then when that student enrolls in a new school, someone will have to go into a computer system and add them to each of their new teacher's rosters. Ditto for students changing classes, ditto for teachers needing to be replaced and on and on. Hundreds of thousands of changes throughout the school year will need to be performed, and all this on top of hoping the initial set up of millions of teacher/student linkages is error free each year to begin with!

Because Value add is longitudinal, i.e. results from previous years are used to make current year calculations, any errors from previous years will also be carried over, so it isn't like each year allows for a fresh start either. Indeed, as time rolls by, the errors may be compounding.

According to Battelle's own presentation this system hasn't worked to date in South Carolina, Texas, Oklahoma, nor Ohio - which is set to expand it to every school district.

How much confidence can anyone have in a system that will be used for high stakes decisions such as pay and employment, that relies on such gargantuan administrative tracking that has proven to be as utterly unreliable as this?

Link Before You Leap