Battelle Blasts Papers decision

From our mailbag, Battelle for Kids condemns the Plain Dealer and NPR's decision to publish teacher's value-added scores, calling it "the poster child for name, blame, and shame and the antithesis of our approach to using value-added data"

To: All SOAR districts
From: Jim Mahoney and Bobby Moore
Date: June 17, 2013

Yesterday, a three-part series on value-added was launched by The Cleveland Plain Dealer and State Impact Ohio. It includes both articles and radio segments specific to value-added analysis as a measure of teacher effectiveness. Highlighted in the articles is a link to a database of teacher ratings, hosted by The Plain Dealer and the State Impact Ohio partnership.

Currently, Ohio laws governing the release of teacher records would apply to teacher value-added results. Thus, teacher level value-added information is subject to public records requests through ODE. Through The Plain Dealer and State Impact Ohio database, the general public can now access a teacher's overall composite rating derived from two years of his/her results in grades 4-8 math and reading. These data reflect information for less than 1/3 of the math and reading, grades 4-8 teachers in Ohio.

Battelle for Kids was not aware these ratings would be published in this way, at this time.

While Battelle for Kids does support the use of value-added information for school improvement and as one of several components of a multi-measures evaluation system, value-added should NOT be used in isolation to draw conclusions about a teacher's effectiveness.

Multiple data points over time from multiple perspectives are crucial because teaching and learning and the evaluation of teaching and learning are complex.

Therefore, we are NOT supportive of these ratings being publically available and discourage promoting the use of this public database.

Talking points and articles, to support your local conversations, are available on the Ohio Student Progress Portal.

http://cts.vresp.com/c/?BattelleForKids/f43a0e1b46/fb8aa9ca4e/313346eb88/sflang=en

Obviously, this is the poster child for name, blame, and shame and the antithesis of our approach to using value-added data.

Please call if you have any questions.

Thank you for all you do for Ohio's students!

-Jim and Bobby

Improving the Budget Bill Part I

Hb 59, the Governor's budget bill can be significantly improved during the legislative process. We're going to detail some of the ways improvements can be made.

Improvements can first start by correcting a major policy flaw inserted into HB555 at the last minute. HB 555 radically changed the method of calculating evaluations for about 1/3 of Ohio's teachers. If a teacher's schedule is comprised only of courses or subjects for which the value-added progress dimension is applicable - then only their value-add score can now be used as part of the 50% of an evaluation based on student growth. Gone is the ability to use multiple measures of student growth - i.e. Student Learning Objectives or SLO's.

Therefore we suggest the legislature correct this wrong-headed policy by repealing this provision of HB555.

Furthermore, greater evaluation fairness could be achieved by lowering the number of absences a student is allowed before their test scores can be excluded from a teacher's value-add score. Currently a student needs to be absent 60 times - or 1/3 of a school year. This is an absurd amount of schooling to miss and still have that student's score count towards the evaluation of his or her teacher. This absence exclusion should be lowered to a more reasonable 15 absences.

Value-add should not be used to punish teachers on evaluations, instead it should be just one component of a multiple measure framework, and a tool to help teachers improve student learning. HB555 moved us much further away from that goal.

Major evaluation change slipped into HB555

Over the Holidays, the Governor signed the latest education bill, HB 555. StateImpact has a decent rundown of most of the items that were addressed in the bill. However, one major piece of policy change made in HB555 has gone totally unreported, not surprisingly since it was done in the 11th hour, with no testimony or hearings. Indeed, we're doubtful many lawmakers even knew it had been slipped in there.

Sec. 3319.112. (A) Not later than December 31, 2011, the state board of education shall develop a standards-based state framework for the evaluation of teachers. The state board may update the framework periodically by adoption of a resolution. The framework shall establish an evaluation system that does the following:

(1) Provides for multiple evaluation factors, including student academic growth which shall account for fifty per cent of each. One factor shall be student academic growth which shall account for fifty per cent of each evaluation. When applicable to the grade level or subject area taught by a teacher, the value-added progress dimension established under section 3302.021 of the Revised Code or an alternative student academic progress measure if adopted under division (C)(1)(e) of section 3302.03 of the Revised Code shall be used in the student academic growth portion of an evaluation in proportion to the part of a teacher's schedule of courses or subjects for which the value-added progress dimension is applicable.

If a teacher's schedule is comprised only of courses or subjects for which the value-added progress dimension is applicable, one of the following applies:

(a) Beginning with the effective date of this amendment until June 30, 2014, the majority of the student academic growth factor of the evaluation shall be based on the value-added progress dimension.

(b) On or after July 1, 2014, the entire student academic growth factor of the evaluation shall be based on the value-added progress dimension. In calculating student academic growth for an evaluation, a student shall not be included if the student has sixty or more unexcused absences for the school year.

If you're not familiar with legislative language, here's the summary

HB 555 radically changes the method of calculating evaluations for about 1/3 of Ohio's teachers. If a teacher's schedule is comprised only of courses or subjects for which the value-added progress dimension is applicable - then only their value-add score can now be used as part of the 50% of an evaluation based on student growth. Gone is the ability to use multiple measures of student growth - ie Student Learning Objectives or SLO's.

Teachers and school districts have spent countless months collaborating on the development and implementation of an evaluation system originally detailed in HB153 - only to now find the rules of the game changed at the 11th hour. Furthermore, the change is regressive. We have detailed the growing list of research that demonstrates the very real and serious problems with heavy reliance on value-add, and the need to offset these problems by using multiple measures of student growth. See here, here, here, and here for examples.

How Do Value-Added Indicators Compare to Other Measures of Teacher Effectiveness?

Via

Highlights

• Value-added measures are positively related to almost all other commonly accepted measures of teacher performance such as principal evaluations and classroom observations.
• While policymakers should consider the validity and reliability of all their measures, we know more about value-added than others.
• The correlations appear fairly weak, but this is due primarily to lack of reliability in essentially all measures.
• The measures should yield different performance results because they are trying to measure different aspects of teaching, but they differ also because all have problems with validity and reliability.
• Using multiple measures can increase reliability; validity is also improved so long as the additional measures capture aspects of teaching we value.
• Once we have two or three performance measures, the costs of more measures for accountability may not be justified. But additional formative assessments of teachers may still be worthwhile to help these teachers improve.

Introduction

In the recent drive to revamp teacher evaluation and accountability, measures of a teacher’s value added have played the starring role. But the star of the show is not always the best actor, nor can the star succeed without a strong supporting cast. In assessing teacher performance, observations of classroom practice, portfolios of teachers’ work, student learning objectives, and surveys of students are all possible additions to the mix.

All these measures vary in what aspect of teacher performance they measure. While teaching is broadly intended to help students live fulfilling lives, we must be more specific about the elements of performance that contribute to that goal – differentiating contributions to academic skills, for instance, from those that develop social skills. Once we have established what aspect of teaching we intend to capture, the measures differ in how valid and reliable they are in capturing that aspect.

Although there are big holes in what we know about how evaluation measures stack up on these two criteria, we can draw some important conclusions from the evidence collected so far. In this brief, we will show how existing research can help district and state leaders who are thinking about using multiple measures of teacher performance to guide them in hiring, development, and retention.

[readon2 url="http://www.carnegieknowledgenetwork.org/briefs/value-added/value-added-other-measures/"]Continue reading...[/readon2]

Studies Give Nuanced Look at Teacher Effectiveness

The massive Measures of Effective Teaching Project is finding that teacher effectiveness assessments similar to those used in some district value-added systems aren't good at showing which differences are important between the most and least effective educators, and often totally misunderstand the "messy middle" that most teachers occupy. Yet the project's latest findings suggest more nuanced teacher tests, multiple classroom observations and even student feedback can all create a better picture of what effective teaching looks like.

Researchers dug into the latest wave of findings from the study of more than 3,000 classes for a standing-room-only ballroom at the American Educational Research Association's annual conference here on Saturday.

"The beauty of multiple measures isn't that there are more of them—more can be more confusing—these need to be alligned to the outcomes we care about," said Steve Cantrell, who oversees the MET project for the Bill & Melinda Gates Foundation. Existing teacher evaluation systems often use indicators that are not effective at guaging student achievement, and moreover that lump teachers into too-simplistic categories.

"The middle is a lot messier than a lot of state policies would lead us to believe," Cantrell said. "Teachers don't fall neatly into quartiles. Based on the practice data, if I look at the quartiles, all that separates the 25th and 75th on a class (observation) instrument is .68—less than 10 percent of the scale distribution. In a lot of systems, the 75th percentile teacher is considered a leader and the 25th percentile considered a laggard. ...This would suggest they're a lot closer than being off by two categories."

[readon2 url="http://blogs.edweek.org/edweek/inside-school-research/2012/04/the_most_and_least_effective.html"]Continue reading...[/readon2]

Top 3 Today

Your top 3 news stories for today

1. Kasich Budget Could Destroy Over 51,000 Jobs
   
   Innovation Ohio conservatively estimates that in the absence of large local tax increases to offset the Administration’s proposed cuts to K-12 education, higher education, state personnel, and local governments, 51,052 direct jobs are likely to be lost.
2. BUDGET BILL MIRRORS SB5 ATTACKS ON TEACHERS
   The fight opens up on multiple fronts.
3. Will upset race in Wisconsin energize SB5 referendum race in Ohio?
   What do you think?

Join us at facebook.com/JointheFuture or leave a comment (click on the post title).