On Teacher Evaluations, Between Myth And Fact Lies Truth

Controversial proposals for new teacher evaluation systems have generated a tremendous amount of misinformation. It has come from both “sides,” ranging from minor misunderstandings to gross inaccuracies. Ostensibly to address some of these misconceptions, the advocacy group Students First (SF) recently released a “myth/fact sheet” on evaluations.

Despite the need for oversimplification inherent in “myth/fact” sheets, the genre can be useful, especially about topics such as evaluation, about which there is much confusion. When advocacy groups produce them, however, the myths and facts sometimes take the form of “arguments we don’t like versus arguments we do like.”

This SF document falls into that trap. In fact, several of its claims are a little shocking. I would still like to discuss the sheet, not because I enjoy picking apart the work of others (I don’t), but rather because I think elements of both the “myths” and “facts” in this sheet could be recast as “dual myths” in a new sheet. That is, this document helps to illustrate how, in many of our most heated education debates, the polar opposite viewpoints that receive the most attention are often both incorrect, or at least severely overstated, and usually serve to preclude more productive, nuanced discussions.

Let’s take all four of SF’s “myth/fact” combinations in turn.

I’m almost tempted to leave this one alone. Putting aside the opinion-dressed-as-fact that “meaningful” evaluations must include “objective measures of student academic growth,” it’s fair to argue, with caution, that the value-added components of new teacher evaluations would not necessarily “penalize” teachers whose students begin the year far behind, as the models attempt to control for prior achievement (though the same may not be stated so strongly for other components, such as observations).

However, whether the models account for lower-scoring students covers only a small slice of the real issue here. Rather, as I’ve noted before, the better guiding question is whether they can to an acceptable degree account for all the factors that are outside of teachers’ control.

[readon2 url="http://shankerblog.org/?p=8093"]Continue reading...[/readon2]

Do Different Value-Added Models Tell Us the Same Things?

Via

Highlights

• Statistical models that evaluate teachers based on growth in student achievement differ in how they account for student backgrounds, school, and classroom resources. They also differ by whether they compare teachers across a district (or state) or just within schools.
• Statistical models that do not account for student background factors produce estimates of teacher quality that are highly correlated with estimates from value-added models that do control for student backgrounds, as long as each includes a measure of prior student achievement.
• Even when correlations between models are high, different models will categorize many teachers differently.
• Teachers of advantaged students benefit from models that do not control for student background factors, while teachers of disadvantaged students benefit from models that do.
• The type of teacher comparisons, whether within or between schools, generally has a larger effect on teacher rankings than statistical adjustments for differences in student backgrounds across classrooms.

Introduction

There are good reasons for re-thinking teacher evaluation. As we know, evaluation systems in most school districts appear to be far from rigorous. A recent study showed that more than 99 percent of teachers in a number of districts were rated “satisfactory,” which does not comport with empirical evidence that teachers differ substantially from each other in terms of their effectiveness. Likewise, the ratings do not reflect the assessment of the teacher workforce by administrators, other teachers, or students.

Evaluation systems that fail to recognize the true differences that we know exist among teachers greatly hamper the ability of school leaders and policymakers to make informed decisions about such matters as which teachers to hire, what teachers to help, which teachers to promote, and which teachers to dismiss. Thus it is encouraging that policymakers are developing more rigorous evaluation systems, many of which are partly based on student test scores.

Yet while the idea of using student test scores for teacher evaluations may be conceptually appealing, there is no universally accepted methodology for translating student growth into a measure of teacher performance. In this brief, we review what is known about how measures that use student growth align with one another, and what that agreement or disagreement might mean for policy.

[readon2 url="http://www.carnegieknowledgenetwork.org/briefs/value-added/different-growth-models/"]Continue reading...[/readon2]

Do Value-Added Methods Level the Playing Field for Teachers?

Via

Highlights

• Value-added measures partially level the playing field by controlling for many student characteristics. But if they don't fully adjust for all the factors that influence achievement and that consistently differ among classrooms, they may be distorted, or confounded (An estimate of a teacher’s effect is said to be confounded when her contribution cannot be separated from other factors outside of her control, namely the the students in her classroom.)
• Simple value-added models that control for just a few tests scores (or only one score) and no other variables produce measures that underestimate teachers with low-achieving students and overestimate teachers with high-achieving students.
• The evidence, while inconclusive, generally suggests that confounding is weak. But it would not be prudent to conclude that confounding is not a problem for all teachers. In particular, the evidence on comparing teachers across schools is limited.
• Studies assess general patterns of confounding. They do not examine confounding for individual teachers, and they can't rule out the possibility that some teachers consistently teach students who are distinct enough to cause confounding.
• Value-added models often control for variables such as average prior achievement for a classroom or school, but this practice could introduce errors into value-added estimates.
• Confounding might lead school systems to draw erroneous conclusions about their teachers – conclusions that carry heavy costs to both teachers and society.

Introduction

Value-added models have caught the interest of policymakers because, unlike using student tests scores for other means of accountability, they purport to "level the playing field." That is, they supposedly reflect only a teacher's effectiveness, not whether she teaches high- or low-income students, for instance, or students in accelerated or standard classes. Yet many people are concerned that teacher effects from value-added measures will be sensitive to the characteristics of her students. More specifically, they believe that teachers of low-income, minority, or special education students will have lower value-added scores than equally effective teachers who are teaching students outside these populations. Other people worry that the opposite might be true - that some value-added models might cause teachers of low-income, minority, or special education students to have higher value-added scores than equally effective teachers who work with higher-achieving, less risky populations.

In this brief, we discuss what is and is not known about how well value-added measures level the playing field for teachers by controlling for student characteristics. We first discuss the results of empirical explorations. We then address outstanding questions and the challenges to answering them with empirical data. Finally, we discuss the implications of these findings for teacher evaluations and the actions that may be based on them.

[readon2 url="http://www.carnegieknowledgenetwork.org/briefs/value-added/level-playing-field/"]Continue reading...[/readon2]

How Recent Education Reforms Undermine Local School Governance

Via

Local control has historically been a prominent principle in education policymaking and governance. Culminating with the passage of No Child Left Behind (NCLB), however, the politics of education have been nationalized to an unprecedented degree, and local control has all but disappeared as a principle framing education policymaking.

This brief examines what the eclipse of local control means for our democracy. It distinguishes two dimensions of democracy that are at issue—democratic policymaking and democratic education—and concludes that the effect of NCLB has been to frustrate our democracy along both of these dimensions.

Pb Localcontrol

Vote Yes on Issue 2 - Had Enough Early Vote Tour

The Issue 2 campaign is about to kick off a Vote Yes on Issue 2 - Had Enough Early Vote Tour. You can see their list of stops, and get involved, here.

This comes on the heels of the Toledo Blade endorsing a YES on Issue 2

Issue 2 on this fall’s statewide ballot enables voters to start to reclaim Ohio’s election machinery from the partisan politicians and their special-interest allies who now control it. The reform proposal merits a strong YES vote.

The ballot proposal would amend the Ohio Constitution to change the way district boundaries are revised for the state’s U.S. House delegation and the General Assembly after every federal census. Such redistricting largely determines the level of party competition within Ohio, a battleground state in national elections.

That process now is dictated by the Republican Party, which dominates the legislature and state Apportionment Board — and thus, the drawing of political maps. Republicans have rigged the maps in their favor, giving themselves the edge to win as many as 12 of Ohio’s 16 U.S. House seats and to keep control of both legislative houses for another decade.

Read the entire endorsement here.

The real fight over SB5 is still ahead

Yesterday was the filing deadline for candidates wishing to run for the Ohio General Assembly. We had looked earlier at the impact of incumbents of the Ohio House of Representatives voting for SB5 would have on their reelection chances.

14 SB5 supporters could not survive a 5% swing from their margin of victory in 2010 (2 didn’t even reach the 50% threshold due to a third party taking significant support). With only a 10-seat margin to maintain control, it is quite possible that control of the Ohio House will swing away from the Republicans and back to the Democrats.

Such a swing, could put a halt to the Governors radical agenda and turn the remaining 2 years of his first term into a lame duck effort.

Now some of this calculation is complicated by the recent redistricting, but as Gongwer notes, the 2012 elections are shaping up to be a continuation of the fight over SB5

SB5 Redux?: In some ways, the contest for control of the House next session is shaping up as a proxy battle between the two sides in the fight over the collective bargaining law changes (SB 5) that voters rejected last month in a referendum vote.

House Democrats, for example, noted that a number of educators have filed to run and Speaker Batchelder said the GOP newcomers include an ample amount of businesspeople.
[...]
Rep. Debbie Phillips (D-Athens), the House Democratic Caucus Campaign co-chair, said 2002 Teacher of the Year Maureen Reedy, who is seeking the open 24th House District seat in Franklin County, is among at least 10 teachers running for the House as Democrats.

"State budget cuts and the unfair attacks in SB5 have put educators and our children's education directly in the crosshairs of the Republican's anti-middle class agenda and teachers are standing up, fighting back and getting involved," Rep. Phillips said in a release. "We are very excited to have so many great teachers running for office. They are trusted and well known in their communities, which are two key components of electoral success."

While some candidates might have a difficult task ahead of them due to the gerrymandering of districts, the overwhelming rejection of SB5 is likely to create some very sharp contrasts for voters to decide upon.