Value-added: How Ohio is destroying a profession

We ended the week last week with a post titled "The 'fun' begins soon", which took a look at the imminent changes to education policy in Ohio. We planned on detailing each of these issues over the next few weeks.

Little did we know that the 'fun' would begin that weekend. It came in the manner of the Cleveland Plain Dealer and NPR publishing a story on the changing landscape of teacher evaluations titled "Grading the Teachers: How Ohio is Measuring Teacher Quality by the Numbers".

It's a solid, long piece, worth the time taken to read it. It covers some, though not all, of the problems of using value-added measurements to evaluate teachers

Those ratings are still something of an experiment. Only reading and math teachers in grades four to eight get value-added ratings now. But the state is exploring how to expand value-added to other grades and subjects.

Among some teachers, there’s confusion about how these measures are calculated and what they mean.

“We just know they have to do better than they did last year,” Beachwood fourth-grade teacher Alesha Trudell said.

Some of the confusion may be due to a lack of transparency around the value-added model.

The details of how the scores are calculated aren’t public. The Ohio Education Department will pay a North Carolina-based company, SAS Institute Inc., $2.3 million this year to do value-added calculations for teachers and schools. The company has released some information on its value-added model but declined to release key details about how Ohio teachers’ value-added scores are calculated.

The Education Department doesn’t have a copy of the full model and data rules either.

The department’s top research official, Matt Cohen, acknowledged that he can’t explain the details of exactly how Ohio’s value-added model works. He said that’s not a problem.

Evaluating a teacher on a secret formula isn't a practice that can be sustained, supported or defended. The article further details a common theme we hear over and over again

But many teachers believe Ohio’s value-added model is essentially unfair. They say it doesn’t account for forces that are out of their control. They also echo a common complaint about standardized tests: that too much is riding on these exams.

“It’s hard for me to think that my evaluation and possibly some day my pay could be in a 13-year-old’s hands who might be falling asleep during the test or might have other things on their mind,” said Zielke, the Columbus middle school teacher.

The article also performs analysis on several thousands value add scores, and that analysis demonstrates what we have long reported, that value-add is a poor indicator of teacher quality, with too many external factors affecting the score

A StateImpact/Plain Dealer analysis of initial state data suggests that teachers with high value-added ratings are more likely to work in schools with fewer poor students: A top-rated teacher is almost twice as likely to work at a school where most students are not from low-income families as in a school where most students are from low-income families.
[…]
Teachers say they’ve seen their value-added scores drop when they’ve had larger classes. Or classes with more students who have special needs. Or more students who are struggling to read.

Teachers who switch from one grade to another are more likely to see their value-added ratings change than teachers who teach the same grade year after year, the StateImpact/Plain Dealer analysis shows. But their ratings went down at about the same rate as teachers who taught the same grade level from one year to the next and saw their ratings change.

What are we measuring here? Surely not teacher quality, but rather socioeconomic factors and budget conditions of the schools and their students.

Teachers are intelligent people, and they are going to adapt to this knowledge in lots of unfortunate ways. It will become progressively harder to districts with poor students to recruit and retain the best teachers. But perhaps the most pernicious effect is captured at the end of the article

Stephon says the idea of Plecnik being an ineffective teacher is “outrageous.”

But Plecnik is through. She’s quitting her job at the end of this school year to go back to school and train to be a counselor — in the community, not in schools.

Plecnik was already frustrated by the focus on testing, mandatory meetings and piles of paperwork. She developed medical problems from the stress of her job, she said. But receiving the news that despite her hard work and the praise of her students and peers the state thought she was Least Effective pushed her out the door.

“That’s when I said I can’t do it anymore,” she said. “For my own sanity, I had to leave.”

The Cleveland Plain Dealer and NPR then decided to add to this stress by publishing individual teachers value-added scores - a matter we will address in our next post.

Adequately funding my school

JTF recently recieved this essay from Worthington City Schools senior, Hassan Mizra.

Education helps broaden the minds of young individuals to help them achieve success in the future. All across America students go to school to learn and prepare for their futures. Just imagine the students walking into a classroom with new desks with four stable legs, new chairs that aren’t cracked or missing parts and sitting down to their personal laptop provided by the school. This sounds like a school that most parents would want their children to attend. Wouldn’t it be great if all schools had all these? Wouldn’t it be fascinating if all students were able to partake in an improved quality of education for every school?

The state's previous budget cut public school funding by $1.8 billion, which ultimately hurt Ohio's public education system. Ohio Governor John Kasich introduced a new state budget, which proposed a reform of the school funding formula. The new budget promised more money to the less funded school districts, but the promise proved empty. But the Governor, through the Republican dominated legislature, is doing the exact opposite. They are continuing to underfund public schools while increase funding to charter schools and further pushing public education into the hands of private corporations.

For the 2011-12 school year, Worthington Schools received $54,952,536 all total funds, and Olentangy Local Schools received $50,863,323. These two districts are 2 of the best in central Ohio, where they benefit from higher than typical property valuations. An essential aspect for each of these districts' high ratings is because they receive the necessary funding that a public school district should have.

However, the underfunding of public schools in Ohio is an enormous issue that affects many people, especially students. There is a need to re-work the current formula used by Ohio to determine how school funds are disbursed and also to increase public support for education funding. Limited funds for public schools have primarily affected the poor and have put them at a disadvantage in getting a quality education. Whitehall, generally speaking, has a lower property valuation. The schools in Whitehall do not receive the sufficient funding they deserve as a public school district.

Unequal funding throughout the state demonstrates the unfairness some school districts face. Is it fair that schools that reside in low property valuation areas don’t have the necessities to educate their youth?

The reason that some schools can't do things like buy computers and maintain their buildings to begin with is because the school funding system is so ineffective. The US government pays only 7% of all school money, and the rest is up to the state and the local tax-payers. Whatever money the state won't pay is paid as school income tax or property taxes, which are higher or lower depending on how much the property is worth, and the incomes of the districts residents. But this means that schools in poor neighborhoods get little money while wealthy schools get nearly all they need.

The Governor needs to start funding public education fairly and adequately. According to the Ohio Association of Independent Schools (OAIS) the vast majority Ohio students, roughly 1.85 million attend public schools, so it would make sense for the Governor to turn his attention towards public education. Despite whether a child's parents are wealthy or poor, it is in everyone's interest to guarantee that America's future generations are both highly skilled and well educated.

What if Finland’s great teachers taught in U.S. schools?

Finland’s Pasi Sahlberg is one of the world’s leading experts on school reform and the author of the best-selling “Finnish Lessons: What Can the World Learn About Educational Change in Finland?” In this piece he writes about whether the emphasis that American school reformers put on “teacher effectiveness” is really the best approach to improving student achievement.

Many governments are under political and economic pressure to turn around their school systems for higher rankings in the international league tables. Education reforms often promise quick fixes within one political term. Canada, South Korea, Singapore and Finland are commonly used models for the nations that hope to improve teaching and learning in their schools. In search of a silver bullet, reformers now turn their eyes on teachers, believing that if only they could attract “the best and the brightest” into the teaching profession, the quality of education would improve.

“Teacher effectiveness” is a commonly used term that refers to how much student performance on standardized tests is determined by the teacher. This concept hence applies only to those teachers who teach subjects on which students are tested. Teacher effectiveness plays a particular role in education policies of nations where alternative pathways exist to the teaching profession.

In the United States, for example, there are more than 1,500 different teacher-preparation programs. The range in quality is wide. In Singapore and Finland only one academically rigorous teacher education program is available for those who desire to become teachers. Likewise, neither Canada nor South Korea has fast-track options into teaching, such as Teach for America or Teach First in Europe. Teacher quality in high-performing countries is a result of careful quality control at entry into teaching rather than measuring teacher effectiveness in service.

In recent years the “no excuses”’ argument has been particularly persistent in the education debate. There are those who argue that poverty is only an excuse not to insist that all schools should reach higher standards. Solution: better teachers. Then there are those who claim that schools and teachers alone cannot overcome the negative impact that poverty causes in many children’s learning in school. Solution: Elevate children out of poverty by other public policies.

For me the latter is right. In the United States today, 23 percent of children live in poor homes. In Finland, the same way to calculate child poverty would show that figure to be almost five times smaller. The United States ranked in the bottom four in the recent United Nations review on child well-being. Among 29 wealthy countries, the United States landed second from the last in child poverty and held a similarly poor position in “child life satisfaction.” Teachers alone, regardless of how effective they are, will not be able to overcome the challenges that poor children bring with them to schools everyday.

[readon2 url="http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/05/15/what-if-finlands-great-teachers-taught-in-u-s-schools-not-what-you-think/?wprss=rss_education"]Continue reading...[/readon2]

The Foolish Endeavor of Rating Ed Schools by Graduates’ Value-Added

Via School Finance 101.

Knowing that I’ve been writing a fair amount about various methods for attributing student achievement to their teachers, several colleagues forwarded to me the recently released standards of the Council For the Accreditation of Educator Preparation, or CAEP. Specifically, several colleagues pointed me toward Standard 4.1 Impact on Student Learning:

4.1.The provider documents, using value-added measures where available, other state-supported P-12 impact measures, and any other measures constructed by the provider, that program completers contribute to an expected level of P-12 student growth.

http://caepnet.org/commission/standards/standard4/

Now, it’s one thing when relatively under-informed pundits, think tankers, politicians and their policy advisors pitch a misguided use of statistical information for immediate policy adoption. It’s yet another when professional organizations are complicit in this misguided use. There’s just no excuse for that! (political pressure, public polling data, or otherwise)

The problems associated with attempting to derive any reasonable conclusions about teacher preparation program quality based on value-added or student growth data (of the students they teach in their first assignments) are insurmountable from a research perspective.

Worse, the perverse incentives likely induced by such a policy are far more likely to do real harm than any good, when it comes to the distribution of teacher and teaching quality across school settings within states.

First and foremost, the idea that we can draw this simple line below between preparation and practice contradicts nearly every reality of modern day teacher credentialing and progress into and through the profession:

one teacher prep institution –> one teacher –> one job in one school –> one representative group of students

The modern day teacher collects multiple credentials from multiple institutions, may switch jobs a handful of times early in his/her career and may serve a very specific type of student, unlike those taught by either peers from the same credentialing program or those from other credentialing programs. This model also relies heavily on minimal to no migration of teachers across state borders (well, either little or none, or a ton of it, so that a state would have a large enough share of teachers from specific out of state institutions to compare). I discuss these issues in earlier posts.

Setting aside that none of the oversimplified assumptions of the linear diagram above hold (a lot to ignore!), let’s probe the more geeky technical issues of trying to use VAM to evaluate ed school effectiveness.

There exist a handful of recent studies which attempt to tease out certification program effects on graduate’s student’s outcomes, most of which encounter the same problems. Here’s a look at one of the better studies on this topic.

• Mihaly, K., McCaffrey, D. F., Sass, T. R., & Lockwood, J. R. (2012). Where You Come From or Where You Go?

Specifically, this study tries to tease out the problem that arises when graduates of credentialing programs don’t sort evenly across a state. In other words, a problem that ALWAYS occurs in reality!

Researchy language tends to downplay these problems by phrasing them only in technical terms and always assuming there is some way to overcome them with statistical tweak or two. Sometimes there just isn’t and this is one of those times!

[readon2 url="http://schoolfinance101.wordpress.com/2013/02/25/revisiting-the-foolish-endeavor-of-rating-ed-schools-by-graduates-value-added/"]Continue reading...[/readon2]

Education News for 04-04-2013

Local Education News

• Toledo Public Schools audit identifies $100M in savings over 5 years (Toledo Blade)

A private consulting firm that conducted a performance audit of Toledo Public Schools recommends moves the firm says would save the district about $100 million over five years…Read more…

• State Auditor: Noncompliance Findings For Columbus City Schools (WBNS)

State Auditor Dave Yost released documents overnight detailing findings from an annual audit of the Columbus City School district.…Read more…

• Strongsville Strike: Negotiation meeting ends with no deal (WOIO)

A federal mediator met with both sides in the ongoing Strongsville teachers strike Wednesday morning. But after 14 hours of negotiating, no deal was made.…Read more…

• Assaulting bus drivers, passing school buses could bring stiffer penalties in Cleveland (Cleveland Plain Dealer)

City Council's Public Safety Committee approved two ordinances this morning that would stiffen the penalty for passing school buses and add assaults on a public transit worker to the list of misconduct on a public bus or train.…Read more…

• Left behind? (Marietta Times)

Some local school officials are concerned about having the hardware and capacity to administer new computer-based state tests set to debut in 2014-15.…Read more…

• Monroe approves superintendent contract (Middletown Journal)

The Monroe school board in late March approved a two-year contract for Dr. Phil Cagwin as superintendent, confirmed district treasurer Holly Cahall.…Read more…

• Schools face up to $1.3M cuts (Springfield News-Sun)

Federal sequestration budget cuts could cost Clark and Champaign County schools up to $1.3 million for tutors and other staff members that help disadvantaged students, at the same time that districts are facing new state reading requirements.…Read more…

• Preschool project hinging on quality (Cincinnati Enquirer)

“A chair for every child in a quality preschool.” That’s the mission of Preschool Promise, an ambitious plan to expand preschool access to 1,000 more children from low- and middle- income families annually in Greater Cincinnati……Read more…

• Mock school shooting gives true experience in Boardman (Youngstown Vindicator)

Two Boardman High School students lay helpless, one in a first-floor hallway bleeding from her arm, one on the stairs missing half of his leg.…Read more…

• Activists complain of inaction by Youngstown school board (Youngstown Vindicator)

A group of activists and city school parents, frustrated with what they see as a lack of response to its concerns from the school board, is launching its own action plan to improve the quality of education.…Read more…

Why are Ohio's charter schools so poor?

In a previous post we took a look at the difference in performance between Ohio's traditional public schools and their charter school counterparts, and discovered a wide and growing gap. Why does this gap exist?

One of the clear reasons is in the quality of the workforce. Ohio's traditional public schools have invested a lot of time and money developing an experienced, highly qualified teacher workforce, an investment that Ohio's charter schools have resisted or failed to developing.

Steve Dyer at Innovation Ohio took a look at the latest teacher data made available by ODE and discovered that the typical (i.e. the mode) charter school teacher has 0 years of experience, while their traditional school counterpart had 14 years of experience. We took a look at this too, and came to the same conclusion.

Our analysis also showed that the average level of experience of a charter school teacher is only 4.9 years, while in traditional schools that figure is over 14 years.

Dyer also discovered, what he called "two equally stunning statistics"

1) The average Traditional Public School building has about 2/3 of its teachers with a masters degree. The average Charter School building has about 1/3 of its teachers with a masters degree.
[...]
2) About 1 out of 3 Charter Schools in Ohio have at least some core courses taught by someone with a temporary teaching certificate. Of the more than 3,200 Traditional Public School buildings in the report card data, not a single one has core courses taught by teachers with temporary certificates.

Furthermore, over 10% of Ohio's charter schools have class sizes greater than 25 students per teacher, 20 of them more than 30 students per teacher!

The major innovation attempted by Ohio's charter school community has not been to find ways to deliver higher quality education, but instead to find ways to minimize teacher costs in order to maximize profits. This is made very clear when once looks at this experience and qualification differences, education outcome quality differences and the subsequent difference in average salaries where the charter school average is $33,993 and traditional schools $57,303.