Value-added: How Ohio is destroying a profession

We ended the week last week with a post titled "The 'fun' begins soon", which took a look at the imminent changes to education policy in Ohio. We planned on detailing each of these issues over the next few weeks.

Little did we know that the 'fun' would begin that weekend. It came in the manner of the Cleveland Plain Dealer and NPR publishing a story on the changing landscape of teacher evaluations titled "Grading the Teachers: How Ohio is Measuring Teacher Quality by the Numbers".

It's a solid, long piece, worth the time taken to read it. It covers some, though not all, of the problems of using value-added measurements to evaluate teachers

Those ratings are still something of an experiment. Only reading and math teachers in grades four to eight get value-added ratings now. But the state is exploring how to expand value-added to other grades and subjects.

Among some teachers, there’s confusion about how these measures are calculated and what they mean.

“We just know they have to do better than they did last year,” Beachwood fourth-grade teacher Alesha Trudell said.

Some of the confusion may be due to a lack of transparency around the value-added model.

The details of how the scores are calculated aren’t public. The Ohio Education Department will pay a North Carolina-based company, SAS Institute Inc., $2.3 million this year to do value-added calculations for teachers and schools. The company has released some information on its value-added model but declined to release key details about how Ohio teachers’ value-added scores are calculated.

The Education Department doesn’t have a copy of the full model and data rules either.

The department’s top research official, Matt Cohen, acknowledged that he can’t explain the details of exactly how Ohio’s value-added model works. He said that’s not a problem.

Evaluating a teacher on a secret formula isn't a practice that can be sustained, supported or defended. The article further details a common theme we hear over and over again

But many teachers believe Ohio’s value-added model is essentially unfair. They say it doesn’t account for forces that are out of their control. They also echo a common complaint about standardized tests: that too much is riding on these exams.

“It’s hard for me to think that my evaluation and possibly some day my pay could be in a 13-year-old’s hands who might be falling asleep during the test or might have other things on their mind,” said Zielke, the Columbus middle school teacher.

The article also performs analysis on several thousands value add scores, and that analysis demonstrates what we have long reported, that value-add is a poor indicator of teacher quality, with too many external factors affecting the score

A StateImpact/Plain Dealer analysis of initial state data suggests that teachers with high value-added ratings are more likely to work in schools with fewer poor students: A top-rated teacher is almost twice as likely to work at a school where most students are not from low-income families as in a school where most students are from low-income families.
[…]
Teachers say they’ve seen their value-added scores drop when they’ve had larger classes. Or classes with more students who have special needs. Or more students who are struggling to read.

Teachers who switch from one grade to another are more likely to see their value-added ratings change than teachers who teach the same grade year after year, the StateImpact/Plain Dealer analysis shows. But their ratings went down at about the same rate as teachers who taught the same grade level from one year to the next and saw their ratings change.

What are we measuring here? Surely not teacher quality, but rather socioeconomic factors and budget conditions of the schools and their students.

Teachers are intelligent people, and they are going to adapt to this knowledge in lots of unfortunate ways. It will become progressively harder to districts with poor students to recruit and retain the best teachers. But perhaps the most pernicious effect is captured at the end of the article

Stephon says the idea of Plecnik being an ineffective teacher is “outrageous.”

But Plecnik is through. She’s quitting her job at the end of this school year to go back to school and train to be a counselor — in the community, not in schools.

Plecnik was already frustrated by the focus on testing, mandatory meetings and piles of paperwork. She developed medical problems from the stress of her job, she said. But receiving the news that despite her hard work and the praise of her students and peers the state thought she was Least Effective pushed her out the door.

“That’s when I said I can’t do it anymore,” she said. “For my own sanity, I had to leave.”

The Cleveland Plain Dealer and NPR then decided to add to this stress by publishing individual teachers value-added scores - a matter we will address in our next post.

Education News for 01-22-2013

State Education News

• Ohio now restricts school’s use of seclusion rooms, physical restraint (Athens Messenger)

For the first time, Ohio has a policy that limits a school’s use of seclusion and restraint for difficult students. Schools must now adopt positive behavior interventions and support…Read more…

• School rules guide whistle-blowers (Columbus Dispatch)

Amid a statewide investigation into data manipulation in schools, districts are creating rules to guide employees if they want to report workers who violate laws or ethics.…Read more…

• Schools await Kasich’s funding model (Lima News)

They want more money and a school-funding system that is fair. But area school officials also just hope for a little honesty.…Read more…

• Turning the page (Mason HS Chronicle)

More third graders than ever before could be held back next year. Due to recent legislation that alters current reading level standards for the 2013-14 school year.…Read more…

• Ranking brings school funding model under scrutiny (Middletown Journal)

Ohio recently was ranked 17th in the nation for its school finance, despite the fact that Ohio’s school funding model has been declared unconstitutional three times since 1997.…Read more…

Local Education News

• Could shared services save Ohio districts $1B a year? (Cincinnati Enquirer)

State leaders are urging school districts, like other local government agencies, to share services and costs, operate more efficiently and reserve more tax dollars for core purposes.…Read more…

• Board, referee clash over teacher firing (Findlay Courier)

A Liberty-Benton veteran teacher, fired this week by the school board, should not have been terminated because the board failed to provide documented evidence for its claim that he had a history of classroom management problems…Read more…

• School patrols increase (Lisbon Morning Journal)

Ever since last month's school shooting in Newtown, Conn., sheriff's deputies have been performing security checks during the school day at five school districts.…Read more…

• Safer schools start with information (New Philadelphia Times)

School districts in Tuscarawas, Carroll, Harrison and Belmont counties are being offered free technology that would assist first responders dealing with emergencies at area schools.…Read more…

• Kiwanis takes stand against bullying (Portsmouth Daily Times)

The Kiwanis Club of Portsmouth has taken a stand against bullying and did so with a recent presentation by club president John Johnson.…Read more…

• Deputies could be added to help with Clark schools (Springfield News-Sun)

New Clark County sheriff’s deputies could be hired as part of a program to boost security in schools, a local response to the Newtown, Conn., elementary school shootings.…Read more…

• Boosters, PTO slate reception for new superintendent (This Week News)

The Northridge Academic Boosters and Northridge PTO invite the community to a "Welcome to Northridge" for new Superintendent Dr. Chris Briggs.…Read more…

• Chardon Schools' installation of security cameras now almost complete (Willoughby News Herald)

Chardon School District’s goal of installing technologically advanced security cameras in and around all of its buildings is nearing completion.…Read more…

• Liberty schools expecting another audit (Youngstown Vindicator)

Liberty schools are awaiting another report from Ohio Auditor David Yost, after a 2011 financial audit was released earlier this week, schools Superintendent Stan Watson said.…Read more…

Editorial

• Alert to danger (Akron Beacon Journal)

Last week, at the first of five regional training sessions, educators and law enforcement officers received graphic and sometimes emotional lessons about how to respond to the kind of shooting incidents that have gripped the nation’s attention.…Read more…

• Alone in school (Akron Beacon Journal)

Think of isolation rooms and physical restraints, and the mind goes not to schools but to prison cells for violent criminals or to efforts to prevent mentally unstable patients from hurting themselves or others.…Read more…

• Right direction (Columbus Dispatch)

With a new State Board of Education policy limiting the use of “seclusion rooms” for students whose behavior is out of control, Ohio schools and students will be better off than they were before.…Read more…

• Good start (Findlay Courier)

Limiting access to Findlay's elementary buildings won't stop all unwelcome visitors. But it should help. …Read more…

It all comes down to purpose!

A Guest Post from Robert Barkley

So much of the current attacks on public education have been framed inside a concept called the “business model.” As it turns out, many uniformed elected officials, and even many education-bashing business leaders themselves, apparently don’t understand at all the fundamentals of effective businesses.

The centerpiece of effective organizational practice, whether in the private or public sector, is clarity as to purpose. And it’s precisely there that those many critics don’t get it. Ask them what the purpose of education is, and you’ll likely get answers such as, “master the basics…prepare students for work…raise test scores…improve graduation rates…encourage life long learning…get more into college,” and the list goes on.

These are all commendable but they are the results and not the purpose. A well-conceived purpose will achieve all such objectives and more.

So let’s turn to defining the purpose of education. I devoted a full chapter to that topic in a book I self-published about 10 years ago. Following is the primary discussion pulled from that book:

Educators and public policy leaders do not always agree on purpose. Here are some different visions of purpose that illustrate a wide-ranging view and are pulled from some top theorists and resources.

W. Edwards Deming: “The purpose of education is to preserve and nurture joy in learning.” Schools must “increase the positives and decrease the negatives so that all students keep their yearning for learning.” The mission of schools is to maintain enthusiasm while increasing learning.

Based upon fundamental Hellenic philosophy: The purpose of education is to develop students—who are comfortable in meeting their survival needs, who have an increasing capacity and desire for rational thought, who can conduct themselves productively and virtuously and can distinguish what matters most—both in regard to their own interests and those of their community, and who can constructively contribute to the most effective governance of the society in which they find themselves.

Myron Tribus building upon Deming, advocated “creating joy in learning” as the chief aim of education. He then states the criteria for judging educational programs. He says, “A good educational program will emphasize: Knowledge – which enables the learner to understand how what is learned connects to what is already known and permits the learner to analyze new situations; Know-how -- which enables the learner to actually do something with the knowledge thus gained; Wisdom -- which enables the learner to decide when, where or whether to actually use know-how in a particular situation;

Character -- which makes the learner capable of being trusted with knowledge, know-how and wisdom.” Tribus adds, “When I look at a program I look for evidence that the teachers are aware of these four aspects of education and can demonstrate the efforts they are making in all four dimensions of good education.”

Marion Brady: “Each of us has acquired from our society a comprehensive model of reality. The most important task of general education is to help us understand that model, the models of those with whom we interact, and the range of alternative models from which we might choose.”

Paul Woodring: “The goal of a liberal education is to free individuals from the limitations of ignorance, prejudice, and provincialism; to enable them to see the world clearly and in perspective; to develop their intellectual capabilities, increase their sensitivity, and prepare them to make wise, independent judgments.”

Maurice Holt: suggests that we currently have competing needs which he describes as: “To deliver the knowledge and skills that business needs,” versus, “To equip students with the capacity to address the unpredictable problems of adulthood and to establish themselves in a world of growing complexity.”

It is clear that establishing educational purpose is not simply an academic or organizational and managerial process. It is a public policy issue given the level of societal interest, the political nature of education, and the level of public investment. My own espoused purpose for education—obviously taken from Deming: “Engendering increasingly enthusiast learners who continuously seek and achieve the skills necessary to advance their learning, satisfy their natural curiosities, and become contributing citizens.”

Step two in organizational effectiveness is to establish how progress toward the adopted purpose will be measured. And here is why I have brought this topic to the fore. Think of what the policy makers of both major political parties and well-meaning many critics of educational have chosen as their measurement tools. Think standardized tests! Once you reflect upon that you will quickly realize why we are headed in absolutely the wrong direction and why the international leaders in education have abandoned exactly what those in the US are advocating.

Robert Barkley, Jr., is retired Executive Director of the Ohio Education Association, a thirty-five year veteran of NEA and NEA affiliate staff work. He is the author of Quality in Education: A Primer for Collaborative Visionary Educational Leaders; Leadership In Education: A Handbook for School Superintendents and Teacher Union Presidents; and Lessons for a New Reality: Guidance for Superintendent/Teacher Organization Collaboration. He may be reached at rbarkle@columbus.rr.com.

Politics and Education Don't Mix

Governors and presidents are no better suited to run schools than they are to run construction sites, and it's time our education system reflected that fact.

A central flaw of corporate paradigms, as is often noted in popular culture, is the mind-numbing and dehumanizing effect of bureaucracy. Sometimes we are horrified and sometimes we laugh, but arguments for or against the free market may be misguided if we fail to address bureaucracy's corrosive role in the business model.

Current claims about private, public, or charter schools in the education reform movement, which has its roots in the mid-nineteenth century, may also be masking a much more important call to confront and even dismantle the bureaucracy that currently cripples universal public education in the U.S. "Successful teaching and good school cultures don't have a formula," argued legal reformer Philip K. Howard earlier in this series, "but they have a necessary condition: teachers and principals must feel free to act on their best instincts....This is why we must bulldoze school bureaucracy."

Bureaucracy, however, remains an abstraction and serves as little more than a convenient and popular target for ridicule -- unless we unpack what actions within bureaucracy are the sources for many of the persistent failures we associate erroneously with public education as an institution. Bureaucracy fails, in part, because it honors leadership as a primary quality over expertise, commits to ideological solutions without identifying and clarifying problems first, and repeats the same reforms over and over while expecting different results: our standards/testing model is more than a century old.

Public education is by necessity an extension of our political system, resulting in schools being reduced to vehicles for implementing political mandates. For example, during the past thirty years, education has become federalized through dynamics both indirect ("A Nation at Risk" spurring state-based accountability systems) and direct (No Child Left Behind and Race to the Top).

As government policy and practice, bureaucracy is unavoidable, of course. But the central flaw in the need for structure and hierarchy is that politics prefers leadership characteristics above expertise. No politician can possibly have the expertise and experience needed in all the many areas a leader must address (notably in roles such as governor and president). But during the "accountability era" in education of the past three decades, the direct role of governors and presidents as related to education has increased dramatically--often with education as a central plank in their campaigns.

One distinct flaw in that development has been a trickle-down effect reaching from presidents and governors to state superintendents of education and school board chairs and members: people who have no or very little experience or expertise as educators or scholars attain leadership positions responsible for forming and implementing education policy.

The faces and voices currently leading the education reform movement in the U.S. are appointees and self-proclaimed reformers who, while often well-meaning, lack significant expertise or experience in education: Secretary of Education Arne Duncan, billionaire Bill Gates, Michelle Rhee (whose entrance to education includes the alternative route of Teach for America and only a few years in the classroom), and Sal Khan, for example.

[readon2 url="http://www.theatlantic.com/national/archive/2012/04/politics-and-education-dont-mix/256303/"]Continue reading[/readon2]

The Toxic Trifecta in Current Legislative Models for Teacher Evaluation

A relatively consistent legislative framework for teacher evaluation has evolved across states in the past few years. Many of the legal concerns that arise do so because of inflexible, arbitrary and often ill-conceived yet standard components of this legislative template. There exist three basic features of the standard model, each of which is problematic on its own regard, and those problems become multiplied when used in combination.

First, the standard evaluation model proposed in legislation requires that objective measures of student achievement growth necessarily be considered in a weighting system of parallel components. Student achievement growth measures are assigned, for example, a 40 or 50% weight alongside observation and other evaluation measures. Placing the measures alongside one another in a weighting scheme assumes all measures in the scheme to be of equal validity and reliability but of varied importance (utility) – varied weight. Each measure must be included, and must be assigned the prescribed weight – with no opportunity to question the validity of any measure. [1]Such a system also assumes that the various measures included in the system are each scaled such that they can vary to similar degrees. That is, that the observational evaluations will be scaled to produce similar variation to the student growth measures, and that the variance in both measures is equally valid – not compromised by random error or bias. In fact, however, it remains highly likely that some components of the teacher evaluation model will vary far more than others if by no other reasons than that some measures contain more random noise than others or that some of the variation is attributable to factors beyond the teachers’ control. Regardless of the assigned weights and regardless of the cause of the variation (true or false measure) the measure that varies more will carry more weight in the final classification of the teacher as effective or not. In a system that places differential weight, but assumes equal validity across measures, even if the student achievement growth component is only a minority share of the weight, it may easily become the primary tipping point in most high stakes personnel decisions.

Second, the standard evaluation model proposed in legislation requires that teachers be placed into effectiveness categories by assigning arbitrary numerical cutoffs to the aggregated weighted evaluation components. That is, a teacher in the 25%ile or lower when combining all evaluation components might be assigned a rating of “ineffective,” whereas the teacher at the 26%ile might be labeled effective. Further, the teacher’s placement into these groupings may largely if not entirely hinge on their rating in the student achievement growth component of their evaluation. Teachers on either side of the arbitrary cutoff are undoubtedly statistically no different from one another. In many cases as with the recently released teacher effectiveness estimates on New York City teachers, the error ranges for the teacher percentile ranks have been on the order of 35%ile points (on average, up to 50% with one year of data). Assuming that there is any real difference between the teacher at the 25%ile and 26%ile (as their point estimate) is a huge unwarranted stretch. Placing an arbitrary, rigid, cut-off score into such noisy measures makes distinctions that simply cannot be justified especially when making high stakes employment decisions.

Third, the standard evaluation model proposed in legislation places exact timelines on the conditions for removal of tenure. Typical legislation dictates that teacher tenure either can or must be revoked and the teacher dismissed after 2 consecutive years of being rated ineffective (where tenure can only be achieved after 3 consecutive years of being rate effective).[2]As such, whether a teacher rightly or wrongly falls just below or just above the arbitrary cut-offs that define performance categories may have relatively inflexible consequences.

The Forced Choice between “Bad” Measures and “Wrong” Ones

[readon2 url="http://nepc.colorado.edu/blog/toxic-trifecta-bad-measurement-evolving-teacher-evaluation-policies"]Continue reading...[/readon2]

Packed Virtual Classrooms

Later today, Apple will unveil its plans for digital textbooks.

Steve Jobs described textbooks as an '$8 billion a year industry ripe for digital destruction', in conversations with his biographer Walter Isaacson.

Given how much dead tree weight students have to carry around, and how expensive textbooks have become, this is an area ripe for a solution. But as Apple lays out its plans for capturing some of the profits to be had from education, likely with an innovative technology based solution, corporate education reformers have set their sights on using technology to capture profits in an altogether different way.

The Fordham Foundation recently released a paper titled " Creating Sound Policy for Digital Learning, A Working Paper Series from the Thomas B. Fordham Institute. The piece begins

Online learning, in its many shapes and sizes, is quickly becoming a typical part of the classroom experience for many of our nation’s K-12 students. As it grows, educators and policymakers across the country are beginning to ask the question: What does online learning cost? While the answer to this question is a key starting point, by itself it has limited value. Of course there are cheaper ways to teach students. The key question that will eventually have to be addressed is: Can online learning be better and less expensive

At that point the paper descends into the usual rote corporate ed stuff, using anecdote to try to capture the costs and quality of virtual education. The total lack of innovative thought is captured in their first graph.

You will clearly note that it is not technology driving the savings, but instead the slashing of spending on educators. The entire difference between a traditional model and virtual model is in the category of faculty and admin expenditures. Stephen Dyer, at his new blog "10th Period", points out that actual e-school spending in Ohio follows this exact model

Over at Innovation Ohio, I helped write and research a report that pointed out Ohio pays these major eSchool operators enough money for them to provide 15:1 student:teacher ratios, $2,000 laptops and still clear about 30% profit.

However, they don't do that. On average, they have 37:1 student:teacher ratios. Ohio Virtual Academy (run by the infamous, national for-profit K-12, Inc.) has a student:teacher ratio of 51:1, if you can believe it. Anyway, of the $183 million Ohio's taxpayers sent to these eSchools last school year, the schools spent a grand total of $27.5 million on teacher salaries, or about 15% of its money.

E-schooling as envisaged by corporate education reformers doesn't rely upon any technological innovation as a means to deliver high quality education, they use the virtual nature of the model to obfuscate the fact that class sizes can become huge. It's hard for a parent to know their child is crammed in to a packed class with 50 other students if he is sat alone in his bedroom. What you don't see, won't hurt, right?

It's never explained how a teacher can deliver quality to such large classes, in a situation where the virtual nature of the classes already make it naturally more difficult and challenged.

We know from facts certain that Ohio's e-schools are appallingly bad. Even the Fordham Foundation itself found e-school to be terrible

Perhaps before we even begin to consider cost, we ought to sort out the very serious problems we have with quality. What does it matter how cheap something is, if it is not fit for purpose? One might even argue, with tongue not so firmly planted in the cheek that Ohio's e-schools are breaking consumer laws

In common law jurisdictions, an implied warranty is a contract law term for certain assurances that are presumed to be made in the sale of products or real property, due to the circumstances of the sale. These assurances are characterized as warranties irrespective of whether the seller has expressly promised them orally or in writing. They include an implied warranty of fitness for a particular purpose, an implied warranty of merchantability for products, implied warranty of workmanlike quality for services, and an implied warranty of habitability for a home.