Value-added: How Ohio is destroying a profession

We ended the week last week with a post titled "The 'fun' begins soon", which took a look at the imminent changes to education policy in Ohio. We planned on detailing each of these issues over the next few weeks.

Little did we know that the 'fun' would begin that weekend. It came in the manner of the Cleveland Plain Dealer and NPR publishing a story on the changing landscape of teacher evaluations titled "Grading the Teachers: How Ohio is Measuring Teacher Quality by the Numbers".

It's a solid, long piece, worth the time taken to read it. It covers some, though not all, of the problems of using value-added measurements to evaluate teachers

Those ratings are still something of an experiment. Only reading and math teachers in grades four to eight get value-added ratings now. But the state is exploring how to expand value-added to other grades and subjects.

Among some teachers, there’s confusion about how these measures are calculated and what they mean.

“We just know they have to do better than they did last year,” Beachwood fourth-grade teacher Alesha Trudell said.

Some of the confusion may be due to a lack of transparency around the value-added model.

The details of how the scores are calculated aren’t public. The Ohio Education Department will pay a North Carolina-based company, SAS Institute Inc., $2.3 million this year to do value-added calculations for teachers and schools. The company has released some information on its value-added model but declined to release key details about how Ohio teachers’ value-added scores are calculated.

The Education Department doesn’t have a copy of the full model and data rules either.

The department’s top research official, Matt Cohen, acknowledged that he can’t explain the details of exactly how Ohio’s value-added model works. He said that’s not a problem.

Evaluating a teacher on a secret formula isn't a practice that can be sustained, supported or defended. The article further details a common theme we hear over and over again

But many teachers believe Ohio’s value-added model is essentially unfair. They say it doesn’t account for forces that are out of their control. They also echo a common complaint about standardized tests: that too much is riding on these exams.

“It’s hard for me to think that my evaluation and possibly some day my pay could be in a 13-year-old’s hands who might be falling asleep during the test or might have other things on their mind,” said Zielke, the Columbus middle school teacher.

The article also performs analysis on several thousands value add scores, and that analysis demonstrates what we have long reported, that value-add is a poor indicator of teacher quality, with too many external factors affecting the score

A StateImpact/Plain Dealer analysis of initial state data suggests that teachers with high value-added ratings are more likely to work in schools with fewer poor students: A top-rated teacher is almost twice as likely to work at a school where most students are not from low-income families as in a school where most students are from low-income families.
[…]
Teachers say they’ve seen their value-added scores drop when they’ve had larger classes. Or classes with more students who have special needs. Or more students who are struggling to read.

Teachers who switch from one grade to another are more likely to see their value-added ratings change than teachers who teach the same grade year after year, the StateImpact/Plain Dealer analysis shows. But their ratings went down at about the same rate as teachers who taught the same grade level from one year to the next and saw their ratings change.

What are we measuring here? Surely not teacher quality, but rather socioeconomic factors and budget conditions of the schools and their students.

Teachers are intelligent people, and they are going to adapt to this knowledge in lots of unfortunate ways. It will become progressively harder to districts with poor students to recruit and retain the best teachers. But perhaps the most pernicious effect is captured at the end of the article

Stephon says the idea of Plecnik being an ineffective teacher is “outrageous.”

But Plecnik is through. She’s quitting her job at the end of this school year to go back to school and train to be a counselor — in the community, not in schools.

Plecnik was already frustrated by the focus on testing, mandatory meetings and piles of paperwork. She developed medical problems from the stress of her job, she said. But receiving the news that despite her hard work and the praise of her students and peers the state thought she was Least Effective pushed her out the door.

“That’s when I said I can’t do it anymore,” she said. “For my own sanity, I had to leave.”

The Cleveland Plain Dealer and NPR then decided to add to this stress by publishing individual teachers value-added scores - a matter we will address in our next post.

More Sen. Peggy Lehner Please

State Sen. Peggy Lehner (R-Kettering) is proving herself to be an unusual Republic legislator. One who has a keen understanding of education issues, and a willingness to listen and work with educators, not just tow the ideological line.

The first piece of evidence being her attempt to fix the problems with the 3rd grade reading guarantee law, via SB21 which she sponsored and shepherded through the Senate on a 30-1 vote, and then passage through the House (albeit with some questionable changes having been made).

Now comes news of her attempt to bring Ohio's preschool efforts back from the dead

A Senate Republican leader on education policy wants to create a $100 million voucher program over the next two years to allow thousands of low-income Ohio children to attend preschool.

For every dollar Ohio spends on early childhood education, the return is $10 or more, said Sen. Peggy Lehner, R-Kettering. The need to have students enter kindergarten prepared to learn is more vital than ever, she and others argued, especially as the state implements a new requirement that students pass a reading exam in third grade or risk being held back.

“So many of our children come to kindergarten two or three years behind their peers, and we’re trying to catch them up before third grade,” Lehner said. “If we don’t catch them up, they don’t have a prayer of passing that third-grade reading guarantee.”

This would be a welcome policy change of direction after the Governor's shameful evisceration of early childhood education in his previous budget, and unwillingness to restore those cuts in the current proposal

A decade ago, more than twice as many Ohio children were enrolled in the state’s preschool program than now.

According to a recent report by the National Institute for Early Education Research, in 2011-2012 total state enrollment for preschool was 9,379. The state only paid for 5,700 of those students; the rest were paid for by parents, local dollars or federal funds.

Compare that to the 2001-2002 school year when 23,599 Ohio children were enrolled in the state’s preschool program.

Although the situation isn’t unique to Ohio, the state did see the most drastic drop in early childhood education enrollment in the nation over the last decade.

According to NIEER, Ohio’s decline in the number of preschoolers in state funded programs is the result of state budget cuts over the last few years.

Kudos to Sen. Peggy Lehner, and here's hoping more of her colleagues follow her lead of listening to educators concerns.

We note that Steve Dyer at 10th Period has some concerns about this pre-school proposal.

3rd grade reading guarantee changes again

The Ohio House finally moved SB21 out of committee. SB21 is the bill that tries to fix many of the problems raised by the initial third grade reading guarantee legislation. The Committee made a number of changes, but according to the bill's Sponsor, Sen. Peggy Lehner, she expects the Senate will agree with those changes.

According to a Gongwer report, the amendments made by the House include:

The omnibus amendment would add a requirement that all teachers providing reading guarantee services have at least one year of teaching experience unless they meet at least one of the bill's criteria to provide services and is mentored by a teacher with at least one year of experience, according to a Legislative Service Commission comparison document.

It also specifies teachers who qualify to provide services by virtue of a reading endorsement on their license must also have passed the State Board of Education-required assessment for the endorsement only "as applicable."

The latest version eliminates from the list of acceptable reading guarantee qualifications teachers determined by ODE as an "effective reading instructor" and teachers who completed a program from a list of scientifically researched-based reading instruction options.

The amended bill instead adds to the list of acceptable qualifications to include teachers:

• Rated "most effective" for reading instruction for the last two years based on assessments of student growth measures developed by a vendor approved by the state board.
• Rated "above expected value-added" for reading for the two most recent years per criteria established by ODE.
• Holding an educator license for teaching grades PreK-3 or 4-9 issued on or after July 1, 2017. The omnibus amendment requires all new applicants for educator licenses for those grades pass an exam aligned with reading competencies established by the state board.

The state board is required to adopt those competencies Jan. 31, 2014 under the bill, and must cover all reading credentials and training that include an understanding of phonemic awareness, phonics, appropriate use of assessment, appropriate instruction materials, among others.

Starting July 2014, alternative credential and training that qualify a teacher to instruct students identified by the reading guarantee would be aligned with the reading competencies, according to LSC.

Those teachers who do not meet the listed qualifications nor have one year teaching experience would be permitted to provide reading guarantee services if he or she holds an alternative credential or has successfully completed training using research-based reading instruction approved by ODE.

The omnibus amendment also puts in place ramifications for schools that fail to perform on reading aspects of the state report card. Schools would be required to submit improvement plans to ODE if they receive a D or F on the K-3 literacy progress measure and less than 60% of their students who took the third-grade English language arts assessment attained at least a proficient score, according to LSC. Submission of improvement plans would start in 2016.

A school could cease submitting an improvement plan if it receives a grade of C or better on the K-3 literacy measure or at least 60% of students taking the third-grade ELA exam scored proficient or better.

Other changes made in the omnibus amendment include:

• An exemption from the reading guarantee for those limited English proficient students who have been enrolled in U.S. schools for fewer than three years and who have had less than three years instruction in an English as a second language program. Current law exempts those with fewer than two years.
• Allowing schools unable to meet personnel requirements to request a staffing plan beyond the 2013-14 school year. Those submitting plans must also report on progress the school has made in meeting requirements of the law.
• Requiring ODE to study diagnostic assessments for reading and writing in grades K-3 that might be considered for approval by the state board.

The LSC analysis if the changes can be seen in the following document, with the House changes contained in the right hand column.

SB21 As Reported by the House Education Committee

SB21 - Brings changes to 3rd grade reading law

Earlier this week, the Ohio Senate passed SB21 (30-1), a bill that would alter requirements of the 3rd grade reading guarantee. The changes were a positive step, and will make it easier for schools and educators to meet the standard, that previously were nearly impossible to meet.

According to a Gongwer report

The bill would eliminate language that required teachers to "be actively engaged in the reading instruction of students for the previous three years," which was seen as a roadblock to hiring new teachers or other qualified educators and was considered a very difficult or nearly impossible standard to meet.

"Given the importance of the third-grade reading guarantee to the future of our children, we listened very carefully" to the suggestions of principals and superintendents, sponsoring Sen. Peggy Lehner (R-Kettering) said.

The bill also includes

• Closing the loophole whereby a child could avoid being held back by skipping the test.
• Exempts students who have significant cognitive disabilities
• Removes "credential" and replaces it with "completion of a program" which would cover programs, such as Orton-Gillingham, that do not produce a credential upon completion.
• Replaces a value-added score requirement when the teacher is an effective reading instructor, as determined by criteria established by ODE.
• Allows schools the authority to get a waiver by resolution for their action plan required when the district is unable to hire sufficient teachers with the approved credentials.

The lone no vote was Sen. Joe Schiavoni who said he voted against the bill because, although he supports the policy, the lack of funding is a problem.

"We need to put the $130 million, the $100 million-dollar tag on this to help schools pay for this," he said.

Sen. Gardner, who will chair the subcommittee that will hear the K-12 portion of the budget bill, said he expects to see bicameral, bipartisan support to provide more funding to support the goals of the TGRG in that legislation.

Let's hope so. It's not often we get education bills moving in the right direction. This bill still needs to pass the House.

You can read the full text of the bill, here. For those who would prefer a more plain english explanation, here's LSC's analysis.

SB21 - 3rd Grade Reading Guarantee Changes by

Budget brings 2 dead policies back to life

The Governor's 4,200 page budget bill (HB 59) sees the reanimation of 2 education policy ideas that were overwhelmingly rejected in the previous legislature due to their unpopular and deeply destructive nature.

The first provision sees the Governor once again push the corporate reform idea of a statewide parent trigger. Here's the change in law he is proposing

Sec. 3302.042. (A) This section shall operate as a pilot project that applies apply to any school of a city, exempted village, or local school district that has been ranked according to performance index score under section 3302.21 of the Revised Code in the lowest five per cent of all public school buildings statewide for three or more consecutive school years and is operated by the Columbus city school district. The pilot project shall commence once the department of education establishes implementation guidelines for the pilot project in consultation with the Columbus city school district.

(B) Except as provided in division (D), (E), or (F) of this section, if the parents or guardians of at least fifty per cent of the students enrolled in a school to which this section applies, or if the parents or guardians of at least fifty per cent of the total number of students enrolled in that school and the schools of lower grade levels whose students typically matriculate into that school, by the thirty-first day of December of any school year in which the school is subject to this section, sign and file with the school district treasurer a petition requesting the district board of education to implement one of the following reforms in the school, and if the validity and sufficiency of the petition is certified in accordance with division (C) of this section, the board shall implement the requested reform in the next school year:

Over objections to this idea in the previous budget, the policy was scaled back to be a pilot program solely affecting Columbus City Schools. Since this "pilot" began, and despite many of the real and perceived problems with Columbus City Schools, not a single attempt has been made to pull "the parent trigger". Despite the failure of this pilot program, and without any working evidence that such a policy mechanism could succeed, the Governor wants to once again spread this community busting idea throughout the entire state.

Here's what people thought of the idea last time around

For further discussion on the failures of parent trigger laws, our previous posting can be found here.

The second zombie policy idea to be resurrected by the Governor was even more solidly rejected when it was introduced as HB136. HB136 sought to eliminate the restrictions on Ohio's current voucher programs (ʺEd Choiceʺ and "Cleveland Scholarship") and instead open participation statewide on the basis of family income. The idea was so bad that even the author of the bill called it a "potential doomsday" for public education. The bill prompted more than 400 boards of education to pass resolutions opposing the idea and the bill died before receiving a floor vote.

Now it's back, under Sec. 3310.032

Sec. 3310.032. (A) A student is an "eligible student" for purposes of the expansion of the educational choice scholarship pilot program under this section if the student's resident district is not a school district in which the pilot project scholarship program is operating under sections 3313.974 to 3313.979 of the Revised Code and the student's family income is at or below two hundred per cent of the federal poverty guidelines, as defined in section 5101.46 of the Revised Code.

(B) In each fiscal year for which the general assembly appropriates funds for purposes of this section, the department of education shall pay scholarships to attend chartered nonpublic schools in accordance with section 3310.08 of the Revised Code. The number of scholarships awarded under this section shall not exceed the number that can be funded with appropriations made by the general assembly for this purpose.

(C) Scholarships under this section shall be awarded as follows:
(1) For the 2013-2014 school year, to eligible students who are entering kindergarten in that school year for the first time;
(2) For each subsequent school year, scholarships shall be awarded to eligible students in the next grade level above the highest grade level awarded in the preceding school year, in addition to the grade levels for which students received scholarships in the preceding school year.

(D) If the number of eligible students who apply for a scholarship under this section exceeds the scholarships available based on the appropriation for this section, the department shall award scholarships in the following order of priority:
(1) First, to eligible students who received scholarships under this section in the prior school year;
(2) Second, to eligible students with family incomes at or below one hundred per cent of the federal poverty guidelines. If the number of students described in division (D)(2) of this section who apply for a scholarship exceeds the number of available scholarships after awards are made under division (D)(1) of this section, the department shall select students described in division
(D)(2) of this section by lot to receive any remaining scholarships.
(3) Third, to other eligible students who qualify under this section. If the number of students described in division (D)(3) of this section exceeds the number of available scholarships after awards are made under divisions (D)(1) and (2) of this section, the department shall select students described in division (D)(3) of this section by lot to receive any remaining scholarships.

(E) A student who receives a scholarship under this section remains an eligible student and may continue to receive scholarships under this section in subsequent school years until the student completes grade twelve, so long as the student satisfies the conditions specified in divisions (E)(2) and (3) of section 3310.03 of the Revised Code.

Once a scholarship is awarded under this section, the student shall remain eligible for that scholarship for the current school year and subsequent school years even if the student's family income rises above the amount specified in division (A) of this section, provided the student remains enrolled in a chartered nonpublic school.

Eligibility for private school vouchers, in a few short paragraphs is opened up statewide, even if students in a school district have schools rated excellent to attend. When traditional public schools are suffering such draconian budget cuts, siphoning tax payer money to private schools cannot be a reasonable policy. This is, in short, a public education privatization provision.

Correlation? What correlation?

Dublin teacher, Kevin Griffin, brings to our attention this graph, which he describes thusly

The chart plots the Value-Added scores of teachers who teach the same subject to two different grade levels in the same school year. (ex. Ms. Smith teaches 7th Math and 8th Math, and Mr. Richards 4th Grade Reading and 5th Grade Reading.) The X-axis represents the teachers VA score for one grade level and the Y-axis represents the VA score from the other grade level taught.

If the theory behind evaluating teachers based on value-added is valid then a “great” 7th grade math teacher should also be a “great” 8th grade math teacher (upper right corner) and a “bad” 7th grade math teacher should also be a “bad” 8th grade math teacher (lower left corner). There should, in theory, be a straight line (or at least close) showing a direct correlation between 7th grade VA scores and 8th grade VA scores since those students, despite being a grade apart, have the same teacher.

Here's the graph

Looks morel ike a random number generator to us. Would you like your career to hinge on a random number generator?