High stakes failure

It might be becoming apparent to any rational observer that high stakes corporate education policies are failing catastrophically. Where once various data and tests were used to inform educators and provide diagnostic feedback, they are increasingly being used to rank, grade, and even punish.

This is leading to the inevitable behaviors that are always present when such systems are created - whether it was in the world of energy companies such as Enron, or other accounting scandals including those affecting Tyco International, Adelphia, Peregrine Systems and WorldCom, to the more recent scandals involving Lehman Brothers, JPM or Barclays bank.

Here's another example, in news from Pennsylvania

After authorities imposed unprecedented security measures on the 2012 statewide exams, test scores tumbled across Pennsylvania, The Inquirer has learned.

At some schools, Pennsylvania Secretary of Education Ronald Tomalis said, the drops are "noticeable" - 25 percent or more.

In some school systems, investigators have found evidence of outright doctoring of previous years' tests - and systemic fraud that took place across multiple grades and subjects.

In Philadelphia and elsewhere, some educators have already confessed to cheating, and investigators have found violations ranging from "overcoaching" to pausing a test to reteach material covered in the exam, according to people familiar with the investigations.

When trillions of dollars of the world's money is at stake, investing in tight oversight and regulation is imperative, but when it comes to evaluating the progress of a 3rd grader, do we really want to spend valuable education dollars measuring the measurers?

The question becomes even more pertinent when one considers that the the efficacy of many of the measures is questionable at best. Article after article, study after study, places significant questions at the feet of value add proponents, and now a new study even places questions at the feet of the tests themselves

Now, in studies that threaten to shake the foundation of high-stakes test-based accountability, Mr. Stroup and two other researchers said they believe they have found the reason: a glitch embedded in the DNA of the state exams that, as a result of a statistical method used to assemble them, suggests they are virtually useless at measuring the effects of classroom instruction.

Pearson, which has a five-year, $468 million contract to create the state’s tests through 2015, uses “item response theory” to devise standardized exams, as other testing companies do. Using I.R.T., developers select questions based on a model that correlates students’ ability with the probability that they will get a question right.

That produces a test that Mr. Stroup said is more sensitive to how it ranks students than to measuring what they have learned. That design flaw also explains why Richardson students’ scores on the previous year’s TAKS test were a better predictor of performance on the next year’s TAKS test than the benchmark exams were, he said. The benchmark exams were developed by the district, the TAKS by the testing company.

We have built a high stakes system on questionable tests, measured using questionable statistical models, subject to gaming and cheating, and further goosed by the scrubbing of other student data. We've seen widespread evidence of it in New York, California, Washington DC, Georgia, Tennessee, Pennsylvania, and now Ohio.

Policymakers are either going to have to spend more and more money developing better tests, better models, tighter security and more bureaucratic data handling policies, or return to thinking about the core mission of providing a quality education to all students. Either way, when you have reached the point where the State Superintendent talks of criminalizing the corporate education system, things have obviously gone seriously awry.

State Superintendent Stan Heffner, who leads the department, has launched his own investigation and has said the probe could lead to criminal charges against educators who committed fraud.

Virtual schools, virtually useless

Michael Morrison, writing for Decisions Based on Evidence, brings to our attention some recent reports on the failures of virtual schools (or e-schools) in places other than Ohio. Here's findings from Colorado

Minnesota is also finding similar problems

“While the number of course registrations has quadrupled over the last few years, full-time online students have become less likely to finish the courses they start. Course-completion rates for full-time online students dropped from 84 percent in the 2006-07 school year to 63 percent in 2009-10. During this period, several individual online schools experienced large and steady declines in course-completion rates, while only one program showed significant improvement.”

And in Pennsylvania, K-12 Inc.’s Agora Cyber Charter School's results are terrible.

Nearly 60 percent of its students are behind grade level in math. Nearly 50 percent trail in reading. A third do not graduate on time. And hundreds of children, from kindergartners to seniors, withdraw within months after they enroll.

We've mentioned K-12 Inc. before and noted they are Ohio's fastest growing virtual school provider. It appears there is a two fold reason why K-12 is Ohio's fastest growing, a reason that might also indicate why academic performance isn't so stellar. Stephen Dyer at 10th Period notes from K-12's financial filings

In fiscal year 2011, we derived approximately 13% of our revenues from each of the Ohio Virtual Academy and the Agora Cyber Charter School in Pennsylvania. In aggregate these schools accounted for approximately 26% of our total revenues. If our contracts with any of these virtual public schools are terminated, the charters to operate any of these schools are not renewed or are revoked, enrollments decline substantially, funding is reduced, or more restrictive legislation is enacted our business financial condition and results of operations could be adversely affected.

Dyer concludes

This means the laws in Ohio and Pennsylvania are so beneficial to online schools that one of the nation's biggest operators cannot exist without those laws remaining in place. As we reported last year at Innovation Ohio, Ohio Virtual Academy had a 51:1 student-teacher ratio, and this is on top of them getting enough state money to have a 15:1 student-teacher ratio and give $2,000 laptops to every child while still clearing 31.5% profit. In fact, they spend barely 10% of their money on teachers -- easily the lowest percentage of any of the major statewide eSchools. That means 90% of their $59 million in state money they got last year went to things other than teachers. But they don't have buildings, custodians, lunch ladies, or buses to maintain. So what where could the remaining $53 million in Ohio taxpayer money be going?

It would be a shame if K-12's milking the Ohio taxpayer to subsidize their other operations, as their SEC filing indicates it's doing.

An even greater shame that thousands of Ohio's virtual school students are being short changed a quality education at the expense of next quarters financial report.