10 reasons why VAM is harmful to students

[...]No one is asking how value-added assessments may affect the very students that this evaluation system is intended to help. By my count, there are at least ten separate ways in which value-added assessment either does not accurately measure the needs of a student or is actually harmful to a child’s education. Until these flaws are addressed, value-added assessment will be nothing more than a toy for politicians and headline writers, not a serious tool for improving learning.

1. The premise of value-added assessment is that standardized tests are an accurate and decisive measure of student learning. In fact, standardized testing is neither definitive nor especially reliable. City and state exams are snapshots, not in-depth diagnostic tools.

2. Value-added assessments will ultimately require all students to take standardized exams, whether or not such examinations are developmentally appropriate. Kindergarteners and first graders will be subjected to the same pressures of high-stakes testing as older children.

3. Value-added assessments will dramatically increase the number of standardized tests for each student. Children will need to take exams in subjects such art, music and physical education in order to evaluate the teachers of these subjects.

4. The most successful students will get less enrichment work and more test prep. It is actually more difficult to improve the scores of gifted students since they have already done so well on standardized exams.

5. Teachers will need to avoid necessary remediation in order to attain short-term gains in test scores. Most standardized English tests require students to demonstrate high-order thinking skills, yet a growing body of academic research indicates that many children—especially those growing up in poverty—require huge boosts of vocabulary to function well in school. Teachers may be forced to forego a vocabulary-rich curriculum that would have the most long-term benefits for their children. Instead, they will have to focus on the skills that might help students gain an extra point or two on this year’s tests.

[readon2 url="http://www.dailykos.com/story/2013/03/11/1193372/-Ten-Reasons-Why-Value-Added-Assessments-are-Harmful-to-a-Child-s-Education"]Continue reading...[/readon2]

Value-Added and Teacher Branding

The video and report discuss the problems found with Value-add

Audrey Amrein-Beardsley and Clarin Collins of the Mary Lou Fulton Teachers College at Arizona State University present “The SAS Education Value-Added Assessment System (SAS® EVAAS®) in the Houston Independent School District (HISD): Intended and Unintended Consequences”.

The SAS Educational Value-Added Assessment System (SAS® EVAAS®) is the most widely used value-added system in the country. It is also self-proclaimed as “the most robust and reliable” system available, with its greatest benefit to help educators improve their teaching practices. This study critically examined the effects of SAS® EVAAS® as experienced by teachers, in one of the largest, high-needs urban school districts in the nation – the Houston Independent School District (HISD).

Using a multiple methods approach, this study critically analyzed retrospective quantitative and qualitative data to better comprehend and understand the evidence collected from four teachers whose contracts were not renewed in the summer of 2011, in part given their low SAS® EVAAS® scores.

This study also suggests some intended and unintended effects that seem to be occurring as a result of SAS® EVAAS® implementation in HISD. In addition to issues with reliability, bias, teacher attribution, and validity, high-stakes use of SAS® EVAAS® in this district seems to be exacerbating unintended effects.

Here's the video

Kasich pushes public ed to precipice

Already faced with having to deal with massive budget cuts created by the Governor's raid on school funding to balance his own budget, he has now asked all departments, including the Department of Education to provide budget scenarios that include a further 10% cut

State agency heads will be lucky if they get to keep current funding levels when Gov. John Kasich rolls out the next state budget in the spring.

That's the tone being set by a budget guidance document released Monday by Kasich's administration that asks state agencies to plan a pair of scenarios for the 2014-15 budget -- one where they see no growth in funding and a second in which agencies are hit with a 10 percent cut in general revenue funds.

The Governor's own Office of Budget and Management ran the number to discern the impact. It would be ironic if the Governor killed his own education policies via his own fiscal policy.

In areas of academic improvement, the results are dire

• A 10% cut could jeopardize Ohio's ESEA flexibility waiver by reducing services by up to 4,000 hours and not providing the additional support to priority and focus schools required. The 16 fiscal agents (ESCs) that deliver the Ohio Improvement Process for districts in school improvement already do not have the capacity to support all districts in need of support.
• A 10% cut to Ohio's Alternative Education Grant programs could result in as many as 29 programs closing resulting in increased drop-out rates and decreased graduation rates. Students who drop out often result in long term costs to the state through various social service and remediation programs. This funding has experienced a reduction of funding of a little over 40% to what was originally budgeted for in FY09

On Curriculum and assessment, the cuts would end the efforts to increase academic rigor and hinder the newly legislated (and already unfunded) 3rd grade reading guarantee.

• A 10% cut to funding for Content Standards and Model Curricula would jeopardize ODE's ability to manage all work required in statute in this area, such as embedding career connections into Ohio's model curricula. Additional cuts could also jeopardize Ohio's ESEA flexibility waiver by significantly hindering ODE's ability to provide technical assistance to districts in implementing the new academic content standards that begin in the 2013-14 school year. Reductions will jeopardize planned professional development supporting best practices instructional strategies, infusing technology into instruction, and supporting instruction for diverse learners in implementing the new State Learning Standards. This set-aside has experienced a cut of over 28% to amounts originally budgeted for in FY09.
• A 10% cut to Assessment Contracts and & Administration would result in statutory noncompliance for Ohio's student assessment system and would jeopardize Ohio's ESEA flexibility waiver because the funding would not allow the new generation of assessments being developed through the Partnership for Assessment of Readiness for College and Careers (PARCC) and state-developed assessments for science and social studies to begin in the 2014-15 school year.
• A 10% cut to funding for the Kindergarten Readiness Assessment could hinder the release and administration of this literacy assessment which is critical to successful early childhood learning and the implementation of the Third Grade Reading Guarantee included in SB 316.

Cuts would also jeopardize district data collection and analysis, something ODE is already struggling to grapple with as the attendance scrubbing controversy drags on. While we can't currently publish school report cards, these cuts would also risk further future delays putting federal monies at risk.

Under educator quality efforts:

• A 10% cut in funds for Educator Standards and Prep would mean new work required under statute, particularly for teachers and principals, could not be properly managed. Such work includes the implementation of the new teacher and principal evaluation systems required by statute and using evaluation systems to inform decisions including professional development, promotion, retention, placement, tenure and removal of ineffective educators.
• 10% cut to Teacher and Principal Evaluation/Student Growth, could put in jeopardy the continued implementation of the teacher and principal evaluation systems which are included as a part of Ohio's ESEA flexibility waiver. Also, the incorporation of student growth as a metric in these systems, mandated by state law, could be put in jeopardy. LEA requests for support in implementation have been overwhelming and the need for support will continue to grow.

It would be ironic if the Governor killed his own education policies via his own fiscal policy.

Here's the full analysis

Impact of a further 10% cut to Ohio's education budget

3rd grade retention plan could cost $500 million

One of the signature policy initiatives in the Kasich MBR is the proposed change in the 3rd grade reading guarantee.

Specifically, according to LSC analysis, the bill (SB316) makes several changes to the third grade reading guarantee beginning with the 2012-2013 school year. Under current law, the third grade reading guarantee requires school districts and community schools to retain in third grade a student who scores in the "limited" range on the third grade English language arts assessment, unless the student's principal and reading teacher agree that the student is academically prepared for fourth grade or the student will receive intervention services in fourth grade. The bill changes the "cut" score and applies the guarantee to all students who do not receive at least a "proficient" (or passing) score on the assessment. The "limited" score, which currently triggers the guarantee, is the lowest of five scoring ranges and two levels below "proficient."

In short, more students will be held back, and less flexibility will be granted to educators in determining if a student who misses the proficient level can proceed to the fourth grade.

None of this expansion is funded, so let's take a look at what this policy might additionally cost cash strapped schools.

In October 2011, a total of 126,569 3rd grade Ohio public school students participated in the Reading Achievement Test. Here are the aggregated results, according to ODE statistics.

Level	Number	Percent
Advanced	22,987	18.2%
Accelerated	23,619	18.7%
Proficient	28,038	22.2%
Basic	23,574	18.6%
Limited	28,351	22.4%

In recent years the number of students scoring proficient or higher has varied from a high of 67.5% to a low of 53.3%. Remember, according to the new proposal, any student scoring below proficient is likely to be held back and made to repeat 3rd grade.

Again, according to ODE statistics, the median cost per pupil in Ohio per year in 2011 was $9,567.89, with an average of $9,961.57.

Under the new rules, the 51,925 students who failed to reach the minimum proficiency standard would have been at risk of being held back. At a median cost of $9,961 per student, districts could be on the hook for a total of $517,224,925 to fund that many students repeating 3rd grade.

To put that into some perspective, the crisis in Cleveland public schools is caused by a budget shortfall of $65 million. This unfunded manade could pay for that shortfall 8 times over.

Misconceptions and Realities about Teacher Evaluations

A letter, signed by 88 educational researchers from 16 universities was recently sent to the Mayor of Chicago regarding his plans to implement a teacher evaluation system. Because of some of the similarities of the Chicago plan to that of Ohio, we thought we would reprint the letter here.

In what follows, we draw on research to describe three significant concerns with this plan.

Concern #1: CPS is not ready to implement a teacher-evaluation system that is based on significant use of “student growth.” For Type I or Type II assessments, CPS must identify the assessments to be used, decide how to measure student growth on those assessments, and translate student growth into teacher-evaluation ratings. They must determine how certain student characteristics such as placement in special education, limited English-language proficiency, and residence in low-income households will be taken into consideration. They have to make sure that the necessary technology is available and usable, guarantee that they can correctly match teachers to their actual students, and determine that the tests are aligned to the new Common Core State Standards (CCSS).

In addition, teachers, principals, and other school administrators have to be trained on the use of student assessments for teacher evaluation. This training is on top of training already planned about CCSS and the Charlotte Danielson Framework for Teaching, used for the “teacher practice” part of evaluation.

For most teachers, a Type I or II assessment does not exist for their subject or grade level, so most teachers will need a Type III assessment. While work is being done nationally to develop what are commonly called assessments for “non-tested” subjects, this work is in its infancy. CPS must identify at least one Type III assessment for every grade and every subject, determine how student growth will be measured on these assessments, and translate the student growth from these different assessments into teacher-evaluation ratings in an equitable manner.

If CPS insists on implementing a teacher-evaluation system that incorporates student growth in September 2012, we can expect to see a widely flawed system that overwhelms principals and teachers and causes students to suffer.

Concern #2: Educational research and researchers strongly caution against teacher-evaluation approaches that use Value-Added Models (VAMs).

Chicago already uses a VAM statistical model to determine which schools are put on probation, closed, or turned around. For the new teacher-evaluation system, student growth on Type I or Type II assessments will be measured with VAMs or similar models. Yet, ten prominent researchers of assessment, teaching, and learning recently wrote an open letter that included some of the following concerns about using student test scores to evaluate educators[1]:

a. Value-added models (VAMs) of teacher effectiveness do not produce stable ratings of teachers. For example, different statistical models (all based on reasonable assumptions) can yield different effectiveness scores. [2] Researchers have found that how a teacher is rated changes from class to class, from year to year, and even from test to test. [3]

b. There is no evidence that evaluation systems that incorporate student test scores produce gains in student achievement. In order to determine if there is a relationship, researchers recommend small-scale pilot testing of such systems. Student test scores have not been found to be a strong predictor of the quality of teaching as measured by other instruments or approaches. [4]

c. Assessments designed to evaluate student learning are not necessarily valid for measuring teacher effectiveness or student learning growth. [5] Using them to measure the latter is akin to using a meter stick to weigh a person: you might be able to develop a formula that links height and weight, but there will be plenty of error in your calculations.

Concern #3: Students will be adversely affected by the implementation of this new teacher-evaluation system.

When a teacher’s livelihood is directly impacted by his or her students’ scores on an end-of-year examination, test scores take front and center. The nurturing relationship between teacher and student changes for the worse, including in the following ways:

a. With a focus on end-of-year testing, there inevitably will be a narrowing of the curriculum as teachers focus more on test preparation and skill-and-drill teaching. [6] Enrichment activities in the arts, music, civics, and other non-tested areas will diminish.

b. Teachers will subtly but surely be incentivized to avoid students with health issues, students with disabilities, students who are English Language Learners, or students suffering from emotional issues. Research has shown that no model yet developed can adequately account for all of these ongoing factors. [7]

c. The dynamic between students and teacher will change. Instead of “teacher and student versus the exam,” it will be “teacher versus students’ performance on the exam.”

d. Collaboration among teachers will be replaced by competition. With a “value-added” system, a 5th grade teacher has little incentive to make sure that his or her incoming students score well on the 4th grade exams, because incoming students with high scores would make his or her job more challenging.

e. When competition replaces collaboration, every student loses.

You can read the whole letter below.

Misconceptions and Realities about Teacher and Principal Evaluation

Parents Agree – Better Assessments, Less High-Stakes Testing

Educators aren’t alone in being fed up with narrow, punitive student accountability measures. Parents also want well-designed, timely assessments that monitor individual student performance and progress across a range of subjects and skills. That’s one of the key findings in a new study by the Northwest Evaluation Association (NWEA).

NWEA, a non-profit educational services organization headquartered in Portland, set out to find how the views of parents – often ignored in the debate over the direction of public education – stacked up against those of teachers and administrators.

After conducting online surveys of more than 1,000 respondents, NWEA found that these stakeholders essentially want the same thing. Large majorities say that, although year-end tests might provide some sort of useful snapshot, they strongly prefer more timely formative assessments to track student progress and provide educators with the flexibility to adjust their instruction during the school year.

“The research reinforces the notion that no one assessment can provide the breadth and depth of information needed to help students succeed,” explained Matt Chapman, president and CEO of NWEA. “For every child we need multiple measures of performance.”

As the reauthorization of the Elementary and Secondary Education Act (ESEA) slowly moves on Capitol Hill, redefining how student progress is measured will be a key debate. The National Education Association believes it is time to move beyond the No Child Left Behind Law (the 2001 revision of ESEA), scrap the obsession with high-stakes testing and enter into a new phase of education accountability.

“Well-designed assessment systems do have a critical role in student success,” said NEA President Dennis Van Roekel. “We should use assessments to help students evaluate their own strengths and needs, and help teachers improve their practice and provide extra help to the students who need it.”

“I use different types of assessments because all students are different,” explained Krista Vega, a middle school teacher in Maryland and NEA member who participated in the NWEA survey. “I use quizzes, games, teacher-made tests, computerized tests, portfolios, and alternative assignments.”

“What I’m looking for is, first, are they mastering the skill I’m trying to teach, or did they not master the skill? I’m looking to see if there is an area of weakness. I’m looking to see if they have background knowledge sometimes. There’s just a whole range of things that I’m looking for,” Vega said.

Source: Northwest Evaluation Association and Grunwald Associates

According to the survey, it is the types of formative assessments Vega identifies, such as quizzes, portfolios, homework and end-of-unit tests that provide timely data about individual student growth and achievement. Respondents cited these types of assessments as providing educators with the necessary information to pace the instruction and ensure students learn fundamental skills.

Parents are also worried about the narrowing of the curriculum. Large majorities believe it is important to measure students in math and English/language arts but also say it is important to measure performance in science, history, government and civics, and environmental literacy.

The students who are often hurt the most by a restricted curriculum are those who don’t have the opportunities, because of their socioeconomic background, to diversify their learning outside the classroom.

Beyond subject matter, parents and educators believe so-called “higher order” thinking skills such as creativity, communication, problem-solving, and collaboration – so critical in the modern economy and workplace – aren’t being properly measured by current assessment systems.

“It is really, really important,” Vega says “that we prepare students for when they enter the workforce to compete in the 21^st century.”

Read the NWEA Report ”For Every Child, Multiple Measures”

More on NEA’s Position on Student Assessments (Word Document)