Gates Foundation Wastes More Money Pushing VAM

Makes it hard to trust the corporate ed reformers when they goose their stats as badly as this.

Any attempt to evaluate teachers that is spoken of repeatedly as being "scientific" is naturally going to provoke rebuttals that verge on technical geek-speak. The MET Project's "Ensuring Fair and Reliable Measures of Effective Teaching" brief does just that. MET was funded by the Bill & Melinda Gates Foundation.

At the center of the brief's claims are a couple of figures (“scatter diagrams” in statistical lingo) that show remarkable agreement in VAM scores for teachers in Language Arts and Math for two consecutive years. The dots form virtual straight lines. A teacher with a high VAM score one year can be relied on to have an equally high VAM score the next, so Figure 2 seems to say.

Not so. The scatter diagrams are not dots of teachers' VAM scores but of averages of groups of VAM scores. For some unexplained reason, the statisticians who analyzed the data for the MET Project report divided the 3,000 teachers into 20 groups of about 150 teachers each and plotted the average VAM scores for each group. Why?

And whatever the reason might be, why would one do such a thing when it has been known for more than 60 years now that correlating averages of groups grossly overstates the strength of the relationship between two variables? W.S. Robinson in 1950 named this the "ecological correlation fallacy." Please look it up in Wikipedia. The fallacy was used decades ago to argue that African-Americans were illiterate because the correlation of %-African-American and %-illiterate was extremely high when measured at the level of the 50 states. In truth, at the level of persons, the correlation is very much lower; we’re talking about differences as great as .90 for aggregates vs .20 for persons.

Just because the average of VAM scores for 150 teachers will agree with next year's VAM score average for the same 150 teachers gives us no confidence that an individual teacher's VAM score is reliable across years. In fact, such scores are not — a fact shown repeatedly in several studies.

[readon2 url="http://ed2worlds.blogspot.com/2013/01/gates-foundation-wastes-more-money.html"]Continue reading...[/readon2]

Why Pay for Performance Should Get the Sack

The following article discusses the problems with perfoamcne pay in the financial sector, the heat of capitalism. Extrapolating this compensation gimmick to educators as corporate education reformers are seeking to do continues to be proven problematic

By Bruno S. Frey, Professor of Economics at the University of Zurich and Margit Osterloh, Professor (em.) for Business Administration and Management of Technology and Innovation, University of Zürich; and Professor, Warwick Business School. Cross posted from VoxEU

As the bonus culture in the financial sector once again comes under attack, this column rubbishes the typical defence that banks need to pay top dollar to attract the best talent.

Scientific literature has extensively dealt with variable pay-for-performance. Despite the fact that serious problems linked to this approach have thus become obvious, many authors continue to support compensation according to predetermined performance criteria because they are committed to the traditional concept of the ’homo oeconomicus’.
Overall, there has been a marked change of opinion in academia (see for instance Bryson and Freeman 2008 on this site). The idea that people are solely self-interested and materially orientated has been thrown overboard by leading scholars. Empirical research, in particular experimental research, has shown that under suitable conditions human beings care for the wellbeing of other persons. Above all, they are not solely interested in material gains (see eg Frey and Osterloh 2002). Recognition by co-workers is greatly important. Many workers are intrinsically motivated, ie they perform work for its own sake because it is found challenging and worth undertaking. This applies not only to qualified employees but also to persons fulfilling simple tasks. They often are proud of their work and performance.

There are four major arguments against variable pay-for-performance:

• In a modern economy, it is practically impossible to determine tasks that are to be fulfilled in the future precisely enough so that variable pay-for-performance can be applied. In a society continually faced with new challenges, superiors oftentimes find it impossible to fix ex ante what an employee will have to do in the future.
• It would be naïve to assume that the persons subjected to variable pay-for-performance would accept the respective criteria in a passive way and fulfil their work accordingly. Rather, they spend much energy and time trying to manipulate these criteria in their favour. This is facilitated by the fact that employees often know the specific features of their work better than their superiors. The wage explosions observable in many sectors of the economy can at least partly be attributed to such manipulations, eg when managers are able to contract easily achievable performance goals.
• Variable pay-for-performance results in employees restricting their work to those areas covered by the performance criteria. In the literature, this is known as the ’multiple tasking’ problem. This may induce employees to spend considerable time and energy during their work trying to find a better-paid job with another firm. They therefore neglect their tasks insofar as they are not contractually fixed by the performance criteria.
• Variable pay-for-performance tends to crowd out intrinsic work motivation and therewith the joy of fulfilling a particular task. However, such motivation is of great importance in a modern economy because it supports innovation and helps to fulfil tasks going beyond the ordinary.

[readon2 url="http://www.nakedcapitalism.com/2011/09/why-pay-for-performance-should-get-the-sack.html"]Continue reading...[/readon2]