Socioeconomic inequality among U.S. students skews international comparisons of test scores, finds a new report released today by the Stanford Graduate School of Education and the Economic Policy Institute. When differences in countries' social class compositions are adequately taken into account, the performance of U.S. students in relation to students in other countries improves markedly.
An accurate comparison of nations' test scores must include a look at the social class characteristics of the students who take the test in each country, says Stanford education Professor Martin Carnoy. The report, What do international tests really show about U.S. student performance?, also details how errors in selecting sample populations of test-takers and arbitrary choices regarding test content contribute to results that appear to show U.S. students lagging.
In conducting the research, report co-authors Martin Carnoy, a professor of education at Stanford, and Richard Rothstein, a research associate at the Economic Policy Institute, examined adolescent reading and mathematics results from four test series over the last decade, sorting scores by social class for the Program on International Student Assessment (PISA), Trends in International Mathematics and Science Study (TIMSS), and two forms of the domestic National Assessment of Educational Progress (NAEP).
Based on their analysis, the co-authors found that average U.S. scores in reading and math on the PISA are low partly because a disproportionately greater share of U.S. students comes from disadvantaged social class groups, whose performance is relatively low in every country.
As part of the study, Carnoy and Rothstein calculated how international rankings on the most recent PISA might change if the United States had a social class composition similar to that of top-ranking nations: U.S. rankings would rise to sixth from 14th in reading and to 13th from 25th in math. The gap between U.S. students and those from the highest-achieving countries would be cut in half in reading and by at least a third in math.
"You can't compare nations' test scores without looking at the social class characteristics of students who take the test in different countries," said Carnoy. "Nations with more lower social class students will have lower overall scores, because these students don't perform as well academically, even in good schools. Policymakers should understand how our lower and higher social class students perform in comparison to similar students in other countries before recommending sweeping school reforms."
The report also found:
There is an achievement gap between more and less disadvantaged students in every country; surprisingly, that gap is smaller in the United States than in similar post-industrial countries, and not much larger than in the very highest scoring countries.
Achievement of U.S. disadvantaged students has been rising rapidly over time, while achievement of disadvantaged students in countries to which the United States is frequently unfavorably compared – Canada, Finland and Korea, for example – has been falling rapidly.
But the highest social class students in United States do worse than their peers in other nations, and this gap widened from 2000 to 2009 on the PISA.
U.S. PISA scores are depressed partly because of a sampling flaw resulting in a disproportionate number of students from high-poverty schools among the test-takers. About 40 percent of the PISA sample in the United States was drawn from schools where half or more of the students are eligible for the free lunch program, though only 32 percent of students nationwide attend such schools.
(Read more at Stanford)