What Are Standardized Tests Measuring?

If the same test produced different results depending upon whether it was taken online or on paper, what exactly is being measured? Here's ODE recently

And here's an article from Real Clear Education titled "PARCC Scores Lower for Students Who Took Exams on Computers"

Students who took the 2014-15 PARCC exams via computer tended to score lower than those who took the exams with paper and pencil—a revelation that prompts questions about the validity of the test results and poses potentially big problems for state and district leaders.

Officials from the multistate Partnership for Assessment of Readiness for College and Careers acknowledged the discrepancies in scores across different formats of its exams in response to questions from Education Week.

“It is true that this [pattern exists] on average, but that doesn’t mean it occurred in every state, school, and district on every one of the tests,” Jeffrey Nellhaus, PARCC’s chief of assessment, said in an interview.

“There is some evidence that, in part, the [score] differences we’re seeing may be explained by students’ familiarity with the computer-delivery system,” Nellhaus said.

In general, the pattern of lower scores for students who took PARCC exams by computer is the most pronounced in English/language arts and middle- and upper-grades math.

ODE would be better taking the time to understand exactly what all of its tests are actually measuring before getting too excited about how many students took paper vs online tests.