Should We Be Concerned With the Ranking of U.S. Students in International Standardized Test Scores?

In this post we will explore 3 issues, 2 of which lay the foundation necessary to answer the question above.

1) Are standardized tests valid as a metric of student’s academic knowledge or perhaps even general intelligence?

2) What does the ranking of U.S. students truly represent?  Is it a fair ranking?

3) Should we be concerned?

Let’s get to it…

1) Are standardized tests valid as a metric of student’s academic knowledge or perhaps even general intelligence?

Standardized test scores do a decent job of predicting a students’ grades, particularly near-future grades, e.g. SAT scores do a fairly good job of predicting freshmen year GPA’s.  Further out you go, the correlation begins to drop a bit.  They are also correlated to general IQ tests, like the Stanford-Binet and the Wechsler scales.  In fact, as single measures, there are few tools that are more associated to these academic and IQ outcomes.  Having said that, they explain MUCH less than half the variation in these outcomes, meaning there are a LOT of other factors you have to throw into the mix, such as socioeconomic status, a student’s own academic behaviors, parental academic involvement, teachers, the school, peer influences, etc..

2) What does the ranking of U.S. students truly represent?  Is it a fair ranking?

The ranking is fair insofar as there is no particular test bias that would systematically result in an underestimation of U.S. students’ scores relative to other nationalities, which is often how academics would define fairness. Is the ranking itself fair in a more general sense?  Well, that’s where thing get more interesting.  When U.S. schools are included in these international measures of student achievement and aptitude, there is generally an honest attempt at providing a representative sample that captures the diversity of American public schools.  In other words, you don’t have a situation where only schools from the wealthiest districts participate, or the poorest, or the least ethnically diverse, etc.. This is not always the case with other countries. One complaint made in a recent series of tests that showed exceptional performance among Chinese students was that their sample was grossly over-represented by schools in the one of the wealthiest, most technologically advanced and cosmopolitan cities in the Chinese mainland—Shanghai.  Critics have argued, correctly I’d say, that these schools are hardly representative of all schools in China.

3) Should we be concerned?

Yes and no.  In a general sense, I think we want to be ranked higher.  Why not?  We are the wealthiest country in the world.  Our university system is the envy of every other country. We produce more nobel laureates and our scientists publish more in the top scientific journals, etc…  So why shouldn’t our grade-schoolers and high schoolers also perform near the top on these tests.  Fine, but here are two other things to keep in mind.  Our ranking HAS improved over time. We have gone up several spots and are now signficantly above average on these measures.  There is no reason to believe our ranking won’t continue to improve.  We also do not want the cart pulling the horse.  These test scores should be indicative of better and more effective learning, not the other way around.  To use the old cliché, we should not be obsessed with teaching to the test.  We may end up improving our test scores, but killing a lot of the other great attributes we do a good job of inculcating in our students, such as creativity, autonomous effort, intrinsic motivation, etc.  This is not an empty platitude.  Educational researchers in China are VERY concerned with this very issue and are quite active in trying to import American style of teaching and American theories of child development into their early childhood educational programs.

** As is my practice, I will be adding in-text reference citations to claims I have made above as I find time to track them down and update the blog.