Such confusion is standard in the world of “standardized” testing. The seemingly statistical impossibility occurs because “national average” in the widely used tests does not mean the average of all students who took the exam this year. Instead, scores are measured against a sample of students who took the test at a set point in time, sometimes years before. Math scores on the Metropolitan Achievement Test (MAT) used by New York City were compared with a sample from 1985; the “norming group” for the reading test, from a different publisher, was from 1988. “It’s theoretically possible for every child to test above the national average if the norm is old,” says Stephen Koffler, a leading test expert. Some critics claim that is not just possible - but probable, since teachers become more familiar with the exam each year and tend to drill students accordingly.
That does not necessarily mean students are faring well, however. Just last month the federal National Assessment of Educational Progress (NAEP) issued an alarming report that half the nation’s eighth graders are performing just above fifth-grade level in math. Nationwide, only 14 percent of eighth graders scored at seventh-grade level or above. The only good news was that NAEP provided the first state-by-state comparison of what U.S. students know and don’t know. “Too many people believe ‘The nation is at risk, but I’m OK’,” said Education Secretary Lamar Alexander. “This should disabuse them of that notion.”
Far too often, test results show far more about the vagaries of scoring methods than about educational achievement. Amid a national outcry for accountability, testing is burgeoning - to a record 127 million exams each year, by one count. In some districts, test scores are used to determine funding levels, teacher bonuses, superintendent salaries, even student diplomas, so there is enormous pressure to produce high scores. Yet the results are not comparable or even comprehensible. “Parents are confused,” says Francie Alexander, an associate superintendent of the California schools. “They get their child’s grades from the teacher in the classroom, then they get test scores that are supposed to say how the system is doing, and none of it matches.”
Test publishers vigorously defend their “norm based” scoring methods, so long as the proper qualifiers are included. “The key factor is describing performance relative to the norm, where the norm group is described completely,” says Joanne Lenke of Psychological Corp., which publishes the MAT. The trouble is that such information is often relegated to the fine print of news accounts, if it is included at all. New York City school officials insist that their scores do represent real gains: kids are getting more answers right each year. But critics say that, too, could simply mean teachers are getting better at anticipating test questions. What’s more, school officials can “select tests that will make their school systems look good–it’s an unregulated industry,” says former assistant education secretary Chester E. Finn Jr. “The American public needs a credible outside audit.”
Many educators agree - and a consensus is building for a single set of national exams. In April, President Bush called for the creation of “American achievement tests” covering math, English, science, history and geography for all fourth, eighth and 12th graders. Details are still on the drawing boards, but experts agree on several principles: the tests should measure whether students have mastered specific skills at each grade level, as the NAEP tests do, rather than compute averages that can mask the fact that even “top” schools are performing abysmally. The tests should also let students demonstrate their abilities by writing essays and conducting lab experiments, as well as answering multiple-choice questions. If teachers then “teach to the test,” fine. “You need to have the kind of tests you can teach to,” says Lauren Resnick, a top test authority who is helping to devise the standards. “By doing so, you’re providing a good education.”
Agreeing on what 40 million American schoolchildren should know from year to year won’t be easy. Neither will getting the national tests through Congress. But test results that can be readily understood and compared are desperately needed. “If we’re going to pursue excellence, we need to know when we’ve achieved it,” says Linda Darling-Hammond of Columbia University’s Teachers College. One thing is certain: we are not there yet.
