U.S. News and World Report, as part of its expanded rankings business, unveiled its first list of top high schools. It’s a big improvement over Newsweek’s high-school ranking.
But, like with U.S. News’s college rankings, there are some questionable choices that leave the magazine open to criticism. For starters, high schools in 10 states and Washington, D.C., weren’t even considered because they didn’t provide the data U.S. News relied on for its analysis. (Many of those excluded are states with smaller populations; U.S. News claims its analysis covered 93% of the high school population.)
Still, U.S. News took a detailed approach in compiling its list. Newsweek’s Challenge Index, a high-profile ranking of public high schools, is based entirely on the number of advanced, standardized tests (Advanced Placement, International Baccalaureate and Cambridge) taken by students at the schools, divided by the number of graduating seniors. I wrote earlier this year that the list is more advocacy than science, and that its focus purely on the number of tests made it an inadequate measure of overall school quality.
U.S. News says its ranking, which hits newsstands Dec. 3 but has been released online, was inspired in part by the perceived problems with its rival’s methods. Newsweek’s methodology can be explained in a sentence; U.S. News’s partner in its ranking, Standard & Poor’s, explains its chosen tack in a 20-page document. The document begins by criticizing the Challenge Index on several fronts, including that it “ignores the question of quality entirely, because simply taking a test constitutes ‘good’ performance.” (A Newsweek spokeswoman declined to comment.)
U.S. News instead takes a three-part approach. I’m going to simplify it because explaining it in full, as the document above does, involves things like standard deviations and scatterplots. The magazine starts by setting two criteria, based on state tests, for each of the 18,500 high schools initially considered. These criteria are: (1) the school does better than expected, given the percentage of its students who are economically disadvantaged; and (2) the school’s economically disadvantaged students, plus its black and Hispanic students, beat the state average for their peers. only the 1,600 schools that make both cuts are evaluated further by a measure combining participation and performance on Advanced Placement tests. The top 100 are designated “gold” performers and ranked. The next 405 are called “silver,” and the remaining schools that made the first two cuts are awarded the “bronze.”
There are two noteworthy differences between this ranking and the magazine’s controversial, long-running college list. First, there isn’t the same level of choice for secondary education. Nonetheless, U.S. News editor Brian Kelly told me that for those parents and students with some choice, the list can help. It would also be of interest to taxpayers and education professionals. “The idea of improving high-school performance is viewed as essential,” he said. “Going forward, we think this data helps in that debate.”
Also very different: The magazine ranks every college, but more than 90% of high schools initially evaluated don’t make the cut. Even a school that just barely misses on one of the first two criteria is excluded entirely, rather than included but perhaps docked some points. That’s how it often works for high-school students — fail one crucial class, and As in the rest might not get you a diploma — but it may be too draconian for school rankings. “In any statistical analysis you have to make assumptions,” Mr. Kelly said. “You have to establish guidelines that are rational and helpful.” That said, he added that U.S. News may yet tinker with its formula.