Colleges’ Average SAT Scores Should No Longer Be Used in Rankings

The time has come for U.S. News & World Report to stop using SAT scores to rank colleges in their annual Best Colleges list. The practice never made sense as colleges had nothing to do with these scores, and is even more preposterous now that so many students have been unable to take the test as a result of the Coronavirus pandemic.

U.S. News — gatekeepers of the most prominent college rankings’ lists — has long been under fire for a college rankings schema that puts more emphasis on the students they select rather than how they educate them. Nowhere is this preference for inputs over outputs more evident than the weight placed on standardized tests, which students take long before they step foot on campus.

U.S. News annually ranks over 1,400 colleges and universities based on how they stack up on 17 measures including (in descending order of weight): graduation and retention rates, undergraduate academic reputation (a peer assessment), faculty resources, financial resources per student, student selectivity, graduate indebtedness, social mobility (based on Pell grant graduates), and average alumni donations. U.S. News uses publicly available and self-reported data for their rankings.

Testing Troubles

Since last March, millions of students have been unable to sit for the SAT and ACT exams due to the pandemic. While ACT Inc. offers an online, proctored version of their test, not every high school student has adequate access to strong internet service and a quiet, distraction-free place to take the test.

Because of the disruption caused by the pandemic, more than half of all four-year colleges made it either optional for applicants to submit a standardized test score or are not considering test scores at all when making their admissions decisions. As a result, only about two out of every five students who used the Common Application to apply for college for the fall have submitted test scores.

The Fiske Guide to Colleges announced in March that it would not include average SAT or ACT scores in its annual evaluations of colleges since the scores aren’t reliable with so few students submitting them. “We take pride in our reputation as a trusted source of reliable admissions information,” Edward B. Fiske, the Guide’s editor, told Inside Higher Ed. “Rather than publish inaccurate and misleading data, we have decided to omit any reporting of score ranges for the foreseeable future. To do otherwise would be a disservice to our readers.”

U.S. News, however, will still use colleges’ average SAT scores in its rankings, despite the challenges of the pandemic, according to Robert Morse, the magazine’s chief data strategist. “U.S. News cannot predict which ... COVID-19 related changes will be temporary and which will be long-term, but expects that it’s likely most schools surveyed will continue to make significant use of the SAT and ACT,” Morse said via email.

The next edition of rankings to be published in the fall will use the fall 2020 admission and entering class data. Robert Morse said that according to most schools they’ve talked to, this data was not impacted by COVID-19 since the application and admission process was “more or less fully completed pre-COVID.”

Suspect Methodology

Morse argues that the magazine’s methodology reflects the current higher education landscape as “the vast majority of U.S. colleges continue to believe that the SAT/ACT scores are an indicator of student success in college, which is one main reason U.S. News includes it in the methodology.”

Evaluating colleges based on the high school grades and test scores of freshmen doesn’t make sense, as those measures do not provide any information about the quality of education the colleges provide.

Judging schools by those factors simply ensures that colleges everyone expects to be at the top of the list — such as Harvard, Yale, and Princeton — remain there. In fact, most of the criteria that U.S News uses to rank colleges give an advantage to these Ivy League schools and other highly-selective colleges because of their wealth and the fact that they have long been considered the most prestigious.

Standardized test scores have been shown to be representative of the wealth of the students’ family. The SATs have long been criticized for being discriminatory, and test coaching gives wealthy students an unfair advantage. Affluent students can afford expensive tutors to teach them the tricks of the test.

The magazine’s reliance on average test scores has spurred colleges to use a large share of their institutional aid dollars to pursue affluent students with high scores. The result has been a “merit aid arms race” that causes schools to give out ever-increasing non-need-based aid scholarships and tuition discounts so they can raise their average test scores. Meanwhile, these institutions are increasingly leaving low-income and working-class students with greater amounts of unmet financial need, making it more difficult for them to succeed in college.

In defending the use of standardized test scores, Morse has argued that they represent a very small share of a college’s overall ranking. Currently, admitted students’ SAT and ACT score is worth 5 percent of the student selectivity category, which is worth 7 percent of the total rankings. This percentage has been in decline over time. In the 2018 edition test scores were rated at 8.125 percent, and in the 2019 edition at 7.75 percent.

But simply looking at how much the scores count in the rankings undervalues the importance that U.S. News assigns to them. Just consider the substantial role they play in the rankings of colleges that don’t require students to submit test scores when applying.

Growth of 'Test Optional'

Over the last decade, the “test optional” movement — schools that leave it up to students to decide whether to report their scores in their college applications — has been growing. Before the pandemic, more than 1,000 schools had chosen to go test-optional, according to the National Center for Fair and Open Testing (FairTest). Hundreds of other colleges have joined them in the last year, since so many high school students haven’t been able to take the tests.

U.S. News continues to count the average standardized test scores that test-optional colleges report, as long as at least 75 percent of the incoming class submitted scores. The magazine deducts 15 percent off of the colleges’ average scores if they don’t meet that threshold. In other words, the publication penalizes schools where the test optional policy is popular by assigning them an arbitrarily low score.

The penalty is even worse for colleges that go “test blind,” or ignore standardized tests altogether when admitting students. For years, U.S. News excluded these schools from the popular rankings, dissuading all but a handful from doing so. Going unranked in a magazine read by millions of students and families deciding where to go to school is a huge risk for a college as it may appear that the school’s quality has diminished. And it denies the institution the type of exposure its competitors are getting. At least one school — Sarah Lawrence College in New York — that went this route was forced to reverse course, and start accepting test scores again, after acknowledging that being unranked had put it at a competitive disadvantage.

But last year, in the face of the pandemic, US News itself reversed course, announcing that it would, for the first time, include test blind schools in the 2021 rankings, which were published in September. Less heralded was the fact that these schools would continue to be penalized. In the methodology for its 2021 rankings, the magazine revealed that it would assign these schools an arbitrarily low average SAT score — “equal to the lowest test score by a ranked school in their category.”

"A Circular Argument"

Jon Boeckenstedt, the enrollment manager at Oregon State University, said these policies show how absurd the rankings are.

“The ratings are a commercial endeavor designed to sell magazines and/or ad space on a website,” said Boeckenstedt, a long-time critic of the College Board, which administers the SAT. “And the pathetic need of the magazine to scramble for some way to measure institutional quality by making up values for an anachronistic test should point out the absurdity of the exercise.”

“It’s a circular argument, of course,” he continued. “They need test scores to put you in a category, but they’ll make up test scores to keep you in the category they’ve already put you in.”

Boeckenstedt is correct. The U.S. News rankings do not measure institutional quality and have had an outsized and pernicious effect on college behavior. But since they are not going anywhere, they should at least continue to make changes that would entice colleges to become more socioeconomically and racially diverse, instead of penalizing them for doing so.

With millions of students still not able to take the tests as a result of the pandemic, and with so many colleges going test optional or test blind, the time has come for U.S. News to stop using standardized test scores in its rankings. Colleges should be judged by their performance, and not by the test scores or high school grade point averages of the students they enroll.

Enjoy what you read? Subscribe to our newsletter to receive updates on what’s new in Education Policy!