Reporting on College Rankings and Ratings

Various organizations rank the top universities in the United States in an attempt to help prospective college students figure out where to go school.

That’s quite the task with slightly more than 5,900 postsecondary Title IV institutions in the United States in 2020-21, according to the National Center for Education Statistics.

But college rankings have been synonymous with criticism, and several medical and law schools have announced they will no longer participate in the U.S. News & World Report rankings.

Higher education reporters can go beyond the rankings by exploring the methodology, asking university administrators about the rankings and seeing if rankings help underrepresented students.

Early college rankings in the United States looked at the number of distinguished men who studied, graduated from or taught at a particular university. The first U.S. college ranking dates back to 1900 when “Where We Get Our Best Men” ranked universities by how many prominent men attended.

Rankings based on peer-review opinion emerged in the 1920s and became popular as the importance of a university’s reputation outweighed the reputation of a university’s graduates. The U.S. News & World Report was the first to publish undergraduate reputational rankings in 1983.

In the past decade, the federal government has proposed different college-rating systems with only some degree of success. 

Former President Barack Obama tried launching a college-rating system in 2013 that would have ranked every institution of higher education by factoring in their accessibility, affordability and student performance after graduation. This was widely criticized by college administrators, and the Obama administration gave up on a federal college-rating system in 2015. 

Instead, the U.S. Department of Education launched the College Scorecard, which gives information on a school’s cost, graduation rates, school sizes, salary after graduation and student debt, among other factors.

Now the Biden administration is working on publishing a list of programs that are considered to have low financial value for students.

Here’s what else reporters should know when evaluating college rankings – or ratings: 

Look at an Organization’s Methodology for Ranking Colleges.

It’s crucial to know what metrics go into evaluating colleges and understand where organizations are getting their data.

Forbes ranks colleges using seven measures — 20% is alumni salary; 15% is debt; 15% is graduation rate, and 10% is retention rate. The publication uses federal data from the Integrated Postsecondary Education Data System (IPEDS) and College Scorecard.

Though its methodology can change, U.S. News & World Report currently determines its Best Colleges rankings based on 17 measures. Among its ranking factors, 22% is graduation and retention rates; 20% is expert opinion, 10% is financial resources per student, and 5% is social mobility. Seven percent is student selectivity for the entering class — including aspects of the ACT, SAT and high school rankings.

“This is actually a huge controversy with the U.S. News & World Report because they are relying on metrics that folks say don’t really reflect the value of college,” said Jeremy Bauer-Wolf, a senior reporter at Higher Ed Dive.

Expert opinions accounted for one-fifth of each college’s overall rank in 2022 — meaning presidents, provosts and deans of admissions rate the academic quality of peer institutions on a scale of one to five. 

“Pose those questions to administrators: How do you feel about being ranked based on someone else’s opinion of you in the survey?” Bauer-Wolf said. “If an administrator sits down with this and they are just doing this haphazardly or don’t put that much thought into this and it’s a huge weight in the US News and World Reports ranking, well doesn’t that question the legitimacy of it? It’s a question that reporters should ask.”

It’s crucial to double check the data used to determine the rankings, especially since they might not always be accurate. Columbia University was discovered in September 2022 to have used misleading data to earn a higher ranking in last year’s U.S. News & World Report.

“There’s good reasons to check everything a college tells you when you’re a journalist because the sad reality is that experience has proven that it often isn’t true,” said Jon Marcus, higher education editor at The Hechinger Report.

It is also important to know how different categories are emphasized to determine their rankings.

“Depending on what a college chooses to put emphasis, energy, and resources into, they might climb one rankings list but not another because of how these methodologies are broken down,” said Katie Burns, a college admissions counselor at IvyWise, a for-profit New York-based firm of educational consultants that helps students pursue admission to college.

Do College Rankings Help Underrepresented Student Groups? 

First-generation students probably don’t use college rankings when selecting a school, Marcus said.

“I don’t know that the rankings necessarily are going to help them very specifically in speaking to issues like diversity,” he said. “I think it’s even more important to take a step back and help first-generation students know about any resources that they can use to pick a college.”

If an institution wants to move up in the ranking, Bauer-Wolf said they might cut out some prospective students who might not have high SAT or ACT scores because that is a consideration. A 2021 study by Student Aid Policy showed that admissions tests discriminate against minority and low-income students at selective colleges. 

“[The rankings are] so flawed, and it really does perpetuate a system of inequities,” Bauer-Wolf said.

Third Way, a Washington, D.C.-based public policy think tank, offers an Economic Mobility Index that evaluates how well universities serve low-income students and the proportion of low- and moderate-income students. 

Burns hopes more college rankings will better reflect underrepresented groups, such as first-generation college students, in years to come. 

She thinks college rankings could pivot in the next couple of years to highlighting which schools are providing the most social mobility.

“What are the schools that are enabling a student to move from the bottom of the income level to not being on food stamps and not being on government assistance?” Burns said. “Then the question becomes, could that possibly be measured? Maybe through the percentage of Pell Grants, percentage of financial aid or salary after graduation.”