U.S. News

03.24.13

How to Reinvent College Rankings: Show the Data Students Need Most

All rankings are misleading and biased. But they’re also the only way to pick a school. So let’s fix them. Steve Cohen on how schools can help make college lists that matter.

“An Ivy League degree changes everything.“

I’ve heard those exact words dozens of times and inferred their sentiment hundreds more. They undoubtedly were a major contributing factor in the 250,000 applications to the eight Ivies this past year. With only 14,000 freshman slots available, there will be a lot of disappointed families when decisions are announced in a few days.

Unfortunately, I’ve helped perpetuate that perception. And I contributed to making college admissions more competitive. For 30 years, I’ve co-authored bestselling books and provocative articles about how to improve one’s chances of being accepted at a “top” college.

The first edition of our book Getting In! revealed what went on behind the admission committees’ closed doors, and introduced the concepts of packaging and positioning to the college-application lexicon. The newest edition adapts the same principles to the digital age and urges kids to scrub their Facebook pages. But the core message remains: good colleges are not looking for the well-rounded kid—they’re looking to put together the well-rounded class, and you need a hook.

What were revelations in 1983 are common knowledge today—at least among college-bound students, parents, and counselors. They also don’t have to be told that the odds of getting into a “highly selective” school are ridiculously low. Brown and Dartmouth will each accept about 9 percent of applicants; Cornell, Northwestern, and Georgetown about 16 percent. And Harvard, Yale, and Stanford?  Forget about it: less than 7 percent!

Wanting to attend a “name” school isn’t illogical. With the sticker price of attending any of these private universities nudging $250,000, parents are heeding college administrators’ advice that college should not be looked at as an expense, but rather as an investment. And there is nothing illogical in parents wanting a better return on their investment. A college’s brand value—whether that school’s name will be recognized and open employers’ doors—is a reasonable measure of ROI. (An Ivy League degree may not guarantee someone success or even a job. But make no mistake: it does open doors more quickly for an interview.)

What is missing from this calculus is that colleges are very different.  Dartmouth is not at all like Brown; Princeton is completely different from Yale; and Georgetown shouldn’t be discussed in the same breath as Northwestern. Colleges, counselors, and parents talk a lot about finding the right “fit” between a school and a student. In reality, the process is dominated by reputation.

The problem is that college reputations have been hijacked by rankings. Far too many “highly ranked” colleges are gaming the rankings and trying to attract more and more applicants—when the particular college is actually a poor “fit” for many of the kids applying. Colleges want to attract and reject more kids because, perversely, that “selectivity” improves the institution’s ranking.

College presidents publicly bemoan the plethora of college rankings.  Privately, they admit they have to provide the data that feed that maw. They can’t afford to be left off a rankings list. And more than a few colleges have been caught fudging the data to improve their rankings.

The real losers in this system are students and their parents. A bad fit is costly, not just in dollars, but in time, energy, and psychological well-being. My co-authors Paulo de Oliveira and Mike Muska started out as senior admission officers and coaches at Brown, Oberlin, Auburn, and Northwestern. We’ve subsequently counseled thousands of families in the college search and admissions process.  And all too often we’ve seen the same pattern: parents come in with a set of “ranked” colleges and want to spend 90 percent of their effort figuring out how to get Sam admitted.

That’s backward. The emphasis should be on finding the right fit.  But finding the right fit is not easy. Subjective guidebooks like Edward Fiske’s—originally titled The New York Times Selective Guide to Colleges—are very useful and consciously do not include rankings. Ted (who is a friend and occasional collaborator) changed his three-category rating system to make it more difficult to simply add “stars” and rank-list colleges. Even families who can afford to visit lots of colleges and endure the backward-walking tour guides’ patter find that campus personalities soon blur in their memory.

Thus it is not surprising that anxious, busy parents turn to rankings for shorthand solace. Unfortunately, the data that U.S. News and other media companies are collecting are largely irrelevant. As a result, the rankings they generate are not meaningless, just misleading.

The data that U.S. News and other media companies are collecting are largely irrelevant. As a result, the rankings they generate are not meaningless, just misleading.

Some examples: U.S. News places a good deal of emphasis on the percentage of faculty who hold a “terminal degree”—typically a Ph.D. Unfortunately, a terminal degree does not correlate in any way with whether that professor is a good teacher. It also doesn’t improve that professor’s accessibility to students. In fact, there is usually an inverse correlation: the more senior the professor, the less time they have for undergraduates.

U.S. News’ second most heavily weighted factor—after a college’s six-year graduation rate—is a peer assessment of colleges by college presidents, provosts, and admissions deans. You read that right: administrators are asked to evaluate colleges that are competitive with their own school. If not an outright conflict of interest, this measure is highly suspect.

Even some seemingly reasonable “inputs” are often meaningless.  U.S. News heavily weights the number of classes with fewer than 20 students. But small classes are like comfort food: it is what high-school kids are familiar with. They have never sat in a large lecture hall with a spellbinding orator. So it is not something they could envision or value. And while small seminars are sometimes enormously valuable educational experiences, the very best class I took at Brown had more than 150 students in it.

While most rankings suffer from major problems in criteria and inputs, the biggest problem is simpler: all the ranking systems use weightings that reflect the editors’ personal biases. Very simply, some editors’ priorities are undoubtedly going be different from what is important to me. Assuredly, my preferences are different from my kids’. And both will differ markedly from our neighbors’ objectives.

Our research suggests there are four sets of questions that parents want to ask, but have trouble getting answers to:

·      Is this college going help me succeed in life? Is it going to give me a body of knowledge, tools, self-discipline?

·      Am I going to be happy at this place? Is it safe? Will I fit in, make friends, and perhaps find a mate?

·      Will this school’s reputation help open doors to those first few jobs?

·      Can my family afford it?

Colleges say they truly want to attract kids for whom the school will be a good fit. To make good on that promise, colleges need to provide families with insight, not just information; and they need to focus on outputs, not just inputs. Collecting and sharing four sets of very different data would be a good start:

1.     Better insight into the quality of education a student will get on that campus. Colleges need to share the exam scores for all students applying to medical school, law school, business school, and graduate programs. These tests reflect not just the innate ability of the kids who’ve gone to that college, but what they’ve learned in the three-plus years they’ve attended. The colleges also should report full results of medical-law-business-grad school admissions.

2.     Colleges need to assess a campus’ “happiness” coefficient. A happy campus is a more productive learning environment; and one that has a lower incidence of alcohol and drug abuse.

3.     The full debt—or savings depletion—that families incur; not just student debt.

4.     The salaries of graduates one, five, and 10 years after graduation.

A fifth useful metric is what employers—both nationally and regionally—think of graduates from particular colleges. Hiring preferences are a useful proxy for reputation.

The last piece in enabling families to find a better fit will come from entrepreneurs. Some smart “kid” will develop an online tool that will allow students and parents to take this new college-reported data and assign weighting factors to the characteristics that are important to them. The tool would then generate a customized ranking of colleges that reflects the family’s priorities—not some editor’s.

Colleges may complain about the rankings, but they are complicit in perpetuating them. It is reminiscent of the classic Claude Raines line in Casablanca: “I’m shocked, shocked to find that gambling is going on in here!” If colleges really want kids for whom their college is a good fit, they will collect and publish the types of honest data that will give families a better basis for smart decisions.