“Economic diversity” has entered the language as a standard way to talk about inequality in higher education. The term usually refers to the share of students at a college who come from lower-income families. It’s an important issue, but “diversity” is the wrong way to talk about it. Quibbling over words may seem trivial, but getting this issue right is important both for addressing inequality and for improving colleges and universities.

“Diversity” suggests many kinds of faces, smiling together. It implies we are all basically equal, only different. The goal of diversity is to get different life histories, experiences, and perspectives into the same classroom, dorm, or, for that matter, corporate boardroom or officers’ barracks. Although race is still arguably the paradigm of diversity policy, schools and other institutions also aim at gender diversity (including diversity in sexual orientation), geographic diversity, ideological diversity, and so on. It’s how Americans talk about whether our institutions are fair representatives of the country.

When I was admitted to Harvard, the Pell Grant-eligible son of a carpenter from West Virginia, I’m sure I counted on someone’s list of economic (and geographic) diversity wins. And in fact, I had learned different things working summers with the building crew than some of my classmates had learned at the club, the firm, or the bank. I knew what it meant to have no health care, and how it felt to get up at 5:30 in the morning with a day of grimy sweat ahead of you—which is totally different from any kind of nice, clean sweat, and not romantic at all.

I felt proud at how much my dad’s crew respected him: He wasn’t a boss or foreman, and wouldn’t have wanted to be, but after a while he was the senior man on the crew, and he had good judgment and a gentle manner. I brought all of that experience into my college classrooms, along with other stuff—a scholarship to Exeter, my mother’s master’s degrees—that was not working-class at all. I guess that was diversity, and I guess it was fine for Harvard to want it. But as a model of economic fairness, or of the role of universities in social mobility, it is unsatisfying.

All this diversity is a good thing, as far as it goes, but it’s also an awkward compromise, the bruised fruit of a dream deferred that goes back to the civil rights movement. The dream was of a much deeper equality than we’ve achieved.

That struggle against white supremacy aimed at three goals. One was formally equal treatment: an end to segregation and other laws and social practices (like exclusive clubs) that made different rules for different people. The second was dignity and inclusion: an end to the culture of humiliation and invisibility that came along with white supremacy and helped to uphold it. Third was economic redress and opportunity: a full chance to join an economy that built up advantages for white people over nearly two centuries. These advantages don’t go away just because segregation ends, and they persist even among people who pride themselves on showing, and feeling, equal respect for everyone.

Promoting diversity focuses on the second goal, dignity and inclusion. It’s an acknowledgement that formally equal treatment isn’t enough because there are so many informal barriers in the way of the disadvantaged. But it stops short of advancing economic redress and opportunity.

The official theory of diversity, as embraced by the Supreme Court, public and private universities, the military, and the private sector, is that it (1) benefits everyone in those institutions by exposing them to unfamiliar viewpoints; (2) makes the institutions responsive to diverse experiences, which makes them work better—whether at boot camp or in marketing meetings; and (3) makes the institutions more legitimate in the public eye by ensuring that they represent many kinds of people. (The Army is disproportionately black and Latino; Cal-Berkeley is disproportionately white and Asian; if they were much more so, people might begin to wonder whether we are a de facto segregated society. They might lose faith.)

Of course promoting diversity often has the side effect of creating mobility for economically disadvantaged groups, but that isn’t the primary or official goal. In fact, the Supreme Court has said that it can’t be the goal, at least where racial diversity is the issue. There’s a serious debate about whether diversity is a worthy aim or window dressing. People on both the left and right dismiss it as elite liberal hypocrisy. The left wants to do more about economic inequality, the right less—often, as with Supreme Court Justice Clarence Thomas, because it sees anti-poverty efforts as counter-productive.

But “economic diversity” is a square peg in a round hole, no matter how you feel about round holes. Why? Well, consider this. Going to college or medical school is supposed to make a working-class student stop being working class. If it succeeds, that’s what it does. No other branch of diversity policy has this goal. Bringing difference into institutions to change the institutions—at least a little—is what diversity is supposed to do. “Economic diversity,” by contrast, brings economic difference into higher education in order to liquidate the difference.

There’s a good reason for this. Unlike diversity, access to higher education for the poor and working class is straight-up aimed at economic fairness and opportunity. It may—probably will—have the side effect of bringing different perspectives to the classroom, but that’s not the point. In a time when economic mobility has slowed, inequality is growing, and average wages and family incomes are stagnating or falling, economic opportunity isn’t just about whether schools are representative; it’s about whether the country’s economy produces anything resembling fair opportunity.

This is also why talking about whether Harvard or the University of Wisconsin does a better job of creating “economic diversity” is beside the point. This isn’t about the good offices of elite institutions, because it’s a much bigger problem than university policy can address. Who goes to college ultimately depends on how the country funds and runs not just higher education but the public elementary and high schools. And if it’s economic mobility and fairness we’re worried about, we’ll also have to think about higher minimum wages, making Obamacare work (and improving it), finding a way to bring unions back to life—the kinds of policies that helped build a middle-class society in the 20th century.

Knocking—or praising—schools for their share of Pell Grant recipients really is rearranging the deck chairs here. We need to talk about economic fairness, even justice—an antique-sounding but still important word—not economic diversity. Unless everyone has health care, security, a good retirement, and dignity in their job, people will come to college to escape working-class life, not to represent it, and the idea of diversity between the rich and the poor, the privileged and unprivileged, will be something worse than a joke: a confusion.

This is an argument from the left, but it has a conservative twist. We ask schools, at all levels, to do too much in this country: to make up for deep-seated inequality, to sort children and young adults meritocratically for the market economy, to be more representative and unequal than our divided and unequal society, and, while they’re at it, to pass along precious reserves of memory, beauty, and critical thought that are valuable for their own sakes. Frustration at schools that are trying to do all these things at once spurs periodic fits of bureaucratic intrusion, like the Common Core and the Obama administration’s initiative to rank the country’s colleges and universities along a return-on-investment scale. These only make it harder for education to play its subtle, humanistic, neither-liberal-nor-conservative-but-both role of giving people a space to learn things of no obvious value, because they are strange and interesting, because their value will emerge years later, in the most surprising ways.

This may sound like a luxury, and it is, but it’s also a necessity for a certain kind of civilization. We are more likely to be able to cultivate it in our schools, at all levels, if the rest of the country is reasonably fair, if the economy is working as it should, and if people’s lives are going well. Right now, we’re losing our grip on all these good things at once, and talking about economic diversity is more a symptom than a cure.

I would love to live in an economy where the idea of economic diversity made sense. I would love to see representatives of an honored and secure working class take their place at elite institutions. But that’s not our world. When it comes to economic class, justice and power will do a lot more than diversity to build that fairer world.