Like everybody else, Vernon Jordan hears the complaints about President Obama, that he hasn’t done enough for the people who elected him, and that includes black Americans. “And there is some legitimacy to it,” Jordan told The Daily Beast. “But he’s done a lot more in this area of inequality than George Bush, and he [Bush] got reelected.” That seems to be setting the bar rather low, but inequality is an area of the law, and of life, that Jordan knows a lot about. As a newly minted lawyer, he escorted Charlayne Hunter to integrate the University of Georgia, and he headed the Urban League when the civil-rights movement was in full swing.

Jordan cites the Lily Ledbetter Fair Pay Act, the first piece of legislation Obama signed as president, as emblematic of his leadership. “That was about making a concrete difference in an historic inequality in this country, and he got it done.” Jordan believes Obama’s reelection is more important than his election “because this country is more about re-affirmation than affirmation … the fact that the door to the White House and corner offices in America has opened to black people does not mean that all the doors of America are open–and I do believe he is the best person to open the remaining doors.”

When he talks about growing up in the segregated South, Jordan doesn’t dwell on the separate water fountains and colored bathrooms, or riding in the back of streetcars. He remembers how the public housing where he lived was right next to Atlanta’s five historically black colleges, and the inspiration he drew from that proximity. He remembers how he learned to swim, and to box, and to master public speaking at the colored YMCA, how his mother was always the president of the PTA, and how the church was the center of family life. “I was not in this segregated situation saying, ‘Woe is me,’” he says, reflecting on a life begun in the humblest of circumstances that took him to the highest reaches of power, first as a civil rights leader, and then as an advisor and friend to President Bill Clinton, equally loyal during times of scandal and times of triumph.

Jordan recoils when the media describe him as a powerbroker, Washington insider, or worst of all “a fixer,” but he says that by declining an official post in the Clinton administration after leading the president-elect’s transition team, he could be a better friend to the president. “I believe when you raise your hand, you give up a little bit of your independence, and I didn’t want to do that, and I didn’t think I had to do that to serve him, or be a good citizen and serve the country.” The images of Jordan golfing with Clinton during the height of the impeachment effort testified to their friendship even as Jordan was called to testify before the grand jury.

Jordan recalled how a preacher friend, Dr. Gardner Taylor, would call him about ten at night before he was scheduled to testify the next day and say, “Vernon, let us pray … and he would instruct the Lord, ‘I want you to put your long arm around him and prop him up.’” Whether Jordan gave away any secrets, we’ll never know, but looking back on his long career–he’ll be 77 in August–Jordan has seen dramatic change, not the least of which is the election of Barack Obama. His observation that Obama’s reelection would represent an important reaffirmation no doubt draws on his experience as the only black in his class at DePauw University.

When his parents dropped him off in Greencastle, Indiana, his mother slipped $50 into his pocket, and his father shook his hand and said, “You can’t come home.” What do you mean? Jordan asked. Everybody will be up to chapter six while you’re struggling to get out of the preface, but you can’t come home, his father repeated. What to do? “Read, boy, read,” he said. Four years later, Jordan graduates. His mother gives him $100; his father with the same straight serious face says, “You can come home now.” The message: acceptance at a white institution wasn’t enough, he had to prove himself much the way, in his telling, Obama has “earned the right to be reelected.”

Jordan’s portrait hangs in the National Portrait Gallery, where he sat with chief historian Marc Pachter earlier this month for over an hour of questioning that began with, “Tell me about your mother.” Jordan called her “the center of my life.” She addressed him as “Man,” and while his father would have been happy if his son got a steady job and worked in the post office, like he did, that wasn’t enough for his mother. She pushed her oldest son to join the ROTC; she didn’t know anything about it, but she told him the white ladies she baked cakes for in her catering business were sending their boys to it, “There must be something to it,” she declared. Jordan didn’t rebel. “I was taught to mind,” he said.

Jordan briefly considered becoming a preacher, but a summer working as a bus driver for the Chicago Transit Authority opened his eyes to the big city and he withdrew his applications from three divinity schools. A fourth had rejected him, saying in effect, “We teach Jesus but we don’t teach blacks.” It was 1957 and change was coming to the South. Jordan wanted to be a civil-rights lawyer so he enrolled at Howard University law school, the only law school in the country at that time with a course in civil-rights law. Six months after graduating in 1960, Jordan was in the thick of the civil-rights movement, escorting Charlayne Hunter to integrate the University of Georgia.

Jordan worked on voter registration in the South throughout the 60s. He briefly considered running for Congress in Atlanta only to change his mind two weeks later when he got a job offer from the United Negro College Fund. In 1971, he was tapped to succeed Whitney Young at the helm of the National Urban League. He’s only half joking when he says he’s never gotten over losing an election for student body president when he was in 12th grade. He’d been running since 8th grade, and made what he considered the best speech in the history of high school politics, quoting everyone from Plato to Booker T. Washington. His opponent’s slogan: “Vernon wants the job too bad. Edith (his running mate) makes everybody mad.”

His opponent didn’t even make a speech; he played “Begin the Beguine,” and said his mother can’t be president of the PTA (like Vernon’s) because she has to work, the food in the cafeteria is bad (which it was), but he’d gotten the students brand new salt and pepper shakers. Jordan is still smarting from this early lesson in politics. Negative campaigning works, and it’s important to give voters something tangible. “They could put those salt and pepper shakers in their hand,” he says, shaking in head in wonderment. “It killed me.”