If Alison Lundergan Grimes pulls off an upset victory over Senate Minority Leader Mitch McConnell in November, she may have Tea Partiers to thank. But don’t expect conservative voters to rally to Grimes—they simply aren't getting behind McConnell.

Although the five-term Republican ended up winning a a tough primary challenge from businessman Matt Bevin handily, the race was one of the most vicious and ugly this cycle, with Bevin’s campaign badly damaged in a cockfighting scandal. And while some disaffected conservatives have backed McConnell in November, Bevin has still not endorsed him, and many of the Tea Party groups that supported the businessman are following suit. As a result, in a recent poll, nearly 20% of self-identified Republicans didn't support McConnell

Scott Hofstra, spokesman for the United Kentucky Tea Party, said he isn’t backing McConnell and “taking the lesser of the two evils approach.” Many conservatives, Hofstra predicted, either will leave the box on the ballot for U.S. Senate empty or will vote for David Patterson, the libertarian candidate. The senator has alienated many Tea Partiers and has yet to reach out to bridge the gap, Hofstra said. The divide was opened further, he added, by McConnell’s open support for Thad Cochran in the Mississippi Senate runoff and McConnell’s association with pro-Cochran ads that many conservatives assailed as race-baiting. “If there were some people on the fence after what happened in Kentucky, the Mississippi incident really put them over the edge,” Hofstra said. Still, he noted that Tea Party dissatisfaction with McConnell wasn’t winning Grimes their votes. “I haven’t talked to anybody who would vote her.”

Andrew Schachtner, president of the Louisville Tea Party and a former Bevin campaign staffer, sounded somewhat more restrained than Hofstra in his comments to The Daily Beast. He said his group “was focusing on state and local issues” instead of the Senate race. In particular, he said, the Louisville Tea Party was prioritizing a local state house candidate to help Republicans gain control of the Kentucky House of Representatives. As for the McConnell-Grimes race, Schachtner said he had decided whom he would vote for but declined to disclose that candidate’s identity.

It’s not just Grimes who stands to benefit from Tea Party doubts about McConnell. Patterson, a policeman from the central Kentucky town of Harrodsburg, is seeking to get on the ballot as the Libertarian candidate. A recent poll put Patterson at 7 percent in the race and poised to be a spoiler. In an interview with The Daily Beast, Patterson said he felt confident that his team would get the necessary signatures and noted that he was already preparing in advance for any legal challenge.

But Patterson said that at the moment he is not directly appealing to disaffected Tea Partiers. While he said his campaign “will definitely appeal to those…who are absolutely fed up with the way McConnell has been voting in D.C.,” the Libertarian was aiming to attract those voters who feel “disenfranchised” by the two-party system. While he took pains to say he wasn’t “modeling his campaign after anybody,” he mentioned Gary Johnson’s presidential campaign as a possible inspiration: “more fiscally conservative than the Republicans and more socially accepting than the Democrats.”

Yet despite lingering anger over the primary and the potential appeal of Patterson, some Tea Partiers are rallying around McConnell. Frank Simon of the Tea Party of Kentucky said he is “very much for McConnell over Grimes…Politics is always the lesser of two evils. McConnell is always much better than Grimes. Grimes is pro-abortion.” But Simon said he couldn’t “speak for all the Tea Parties.” Instead, his group’s approach is to try “to make friends with people, because that’s the only way we’re going to survive,” he said. “We’re trying to overlook problems that we had before and trying to pull together to survive November.”

Simon said the bulk of his group’s efforts would involve a voter guide that was put out “not under this organization” but through a political action committee and another organization, which would go out to about 30,000 people. He told The Daily Beast that “unless something unforeseen happens, it will endorse McConnell over Grimes.” The bulk of the voter guides will go out through churches, which is allowed, he said, because it “also gives the pro-abortion endorsements, pro-homosexual endorsements, pro-labor endorsements.”

Although McConnell has had a slight edge over Grimes in recent polls, most political observers consider the race a tossup, and it looks likely to be tight through Election Day.