TOKYO, Japan — It’s not easy being a man who calls himself a “feminist” in a political party of heckling sexist pigs. Japanese Prime Minister Shinzo Abe claims he wants to bridge Japan’s huge gender gap. But some infamous jeers by Abe’s underlings are turning to boos and hisses for his leadership.

As previously reported in The Daily Beast, sexist taunts made by male members of the Tokyo Metropolitan Assembly have sparked protest throughout Japan. The public is demanding the resignation of the members who called out, “Hey you, should hurry up and get married!” and “Can’t you have babies?” while Your Party member Ayaka Shiomura was speaking on issues of raising children and giving birth in Japan.

The heckling and slurs were all coming from the seats of the Liberal Democratic Party (LDP) faction, which is the ruling party in the Tokyo Assembly and in the nation. Prime Minister Abe is not only the leader of Japan, he is also the Director-General—the sosai —of the LDP.

On Monday, LDP Assemblyman Akihiro Suzuki came forward and admitted that he was one of the assembly members who yelled out sexist comments during Shiomura’s speech. He had previously denied it on national television.

Suzuki publicly apologized and bowed to Shiomura, saying, “The low birth rate and tendency to marry later is a problem, so I wanted women to hurry up and marry and I said that. I deeply regret that I did not give enough consideration to women who want to marry but can't,” he said, according to the Asahi Shimbun.

Public reaction to the news has been massive. Every major newspaper and television station in Japan is covering the story. One unknown angry citizen threw 20 raw eggs at Suzuki’s office. The Tokyo Police Department are investigating whether there is any connection to the egg assault and his heckling. The police managed to seize an empty egg carton left behind at the scene; they already have a male suspect in custody, they say.

On Tuesday, Ms. Shiomura held a press conference at the Foreign Correspondents’ Club of Japan and discussed the sexual harassment scandal. “Because it took so many days [for Mr. Suzuki to come forward], my feelings are complicated,” she said. Not everyone responsible has come forward, she noted, then added, graciously, “I realize that the longer one waits in a situation like this to come forward, the more courage it takes to actually raise your hand and admit what you have done. In that sense I would like to acknowledge the courage it took for [Suzuki] to do that.”

Shiomura’s bid to have the people who allegedly harassed her reprimanded was rejected by the Tokyo Metropolitan Assembly because she could not name the people who made the taunts.

“It’s a shame, but I felt as if they did not want to make the problem big,” said Shiomura. “I got the impression that they wanted to make it seem as if there was no problem.”

Prime Minister Shinzo Abe, as the head of the LDP, reportedly visited Your Party leader Keiichiro Asao and apologized for the incident. That may not be enough.

“Abe doesn’t realize that he needs to speak to the nation, not just to Shiomura or Your Party in particular, but that’s common of everyone else involved in the story,” said Koichi Nakano, a professor of political science at Sophia University.

It may be no coincidence that on the same day Shiomura spoke to the foreign press about the incident, the official blog for “The Association to Support Shining Women” was launched with a written entry from Shinzo Abe himself. The Japanese government created the association on March 28 this year.

In his entry, Abe wrote: “The government will enthusiastically support women who are working and those who are in the home so that they can utilize their experiences and play an active role in society.”

Right. Abe is in a tough situation. He has argued repeatedly for what he calls womenomics, getting women out of the house and into the workforce. But the example set by members of his own party puts the lie to such liberal-sounding propositions.

For a man who can’t seem to apologize coherently on other issues, this case is a nightmare. His waffling on apologizing to Korean women who were sex workers in wartime Japan has enraged many. It is possible that apologizing to a woman—even if she is a Japanese woman, younger than himself, and a member of the opposition—may be more than he can stomach. Judging by the hastily erected web page, he’s already feeling queasy.

A representative from the Cabinet Public Relations Office told The Daily Beast in a phone call, “We are in no position to comment on this issue.” But as the president of the LDP, Abe is in a position where he must say something. In the highly vertical society that is Japan, the buck stops with him. And, of course, Ms. Shiomura knows that. “I would like to hear a comment from Prime Minister Abe,” said Shiomura. “I want to hear his opinion.”

In the LDP, which was founded with yakuza money and mirrors yakuza societal structure, the top dog, aka the sosai, has to take responsibility for the hoodlums underneath him and they must acknowledge their errors. Suzuki himself clearly has a good idea of the yakuza mindset—he belonged to an ultranationalist group that works with Nihon Seinensha, the violent political arm of the yakuza group, the Sumiyoshi-kai. Nihon Seinensha has been widely praised by former LDP bigwig and previous Tokyo Governor Shintaro Ishihara, famous for his remarks that “women who can’t reproduce are useless and a crime.” While we’re on the subject of crime—the yakuza are certainly the most sexist group in Japan—they have no female leaders or executives at all.

So Prime Minister Abe can either grit his teeth and apologize for the rudeness of his men or lose some digits—in the sense of approval ratings rather than finger chopping. How he handles the situation will either make him beloved by Japan’s “shining women” or could cast an ominous shadow over his political fortunes.