As Ruth Madoff swept into Palm Beach last month with a quintet of girlfriends, her $7,500 Birkin bag dangling, her husband’s 74-year-old sister, who was ruined by Bernie’s scam, was watering plants and driving people to the airport just to make ends meet.

Sondra Wiener, forced to make pocket money like an out-of-work laborer, endures the pity of her neighbors. After her brother's scheme collapsed, she also put her home in a gated community outside Palm Beach up for sale. Her brother mailed her and other family members Cartier, Tiffany, and other expensive jewelry in December (which violated a court order and were repossessed), but sources say they do not think her sister-in-law Ruth has given her money. And Ruth is believed to have plenty, even now. Although authorities have seized her property and bank accounts—most recently the Palm Beach home and the antique yacht restored by Bernie—investigators describe caches of laundered funds hidden around the world in Ruth’s name.

When Carl Shapiro heard of Bernie’s arrest, friends thought that the 96-year-old was having a coronary. But this Sunday, he looks surprisingly complacent.

During a recent visit to Palm Beach, I found that the ethos of this balmy, nonchalant Florida retreat for the elite had been disturbed. The palms lining the streets slap menacingly in the wind, luxurious Worth Avenue is eerily empty, abandoned by the economy and the evaporation of capital into Bernie's fantasy bourse. Madoff's victims, in addition to everything else, are spooked. They don’t want their names known because they are afraid of being called before a grand jury or afraid of being called to give back profits even though they sustained losses. Some are even afraid of being called on by the Russian mob, which allegedly was scammed too.

The name Madoff inspires an eerie silence. In particular, the very mention of Ruth seems to arouse fear. Those few who knew her well won't talk about her, those who knew her less well will only talk anonymously.

“Everyone down here thinks she was involved in the Ponzi scheme," said a leading socialite in Palm Beach. "She and Bernie were always collaborators. Look, let me tell you what happened to my friend. Her husband wouldn’t let her invest with the Madoffs. Then after her husband died, Bernie wasn’t taking any new investors. She talked to Ruth who said give Bernie a call and she told him 'Ruth knows what this is all about' and Bernie simply said 'OK, I'll take your investment.' And this woman gave him the store and rued the day she had met Ruth Madoff."

"She came into town briefly, I think it was a couple of weeks before her husband was jailed," said one shopkeeper in Palm Beach's most famous commercial street. "There were five friends with her and they were making their way down Worth Avenue. She had this huge Hermes Birkin bag floating around—you have to get on a two year waiting list for one of those you know." Another observer added that the group browsed through boutiques like CJ Laing and Angela Moore and had lunch at an Old World Old Money restaurant called Renato's.

"We thought it was odd," the shopkeeper said. "Where did she get these girlfriends? We had never seen her before without Bernie." She reportedly stayed at the famous Breakers Hotel—where Bernie has in the past rented a cabana—perhaps because she had word that authorities would seize her multimillion dollar waterfront home and did not want to be in it when it happened. The wood shuttered balconied mansion, plus two boats and the yacht, were actually seized several days ago, the locks changed, and Ruth Madoff banned forever.

The Madoffs, for all their extravagant possessions, did not aspire to be part of The Crowd in Palm Beach. They did not go to most charity balls and parties, and simply refused, except on the occasion of Carl Shapiro's birthdays, to go to Colette's, the eating club of the social high-hatters. "I don’t think we even have a picture of them," said a reporter at the Palm Beach Daily News, the society newspaper they call The Shiny Sheet.

Instead, the Madoffs liked to go to movies and attend movie-club discussions. "Ruth is really smart. You don’t get into Queens College and then get a master's if you aren’t," said an acquaintance.

Though they could afford any restaurant in town, the couple favored informal bistros and grills, like Cucina's where they went three or more times a week. There, when owner Nick Coniglio stopped to talk to them, Ruth became "kind of withdrawn. She wasn't pushy, she didn’t wear a lot of ostentatious jewelry, She let Bernie do the talking, you know, the traditional supportive wife." From afar, he noticed how much they talked to each other. "She had this look of authority. She sat there smoking cigarettes, whispering to Bernie."

Added a close friend: “Bernie never drank and Ruth had just a little wine. They were just antisocial."

One Palm Beach entrepreneur who lost money to Madoff once had the rare experience of seeing them at a yacht party. "They were standing in the corner, I went up and made small talk, something about how I admired what he was wearing. He walked away and she followed. We asked them to three dinner parties; they accepted two and then didn’t show up at either of them."

“Ruth is a mousy little woman, really,” said one prominent political figure in town. But another woman remembered, “she wore these little cashmere sweater sets that must have cost $1,000. And she had her hair done at the fanciest salon, Frédéric Fekkai, even though it was just the simplest little blond bob."

So who did she dress for? She did not strut forth like many Palm Beach women, who make up the bulk of the population and who admit they dress for each other. “She was all for Bernie,” said one hostess on the party scene. "She was very conscious of him. Very watchful.”

The Palm Beach Country Club is so exclusive that initiation fees are more than $300,000; applicants must go on a waiting list and meanwhile prove that they have given millions of dollars mostly to Jewish charities. The club was originally established in answer to the WASP clubs around town that ardently kept Jews out. It is where Bernie had his small circle of friends, particularly Robert Jaffe, solicit people for his fraudulent funds.

It is estimated that one-third of the club had invested. "The day after it all exploded, they came in stunned, crying," said one member.

But most of the other two-thirds said, "Bernie who?” They hadn’t heard of him, although he had been a member since 1994. "I was on the board of Nasdaq around the same time as Madoff,” said Gene Isenberg, a prominent businessman, “and I never met him at all.”

Having brunch with friends at the club, I can almost see Ruth and Bernie, sitting in back at a table for two, scouting the terrain like foxes. They have played golf, as they always do, by themselves. When they wish to eat, they go up to the buffet table in unison.

I watch the members. Impeccably dressed, with flawless manners, float by me, their plates piled high with salads, talking quietly so as not to disturb another table’s conversation. I notice that the diners are mostly older people, the men wearing khakis or linen shorts in red or tangerine, the women in gold sandals and splashy sundresses. The club rooms overlook the shivering palms and brilliant green of the golf course, but hardly anyone is on it. "That is what happens," laughs Isenberg. "Forget golf, we like to eat more than anything else."

The Madoffs' closest friends are there: Carl Shapiro, the philanthropist who thought of Bernie as a son. Bernie would give the kick-off toast at Carl's birthday party every five years, thanking him for all he had done for him over the years. Indeed, two weeks before his Ponzi scheme crashed, Carl sent him $250 million. When he heard the news of Bernie’s arrest, friends thought that the 96-year-old was having a coronary. They said that he felt it was like “a knife in my heart.” But this Sunday, in the club's luxurious surroundings, he looks surprisingly complacent. Tanned, freckled, dressed in beige linen pants and a camel cashmere sweater, he smooths back his white hair and heads up a table of friends and relatives. Among them are Carl’s daughter, Ellen Jaffe, the sleeves of her white jacket, elegant with woven black trim. She walks erect, carrying her plate out before her with fragile dignity.

A woman stops by our table and looks at her: “Ellen was so snobbish before all this happened, she wouldn’t even say hi to me. Now her husband is being pursued by the law and she is embarrassed. So of course, she couldn’t be friendlier to me."

Robert Jaffe, Ellen’s husband, is absent from the room. Weeks ago, he had refused to obey a subpoena calling him to testify about Cohmad Securities, the company that shared offices with Madoff Securities, and seemed to be strictly a funnel for client money into Madoff funds. According to a court document, Jaffe and other Cohmad executives received exorbitant commissions for every investment they acquired. Jaffe had frequent business breakfasts with Madoff at Cucina and the two live four doors apart in opulent houses on the water.

Robert Jaffe has not been charged with any crime, but that does not stop people from arguing about his guilt. "They are very social—fun, lovely people and we don't think Bob is capable of this kind of fraud. Now they are treated like lepers and it is a terrible shame," said one member of the club.

But others disagree. At the Palm Beach Grill, another favorite of Bernie’s, with its leather booths, modern three-tiered saucer chandeliers, and two big brass storks at either end of the bar. I met Anka Palitz for dessert. You cannot get into the Palm Beach Grill for an actual dinner unless, according to a regular, you give the maitre d’ an offering of some $200. "That's seed money to guarantee you a good reservation any time, any day, in perpetuity." he explained.

Palitz is a prominent socialite dressed glamorously in red off-the-shoulder chiffon: "Robert Jaffe came up to me and said, 'You got to go to this guy Madoff'. He will get you 11 percent a year. You can’t top that.' Later I found out that this was not just friendly advice from Mr. Jaffe. He would have gotten a big percentage of my investment.

"But I listened to him and I finally said, ‘No, I don't believe you. I don’t believe in things that sound too good to be true.’”

Lucinda Franks is a Pulitzer Prize-winning journalist and author who was on the staff of the New York Times and has written for the New Yorker and the New York Times Book Review and Magazine. Her latest book is My Father's Secret War, about her father, who was a spy for the OSS during World War II.