(Also read at The Wrap: "Cracking Open Ken Starr, Hollywood's Mini-Madoff.")

Desperate to help his girlfriend, Diane Passage, break into the music business, alleged Ponzi-schemer Kenneth Starr brought her to a 2007 party at Denise Rich’s multimillion-dollar New York penthouse, overlooking Central Park. Rich, most famous for lobbying then-President Bill Clinton for a pardon of her ex-husband, fugitive financier Marc Rich, had recently built the most expensive personal recording studio in the city, a $1.1 million monument to music-industry excess.

According to a source in attendance, Rich asked one of her male associates to bring Passage down for a look at the state-of-the-art soundboards, amplifiers, and other toys. They were barely alone for 10 minutes before Starr charged downstairs wondering what was taking so long.

“News of Ken Starr’s arrest does not come as a complete surprise to me,” says Annie Leibovitz.

“He didn’t say much, but he got up real close and put his arm around her,” says this source. “You could tell that he was jealous about her being alone with another man.”

The scene, according to more than a half dozen of Starr’s associates, offers a glimpse into the subconscious insecurity that gripped the financial adviser to boldface Hollywood names such as Warren Beatty, Uma Thurman, Mike Nichols, Jim Wiatt, and others after he divorced his multiple sclerosis-stricken first wife for Passage, the creator of something called “Pole Dance Superstar.”

• Jacob Bernstein: How Uma Sniffed Out a Ponzi Scheme• Allan Dodds Frank: Madoff vs. Starr—Who’s the Real Ponzi King? Indeed, though sources say Starr, 65, wasn’t shy about showing friends pictures of his scantily clad girlfriend—eventually, his wife—demonstrating her technique, they also say his self-assuredness masked a constant worry. Specifically, that Passage, three decades his junior, was out of his league, driving him to allegedly bilk his clients out of $30 million—some of which he used to buy the $7.6 million, five-bedroom, six-and-a-half-bath apartment, complete with a pool and garden, where he was arrested yesterday.

Evidence to support this theory can be gleaned directly from the criminal complaint against Starr, which also includes former New York City Council President Andrew Stein (the U.S. Securities and Exchange Commission filed a separate civil complaint yesterday against Starr, Stein, Passage, and an entity dubbed Colcave LLC, as well).

According to the indictment, the fraud perpetrated by Starr, who managed more than $700 million through his Starr & Company and Starr Investment Advisors firms, began in January 2008 and lasted through April of this year. That timeframe dovetails with Starr’s marriage to Passage at the Wynn Hotel in Las Vegas in June 2007, and a subsequent huge party the couple threw in Central Park. Moreover, many of the items bought by Starr and detailed in the indictment—including a $70,000 diamond bracelet, $16,000 ladies' round brilliant cut diamond chandelier earrings, and $32,000 ladies' two-row diamond wedding band—have the distinct whiff of overcompensation.

“It’s hard to believe that the Ken Starr I know has been indicted for anything,” says noted entertainment lawyer Bert Fields, who previously represented Starr and sat on a few boards with him. “I heard that in the last few years he had problems, but I can’t tell you when they started and this is the first time I’m hearing of any criminal accusations. I’m truly sorry that things got to this point.”

While the media has been quick to liken Starr to Bernie Madoff, the most infamous Ponzi-schemer of all-time, the money laundering, wire fraud, and investment adviser fraud charges against him were much smaller in scope. For those in Hollywood, Starr brings back uncomfortable memories of Dana Giacchetto, the money manager who served three years of a five-year sentence in the early part of the decade for defrauding Tobey Maguire, Leonardo DiCaprio, Michael Ovitz, and the jam band Phish, among others.

“This is what everyone around here fears, a guy who seems too good to be true and turns out to be,” says one high-level Hollywood agent.

Starr’s current position—in a Manhattan jail cell—is worlds away from the phenomenal success he enjoyed at the beginning of his career. Starr basically built his business on the back of Listerine heiress Bunny Mellon (at 99 years of age, Mellon is having a very bad year: She indirectly helped fund John Edwards’ dalliance and coverup with Rielle Hunter, and is also believed to be one of the clients Starr allegedly bilked). No less an authority than Pete Peterson, the legendary investment banker, co-founder of The Blackstone Group and one of America’s richest men, used to swear by him, according to a source. Much of Starr’s credibility came from Peterson’s endorsement—he would frequently tell prospective clients to “check me out with Peterson” to put them at ease and get them to open their wallets, this source says.

Peterson did not return a call for comment.

Starr has also been a regular attendee at Allen & Co.’s annual mogulfest in Sun Valley, though no one is quite sure how. Each year an elite group of around 250 moguls from the worlds of entertainment, technology, and finance—a list that typically includes such notables as Edgar Bronfman Jr., Michael Eisner, Larry Page and Sergey Brin, to name a few—descend on the sleepy Idaho town for a week of schmoozing and dealmaking. Starr has made the cut more than five years in a row, but sources who attend the conference say they aren’t aware of precisely who at Allen & Co. invites him.

“I’m not sure why he’s there,” says one Sun Valley regular who requested anonymity for fear of being banned from the ultra-secretive conference. “He cloaks himself in so much respectability and has clients of a certain import, so I… figured he was legitimate.”

So did the roughly 200 clients, including celebrities like Wesley Snipes, Annie Leibovitz, and Sylvester Stallone, who received investment advice or business-management services from Starr. Recently, however, as rumors about Starr spread throughout Hollywood, Beatty, Nichols, Leibovitz and others began pulling their money.

“News of Ken Starr’s arrest does not come as a complete surprise to me, and I will follow this story with great interest,” says Leibovitz, who nearly filed for bankruptcy after being introduced to a lending firm called Arts Capital, via Starr. “Ken Starr no longer represents me and has not for some time.”

Perhaps the biggest beneficiary of Starr’s downfall is Wesley Snipes. In an ironic twist, Starr, who was the key witness that helped the government convict Snipes of misdemeanor tax evasion, is now being accused of lying to the authorities and stealing from his clients. Snipes’ lawyers accused Starr of fraud during the trial, but the government took the side of the financier over the celebrity.

“Now we know the lead witness against Snipes, Mr. Starr, was lying all the time and is a perjurer,” says Robert Barnes, the Bernhoft law firm attorney who represented Snipes. “If the government has any honor, they will move to dismiss all charges against Snipes immediately.”

Not everyone was as lucky as Leibovitz and Snipes, however. Sources say Starr’s victims include former NBA star Julius “Dr. J” Erving and Jacob Arabov, better known as bling peddler “Jacob the Jeweler.” According to the complaint, a person believed to be Jacob the Jeweler invested close to $14 million with Starr, and “eventually came to consider him like a brother,” meeting as frequently as once per week.

While Starr’s arrest was shocking, he had exhibited questionable behavior before. A few years ago, Starr, at the urging of his then-lawyer Fields, enlisted infamous wiretapper Anthony Pellicano to dig up dirt on Stallone, who sued his former business manager in a case that was eventually settled out of court.

For all his troubles, Starr did eventually help his new wife break into the music business. Starr helpfully funneled more than a million dollars from Jacob the Jeweler, his wife, Stein and others into Glassnote Records, an independent record label run by former EMI Records and Artemis Records head Daniel Glass, who did not return a call for comment. Passage, according to the indictment, owns 14.4 percent of Glassnote and gets paid $150,000 to serve as a consultant to the label, though she “has no real responsibility,” says a source familiar with the arrangement, “and showed up once per month.”

Correction: Hoda Kotb was initially included in the photo for this article because of a photo caption error.

Plus: Check out more of the latest entertainment, fashion, and culture coverage on Sexy Beast—photos, videos, features, and Tweets.

Peter Lauria is senior correspondent covering business, media, and entertainment for The Daily Beast. He previously covered music, movies, television, cable, radio, and corporate media as a business reporter for The New York Post. His work has also appeared in Avenue, Blender, Black Men, and Media Magazine, and he's appeared on CNBC, Bloomberg, BBC Radio, and Reuters TV.