By the time Dr. Drew Pinsky’s show was canceled by CNN after he engaged in unfounded speculation about Hillary Clinton’s health, no one was shocked.

For years, the Loveline host had been seen as more of a “television doctor,” as Newt Gingrich recently referred to him, than as a doctor who also appears on TV. Most memorably, Pinsky presided over VH1’s Celebrity Rehab from 2008 to 2012, with five cast members dying while the show was still on the air. A sixth, the WWE star Chyna, died of a drug overdose this April, drawing renewed attention to the questionable ethics of the reality show. But if you buy into the Dr. Drew mythos, CNN’s cancellation of the HLN show Dr. Drew on Call was the last chapter in the gradual unraveling of his career, which started with a beloved radio show and ended in headline-grabbing ignominy. Looking back, however, scandal and controversy have plagued Pinsky for decades.

The story of Dr. Drew was only ever going to end this way.

In 2012, The Daily Beast reported details of a $3 billion settlement between the Justice Department and pharmaceutical giant GlaxoSmithKline (GSK), including a federal complaint which alleged that a PR firm for GSK had hired Pinsky to talk about the antidepressant Wellbutrin as a libido booster “in settings where it did not appear that Dr. Pinsky was speaking for [Wellbutrin].” Court documents included invoices for Pinsky totaling $275,000.

The date on those invoices? 1999, when the TV version of Pinsky’s Loveline had only been on MTV for three years.

Pinsky was not charged with any criminal wrongdoing, as The Daily Beast noted at the time, and the doctor released a statement saying that his comments on Wellbutrin were “consistent with [his] clinical experience.” But the damage to his credibility stuck.

Dr. Ford Vox, writing for The Atlantic, observed that Pinsky’s “minor empire” had been “built on audiences who’ve grown to trust his medical opinion.” He added that the GSK settlement “ought to end his career with CNN.” Others were even less charitable. The health blog Blisstree called him a “hack—even during the Loveline years.”

“Even our happy memories of the celebrity psychologist can now be tainted with the information that Dr. Drew was dishonest from the beginning,” the Blisstree writer mourned.

CNN could have dropped Pinsky back in 2012 when the GSK settlement became public. They didn’t. Instead, an HLN spokesperson told The Daily Beast in July of 2012 that the network was not interested in his activity before his 2010 hiring.

But there were plenty of scandals after Dr. Drew was hired by HLN, too. Later in July of 2012, for example, journalist and NYU professor Charles Seife resurfaced his allegations that Pinsky had taken $115,000 from Janssen Pharmaceuticals in 2010 and 2011. Seife reported for Slate that an HLN spokesperson said: “Dr. Drew would provide an on-air disclaimer if he were to do a story involving Janssen Pharmaceuticals.”

In 2011, too, the FDA warned the Pinsky-endorsed company 1-800-GET-THIN not to downplay the risks of lap-band surgery. Pinsky didn’t respond to the Los Angeles Times report on the FDA warning, citing a confidentiality clause in his agreement.

But Pinsky’s TV career only flourished, as did his nasty “habit” of “diagnosing at a distance,” which would eventually land him in hot water in the 2016 election. As Jezebel reported in 2011, Pinsky defended this habit during the very first episode of his HLN show, which aired in April of that year.

“For those of you who say that I can’t diagnose at a distance, I’ve been practicing medicine for 30 years,” Pinsky said at the top of the show. “I have experienced and have studied hundreds, let’s say, thousands of cases. It’s what I do.”

In that sense, when Pinsky raised eyebrows for saying in a radio interview that he was “gravely concerned” about Hillary Clinton’s health and for claiming that she was suffering from a “brain injury,” HLN was getting exactly what the celebrity physician had promised to do when they hired him. The only difference this time around was that he was distance-diagnosing the potential next President of the United States instead of a D-list celebrity.

Now, many of the major chapters in Dr. Drew’s career are coming to a close. In 2013, he left Celebrity Rehab, telling a radio show that he was “tired of taking all the heat” for the untimely deaths of the show’s cast members. Loveline went on the air for the last time this April. And the final episode of HLN’s Dr. Drew on Call is slated for Sept. 22.

Longtime fans of Loveline still remember a time in the 90s when Pinsky came across as an honest, sex-positive radio host who openly discussed HIV and other STDs during an otherwise stigmatizing time. But the longer Pinsky has careened off course—and the more details have emerged about his earlier years—the shorter that period seems.

And if it’s true that it’s not how you start that matters, but how you finish, then Pinsky’s lucrative downhill slide will be his legacy. He may have started as a disarmingly honest radio show host but he finished as a...well, his most recent credit on IMDB says it all: Pinsky last played a pastor in Sharknado 4: The 4th Awakens.

If there’s anything that Dr. Drew should be “gravely concerned about,” it’s his career.