The most shocking aspect of British director Tony Kaye’s acclaimed new drama, Detachment, which goes into limited release in New York on Friday, may be its absence of a totally bonkers backstory.

After all, the man responsible for bringing the movie to the screen is renowned in Hollywood for his world-class freakouts, grandstanding meltdowns, and studio headbanging. Kaye’s infamy has endured for nearly a decade and a half, even if he’s quick to tell you that he’s no longer the cuckoo-clock-crazy music-video eccentric who proclaimed himself the “greatest English director since Hitchcock” even before screening a single frame of his jarring movie debut, American History X, in 1998. “My craziness and ridiculousness and stupidity and idiotic nature was largely an act,” Kaye recently told The Daily Beast.

Detachment explores the ruinous state of the American educational system, and features Adrien Brody as a substitute teacher who steps into a broken system to shake his students out of their deep senses of hopelessness, apathy, and rage. Although the character is deeply repressed himself, he somehow manages to also inspire his fellow faculty members to rise above career burnout—and even help out a teenage hooker along the way.

The ensemble drama (which also features strong turns from Marcia Gay Harden, Mad Men star Christina Hendricks, Bryan Cranston, James Caan, and Lucy Liu) cleaned up on the festival circuit last year, gathering critics’ and audience awards at such festivals as Deauville, Tokyo, and the São Paulo International Film Festival. And when it premiered at New York’s Tribeca Film Festival last April, Detachment was met with a number of ecstatic reviews. “I think Detachment has largely gone like a dream,” the director said. “Two or three more [movies] like that and I’ll be back in business. I want to be in the good graces of [studio] executives.”

Appreciation and humility, however, were hardly the character traits Kaye was recognized for in the late 1990s. After making a big splash as an award-winning TV-commercial director, the would-be auteur landed his breakthrough gig directing American History X—which stars Edward Norton as a neo-Nazi ex-con attempting to break his ties to a white-supremacist gang—for New Line Cinema.

But the director publicly clashed with Norton (calling him a “narcissistic dilettante”). And when the now-defunct studio chose to exercise its contractual right to revoke final cut of the movie from Kaye, he went berserk, retaliating through a series of outlandish, hype-generating stunts that have become legendary in Hollywood—a streak of bad behavior that stands in a class of its own even among the supersized ego trips and tantrums for which the movie industry is known.

In 1998, the director spent $100,000 of his own money on a series of 35 trade-paper ads denouncing American History X’s producer and bad-mouthing Norton via quotations from such historical luminaries as John Lennon and Albert Einstein. “Always bear in mind that your own resolution to succeed is more important than any one thing,” read one ad’s text, quoting Abraham Lincoln.

Then, when Kaye was called in to negotiate with New Line top brass about final edits, he brought spiritual backup with him into the executive suite in the form of a rabbi, a Catholic priest, and a Tibetan monk who mingled uneasily with the studio suits while he filmed the whole ordeal.

And yet Kaye remained so incensed at the producers’ decision to tack on 40 minutes of additional footage to his cut of American History X, he lobbied to have his name removed from its credits, soon discovering that a ruling by the Directors Guild of America prevented his use of a pseudonym. So Kaye demanded the movie be credited to “Humpty Dumpty” and filed a $200 million lawsuit against New Line and the DGA for allegedly rigging the arbitration and violating his freedom of speech.

New Line’s cut of American History X eventually was released to generally strong reviews, and Norton received a Best Actor Oscar nomination for his searing performance in the film, now generally regarded as a cult classic.

While some directors are put in a metaphorical “movie jail” for their perceived failures (and thereby relegated to shooting subpar material featuring C-list stars or straight-to-video releases), Kaye suffered a more ignoble backlash. After his American History X meltdown, the director faced the Hollywood equivalent of Napoleon’s banishment to Elba—a span of almost a decade when Kaye was virtually unemployable as a filmmaker, and experienced untold difficulty simply getting people to return his phone calls.

During that fallow period, he never stopped shooting TV commercials and award-winning music videos, even self-financing a controversial abortion documentary, Lake of Fire, that screened at the Toronto Film Festival. But his time away from Hollywood provoked substantial soul-searching by the director about his self-inflicted wounds.

“When you reach the nadir, when you stare into the black hole, the negation of the negation, you sort of wake up and think, ‘Wait a minute, there’s something I’m not doing right here!’” Kaye exclaimed, breaking into a laugh. “‘It can’t always be them. Maybe it’s me.’ And then you learn that not only can it ‘not always be them,’ it’s never them! It’s always you—me! Once that penny drops, you start to reevaluate.”

That particular epiphany might have been well applied to Kaye’s relationship with Marlon Brando, who befriended the director in 1999 and hired him to film a self-help tutorial the faded movie star was teaching about using acting methods in everyday life. The upshot: the director infuriated Brando by showing up to the event dressed as Osama bin Laden, embarrassing him in front of the students. “I didn’t like that. This is my show, not yours!” Brando reportedly shouted at Kaye.

But a subsequent breakthrough allowed Kaye to ascertain the correlation between his outlandish behavior and his ability to function as a filmmaker. “What I have learned is, if you really want to make challenging movies, if you want to make insane movies, you’ve got to be the sanest person in the room—otherwise no one’s going to trust you with it,” he explained. “Whereas I used to think the only way to make challenging, insane films was to be challenging and insane. So that you can really understand the characters and the form you wanted to tell [the story in]—which I found out to be not right.”

Step one on the comeback trail was supposed to have been the crime drama Black Water Transit, a movie with a long and checkered history even before the director became involved, which Kaye filmed in 2007–08 starring Laurence Fishburne, the late Brittany Murphy, and Stephen Dorff. But before they could wrap up principal photography, producers bankrolling the picture ran out of money and got tied up in litigation. Kaye points out that a couple of other acclaimed directors making movies were caught in the same predicament. “Me, David O. Russell, Taylor Hackford—we all got caught in a situation with a very well-meaning company,” the director said. “They stretched themselves too thin and none of us could make our films.”

But Kaye never stopped knocking on movie production-company doors. And after three years developing the Detachment script (written by Carl Lund), he landed financing for a 20-day shoot from New York indie production company Paper Street Films. Brody signed on as the film’s lead, inspired in part by the fact that his father was a junior-high-school teacher.

Reminded of the director’s troubled Hollywood past, Brody—who won a Best Actor Oscar for his performance in The Pianist—insisted Kaye’s difficult reputation attracted him to the project. “That was part of the reason I wanted to do the movie,” Brody told The Daily Beast, “because I don’t like that shit people are always talking about people’s reputations. I like to determine for myself.”

The actor continued: “He’s an amazing director and a fucking great collaborator, a creative genius. The guy is so spontaneous. So unpredictable … He’s eccentric, and I think sometimes people don’t respect eccentricity.”

For his part, Kaye explains that his new lucidity and aim to mend fences in Hollywood have come about in large part because he has devoted himself to studying Kabbalah—a controversial brand of Jewish mysticism—and now places his faith in a higher power.

“You have to take everything you’ve done, rightly or wrongly, and learn from it,” the director said, adamantly. “You’ve got to know there’s a side urging you to make the right decision. And there’s another side that’s very seductive that’s egging you on to shout back, to slap the guy in the face, to throw the table at the wall, punch the wall, to take out a billion ads all over the world, to make yourself look like an idiot: ‘Go on, eat the cake. It’s good! You’ll enjoy it!’”

“You have to know that satanic character is inside you,” Kaye said. “And every day is like a war. You’re at war with yourself.”

Marlow Stern, assistant culture editor for The Daily Beast, contributed reporting for this story.