Twelve years ago, Andy Serkis received the phone call that would change his life—and motion pictures—forever. Up until that point, the British thespian had garnered a reputation as a gifted character actor, having appeared in several stage productions and independent films, including Mike Leigh’s Career Girls and Topsy-Turvy. But this was something different.

“My agent said, ‘They’re doing this film Lord of the Rings down in New Zealand and they’re looking for a voice for an animated character. It’s about three weeks work.’” Serkis wasn’t impressed. “God, there must be over a dozen characters in Lord of the Rings,’” he said. “Can’t you get me something proper?’”

After an inspiring meeting with the films’ director, Peter Jackson, Serkis caved, and his turn as Gollum—a bug-eyed, schizophrenic, computer-generated creature—in the Lord of the Rings trilogy was a watershed moment for visual effects. For the first time ever, an actor’s performance was able to be effectively translated—using motion capture technology provided by Jackson’s digital visual effects company, Weta Digital—onto a CGI character.

Now, a decade after the release of the first Rings film, The Lord of the Rings: The Fellowship of Ring, another breakthrough has been made in the visual effects realm—the character of Caesar (played by Serkis), an ape possessing human-like intelligence and emotion, in the blockbuster prequel, The Rise of the Planet of the Apes. Film critic Roger Ebert called Caesar, “a wonderfully executed character,” adding, “One never knows exactly where the human ends and the effects begin, but Serkis and/or Caesar gives the best performance in the movie.”

Serkis’ performance capture—as it’s called now—turn as Caesar is so captivating it’s left many awards pundits debating whether Serkis should receive an Academy Award nomination for Best Supporting Actor. Twentieth Century Fox, the studio behind Apes, has revved up an aggressive Oscar campaign on behalf of Serkis’s performance, taking out ‘For Your Consideration’ ads in the major Hollywood trades. So far, it’s proven mildly successful—Serkis received a Critic’s Choice award nomination for Best Supporting Actor, though he was shunned by the more prestigious SAG and Golden Globe awards. But the award everyone really cares about is the Oscar, so should Serkis, for his performance capture accomplishment, be nominated for an Academy Award? And, as Ebert asked, where does the human end and the effects begin?

If you ask Serkis, the answer is clear: “Weta is very loyal to the actor’s performance and are able to interpolate that through the translation process after the director and actors have shot the scene on set,” said Serkis in an interview with The Daily Beast. “They’re very conscious that when they apply the ‘digital makeup,’ it is to honor the actor’s performance; not to bend or enhance it.”

For Gollum in the Lord of the Rings films, Serkis would have a much weaker case. Back then, motion capture technology was in its infant stages, so only the body’s movements—not facial ones—could be captured and translated to CGI, according to Joe Letteri, the four-time Academy Award winning visual effects artist and director of Weta Digital. Letteri, who has worked at Weta on all their films since The Lord of the Rings: The Two Towers, including King Kong, Avatar, Rise of the Planet of the Apes, and The Adventures of Tintin: The Secret of the Unicorn, also says that Serkis’s performance as Gollum couldn’t be captured while he was acting, so the actor would have to “re-perform each of his shots after the fact” on a motion capture stage.

Motion capture technology evolved with 2005’s King Kong, which saw Serkis playing the titular great ape. While Serkis would still have to reshoot his scenes on a motion capture stage ex post facto, the Weta Digital artists were able to capture the face.

“We put markers on Andy’s face and had him in a small area so we could get accurate camera information, then we tracked the markers on his face,” said Letteri. “Because of the similarity of ape facial muscles and human facial muscles, we worked out a way of translating the emotional expressions between the two.”

Avatar took things to a different level, thanks to director James Cameron. The filmmaker helped develop a facial rig—a helmet-mounted camera pointed in front of the actor’s face—allowing Weta to not only do a better job of translating actor’s facial movements to their CGI character, but also to capture the performance with other actors as it happens, instead of having to do reshoots on a separate stage.

Weta employed the same technology used by Avatar to the character of Caesar in Rise of the Planet of the Apes, but then went one step further.

“We redesigned the whole motion-capture system to use active LED markers that we put on the suits, so that allowed it to work outside, in full sunlight,” said Letteri. “It really allowed Andy’s performance to be translated directly, so what he did with the actors got put on the screen as Caesar.”

Directed by Rupert Wyatt, Rise of the Planet of the Apes opens with Will Rodman (James Franco), a scientist at a biotechnology company who is testing a drug on chimpanzees to help cure Alzheimer’s. The drug, however, grants the apes super intelligence. When one of the apes goes on a rampage, all the apes are ordered to be put down, but Will sneaks one baby chimp out of the lab, whom he names “Caesar,” and raises him in his home. With the help of a primatologist (Freida Pinto), Caesar, played by Serkis, grows up to be an ape with genius-level intelligence. In a year filled with memorable movie animals—Joey in Steven Spielberg’s War Horse, Uggie the Dog in The Artist, etc.—Caesar is king.

For Kong, whom Serkis describes as “stoic” and like “a very lonely, psychotic hobo trying to survive,” the actor spent months with gorillas in zoos and even traveled out to Rwanda to study mountain gorillas. Caesar, despite speaking just five words in the film and communicating mostly through grunts and gestures, is quite the opposite.

“He was a young, gifted child who was brought up in a loving household until he has this moment of self-awareness where he realizes that he’s some kind of Frankenstein monster and doesn’t belong,” said Serkis. “So how do we convey his thoughts and emotions without over-anthropomorphizing him? As Caesar develops, it was making him more enigmatic, stiller, and internalizing his thoughts—which goes against a lot of what people think performance-capture is.”

According to director Rupert Wyatt, it’s no coincidence that Serkis excels in performance capture roles—in particular, that of Caesar—because of his uncanny ability to emote through bodily and facial gestures.

“Andy would’ve been perfectly suited for the silent movie era,” said Wyatt. “He’s very expressive, and uses every part of his body and face to convey emotion and narrative. I relate it to the Charlie Chaplins and Buster Keatons of this world; they were great comics and great tragedians as well.”

The actor and director sat down and, like any other performer, mapped out the character of Caesar together. They created an arc for Caesar, made decisions about how Serkis would display him through physical movements, and developed who the character was going to be. In some ways, Serkis’ performance as Caesar could be seen as the epitome of an actor losing him or herself in a role.

“All of the things that you see in the film are exactly the same as if he were playing a human being character, and then some, in many ways, because it takes a very rare skill indeed to play another creature, and do it in a way that audiences can connect with it,” said Wyatt.

Thanks to cutting edge performance capture technology, Caesar’s face is more defined than CGI characters past. According to Letteri, this was accomplished by the ability to input more detail in into the facial system itself, especially the eyes. There was also more movement because the team at Weta Digital had a better idea of how to handle the simulation of the muscles, and the layering that happens underneath the skin that accounts for a lot of the movements audiences are seeing.

And Serkis, clad in his performance capture outfit—a gray leotard with balls covering his body to register movement—was able to perform alongside his other actors in what both Wyatt and Serkis refer to as “a stage performance.” And, according to Wyatt, all of the film’s actors swiftly managed to get into the groove of interacting with Serkis on set—a testament, he says, to both Andy’s ability and that of the other actors, some of whom had their reservations prior to shooting.

“When I first came on set I was concerned about how awkward acting opposite an actor pretending to be an ape in a grey suit with green dots on his face would be,” Freida Pinto told The Daily Beast. “However, if there was one person who could quell that concern and make it the most natural actor to actor process it would have to be Andy. His character’s body language and its movements were terrifyingly identical to that of our cousin primates, however, the thing that stood out for me the most was Andy’s ability to emote in the head space of his ape character Caesar to absolute pitch perfection.” She added, “If I had to feel Caesar’s happiness or fear or determination, Andy made sure that I felt it all. We were emotionally connected to Andy’s performance throughout the film in the same way it is meant to happen on any film project between actors.”

Critics of performance capture acting—and Serkis’ Oscar chances—argue that it can’t be compared to standard acting, since the performance isn’t directly translated onto the screen, but undergoes a digital translation process where teams of visual effects artists can tweak and manipulate details of the character.

“It’s very hard to quantify and it is a judgment call,” said Letteri. “There’s a lot of technology behind the translation, and there are things that you have to do in translation that are not 1:1 to what the performer does. If Andy smiles and the corner of his mouth moves by a given amount, you can’t make Caesar’s mouth move by that exact amount because his mouth is differently proportioned.”

Director Rupert Wyatt, however, doesn’t buy the “translation” argument, instead arguing that the process of crafting an onscreen character via performance capture is very similar to developing any film performance through editing.

“It’s the art of editing,” said Wyatt. “The amount of times I have worked in an edit room with an actor’s material and have created a performance from using certain parts of a take that weren’t even intended for that part of the story; that’s the nature of filmmaking. So, the idea that what Andy performed on set shouldn’t be acknowledged as an entirely legitimate performance doesn’t stack up with me.”

Serkis agrees, adding, “The actor has complete authorship over every emotional or physical beat of the role. I’m not being ridiculous, however. If Caesar’s climbing a 200-ft. redwood tree, I didn’t do that. But neither would an actor; a stunt guy would. We’re just talking about the performance here.”

The 47-year-old Serkis will next appear as Captain Haddock—a garrulous, alcoholic and sidekick to boy-reporter Tintin—in The Adventures of Tintin: The Secret of the Unicorn, filmed entirely in performance capture. He also wrapped filming on The Hobbit—a prequel to the Lord of the Rings split into two separate films—where he’ll reprise the role of Gollum, and, according to Wyatt, will come back for a sequel to Rise of the Planet of the Apes, set right after the events of the first film.

As far as Serkis’ Oscar chances go, however, perhaps we should leave it to his Apes co-star James Franco—himself an Oscar nominee—who’s dabbled in a bit of performance art himself.

“Andy has broken new ground in acting,” Franco told The Daily Beast. “What he does with performance capture is something no other actor can do. The digital aspect of his character is just a necessary shell that goes on top of a masterful performance. If there is any justice, he’ll win the Oscar. No one has yet given a performance like his, ever.”