Joe Letteri interview: ‘Avatar: The Way of Water’ senior VFX supervisor

After changing the visual effects and performance capture game with “Avatar” (2009), James Cameron held a summit with his team 13 years ago to discuss how they could make the sequels even better.

“The biggest part of it was since we didn’t really know was the story was, was more about the process,” “Avatar: The Way of Water” senior visual effects supervisor and Weta director Joe Letteri tells Gold Derby at our Meet the Experts: Visual Effects panel (watch the exclusive video interview above). “So the early discussion was highly technical. It was about building out that pipeline. Later on, once we knew what the story was, the discussions became more about, are we gonna achieve the specific effects that we need — water obviously being a big one — but characters? A lot more characters with speaking parts in this film, a lot more varieties of characters — old, new, young, different species. Overall, we have not stopped working on it since that postmortem, but the actual final push on the film, when Jim started doing the performance capture, was about five years ago.”

Among the several technological advancements the four-time Oscar winner and his team created for the sequel is a new muscle-based facial animation software called APFSA (Anatomically Plausible Facial System) that more accurately captures the intricate movements of a human face. The first “Avatar” utilized the same system that Weta had developed for Gollum for the “Lord of the Rings” trilogy. Though groundbreaking at the time, the system’s limited tools couldn’t eliminate some of the “rubbery” look that would pop through.

SEE ‘Avatar: The Way of Water’: Watch Sigourney Weaver transform into 14-year-old Kiri

“We realized that for the fidelity we needed and for the number of characters, the way that we had been doing it wasn’t going to suffice,” Letteri explains. “We wrote an entirely new facial system based on a neural network that could understand the relationship between muscles and skin and what you’re seeing on the face. And that became the animation tool that we used for the facial system. We would capture the actors which a stereo camera rig, pull information from that, feed it to this network to understand what the muscles were doing and use that to drive the characters. And on top of that, the animators had access to the exact same set of muscles. It was a much more intuitive way of working than we had before. They would do the final polish and anything that needed to be done to put it together.”

In that same vein, the new creatures in “Avatar: The Way of Water” are also based in reality. “Any time you’re doing creature design, you want those touchstones,” Letteri says. “Does the audience recognize something similar? On the first film, we had the direhorses, which were basically horses but with six legs. You try to create some similarity while at the same time you’re bringing something new to it.” The sequel prominently features the tulkun, giant whale-like creatures, one of whom, Payakan, befriends Lo’ak (Britain Dalton). Cameron had drawn exactly what he envisioned the tulkun looking like.

“We would pick it up with our animation team because every character is defined by emotion. The look only gets you so far because it’s never static. Is it going to move believably? Is it going to propel itself believably? What is it going to look like from all different angles?” Letteri continues. “Especially with the tulkun because there are so many shots with Payakan… because he is giant. He’s much bigger than a whale, but Lo’ak is sitting next to him and they’re kind of conversing and he’s only got an eye in frame because that’s all you could fit in frame, so we had to do a lot of animation with just that eye to get the bonding, the moments where they were connecting, moments where there was fear. I think our animators enjoyed working with that because it’s very minimal, so you really have to nudge everything.”

Work on the next three sequels is underway. “Avatar 3” is scheduled to be released in December 2024, and by then, you can expect Letteri and his crew will have some new tricks up their sleeves. “Jim was saying a couple weeks ago [about ‘Avatar: The Way of Water’], ‘OK, well, I think we finally know how to make this movie,’” Letteri states. “Let’s apply that to the next one.”

Leave a Reply

Your email address will not be published. Required fields are marked *

UPLOADED Jan 11, 2023 11:36 am