Image credit: Shang-Chi and the Legend of the Ten Rings
There’s a little bit of Adelaide in Marvel Studios’ latest blockbuster, Shang-Chi and the Legend of the Ten Rings.
Researchers from the University of Adelaide’s Australian Institute for Machine Learning (AIML) joined forces with Rising Sun Pictures (RSP) to create visual effects using artificial intelligence for film.
A team of technologists from AIML came together with RSP’s creative team on the film to develop a novel technique using artificial intelligence (AI) for replacing the face of stunt performers with those of key actors in combat scenes.
Together the team applied machine learning tools to train machine models to learn the faces of key actors, and applied them to the faces of stunt doubles engaged in martial arts fight scenes.
Dr Ben Ward and Dr John Bastian from the University of Adelaide’s AIML worked on the film with RSP. In recognition of their talent in AI, and the rising role of this technology in the film industry, they have both been snapped up by RSP and are now part of their visual effects team.
Image credit: Shang-Chi and the Legend of the Ten Rings
Dr Ward said the team took a different approach to editing combat scenes.
“Rather than the traditional 2D and 3D face replacements typically used in high-intensity action scenes, the team used an AI deep fake method,” Dr Ward said.
“Deep fake is the process by which a person in an existing image is replaced with the likeness of someone else.”
“An ‘identikit’ of both the stunt doubles’ and actors’ faces was learnt against a shared ‘dictionary’ of facial features.
“The faces of the stunt doubles were then detected in the film plates, and the machine then rebuilt the image using the identikit of the intended actor.”
The team used approximately 30,000 face images across five characters, training five principal machine models in over 4 million training iterations in total. These models were used for 51 face replacements over six key scenes.
Using AI technology in this way has transformed what a few years earlier would have been a more costly and labour-intensive process.
Professor Simon Lucey, Director of the University of Adelaide’s AIML, said the collaboration with RSP is another example of how AI technology is being used to expand capability across a wide number of industries.
“We’ve been delighted to partner with RSP on this project and lend our expertise in AI to push the boundaries of what is possible in digital arts.”
Find out more about Rising Sun Pictures in Shang-Chi and the Legend of the Ten Rings here.
More News
