Subscribe now

Technology

AI can edit video in real time to sync new audio to people's lips

By Donna Lu

11 September 2020

Harry Potter

Dubbed films could look more realistic by matching actors’ mouths to the sounds

Everett Collection Inc / Alamy

An artificial intelligence can adjust video footage in real time to make people appear to lip sync to any sound.

Prajwal Renukanand at the International Institute of Information Technology in India, and his colleagues have developed an algorithm that can rapidly alter video footage of a person talking or singing.

The researchers believe the AI could be used to alter videos so that the footage appears to match the sound of different languages, such as in dubbed movies.

The team trained the…

Sign up to our weekly newsletter

Receive a weekly dose of discovery in your inbox. We'll also keep you up to date with New Scientist events and special offers.

Sign up

To continue reading, subscribe today with our introductory offers

Piano Exit Overlay Banner Mobile Piano Exit Overlay Banner Desktop