When Century Star Shivarajkumar’s Ghost released in October last year in Kannada, Tamil, Telugu, Malayalam and Hindi, it seemed that all five versions had the actor’s vocals. While the actor is fluent in Kannada and Tamil, dubbing for the rest of the versions, would have been tough, and yet, it wasn’t. Turns out, Shivarajkumar did not dub for these versions; instead, director MG Srinivas, the director of the film, used AI technology to clone the actor’s voice to give the impression it was him in all the versions.

How voice cloning works

Srinivas has now officially launched the technology, AI Samhitha, for the benefit of other film teams who are serious about providing audience great visual and auditory experience, especially for pan-India releases. “For the best output, an actor has to dub his/her voice in the ‘source’ language, which will then be replicated by trained dubbing artistes. These dubbed versions will then be run on he software, wherein the cloning will happen and the final output will be close to the original. As long as the dubbing artiste is able to match about 50-60% of the original voice’s intonations, the software will do the rest,” says the actor-filmmaker.

Voice cloning will eliminate the need for an actor to have to try and dub in languages alien to him/her, says Srinivas, adding that it will render audio that will then sound flawless to a native audience member. “It will also eliminate voice mismatch between an actor and the dubbing artiste. Very often, when you watch a film dubbed in another language, and you are familiar with how the artiste actually sounds, it may feel jarring when you hear the dubbing artiste’s voice instead,” says Srinivas.

Voice cloning is only the first step, adds the filmmaker. “We are working on technology that will sync the lip movement as well. Let’s say, a film is shot in Kannada, and the actor dubs for it in other languages too, but when you watch, there’s still a mismatch between the lip movement and the audio. This will be sorted. We are still in testing phase for this and are hopeful of making it happen shortly,” he says.

Read Also: comedian paritosh tripathi on karan johar slamming his show for mimicry of him: we will be extra cautious going forward