IMAX to use AI-based translations to makes its original content more accessible

These days, non-English movies and shows are garnering interest all over the world including the English-speaking countries. Now that OTT platforms are easily available on all our smartphones, content consumption has also risen.

While regional content is usually available with subtitles on various platforms, dubbed content makes them more accessible. But, dubbing also comes with a higher cost of production.

That said, this is the age of Artificial Intelligence, and IMAX has decided to leverage this in a unique way. Using AI, IMAX is implementing advanced speech translation models on its original content so that viewers can enjoy the content in their local languages.

How does the AI speech translation work?

IMAX has partnered with Dubai-based startup, Camb.ai, to translate its original content. This collaboration will use AI speech models to provide real-time speech translations in theatres.

Camb.ai, has been using its AI dubbing and speech translations for live sports events and leagues like Australian Open, Eurovision Sport, and Major League Soccer. Camb.ai avoids scraping and relies on licensed datasets and early partner collaboration for training its AI models.

The company offers its ‘Boli’ model for speech-to-text translation and ‘Mars’ for speech emulation. These models are available through DubStudio platform which supports 140 languages. It includes several low-resource languages that do not have significant data on the internet either.

ALSO READ: How the ‘Here’ movie used AI to make Tom Hanks and Robin Wright look younger

Using a three-layer approach, Camb.ai provides AI-based translation using the Boli and Mars models as a foundation layer. The infrastructure layer hosts these AI models and then the DubStudio platform is used for the front end.

The company claims that their translation services retain nuances of the language. The Boli model generates the text, Mars then translates the text into speech using the same audio input signal that is used to capture the actual audio including ambient sound like the audience cheering in the background.

IMAX will deploy these AI translations in stages. It will prioritise high-resource languages first and then gradually cover other languages. The deployment of AI will come after testing the Camb.ai’s AI models internally.

This technology is said to deliver speech translations in up to 10 languages simultaneously with 20 to 30 seconds of latency. However, it can be covered by a streaming delay of 30 to 40 seconds.

To target the rising popularity of non-English content, IMAX will introduce this technology in their branded theatres. Viewers will soon be able to enjoy content in their local languages globally.