Meta’s Movie Gen is an AI model that can generate realistic videos with audio

The AI race has been live for some time now, and several players have jumped onto the bandwagon. In this space, video generation is a popular category for companies to bet on, and we have seen OpenAI doing that with Sora. Meta has now announced its own AI model for video generation, and it competes against Sora with its realistic video generation capabilities.

Let’s see what’s this new Movie Gen model and how it may make the video creation process easier than ever.

What’s Movie Gen and how is it different from the competition?

Meta’s Movie Gen is an AI model that generates videos based on text inputs. Interestingly, it can not only generate new videos, but also edit existing ones. The model can generate audio too and sync it with the visuals.

Meta’s CEO Mark Zuckerberg shared a video sample created by the AI model, where he’s seen doing his leg press sessions, not just in the gym, but also at several other locations with different settings, thanks to Movie Gen.

Although there’s no audio in the video sample shared by Zuckerberg, the AI model can also generate audio, that is in line with the video, something that OpenAI’s Sora lacks. The latter focuses only on the video.

Apart from this, Meta has also shared a couple of other video samples of the AI model, and they look high-res and real. Meta confirmed that with Movie Gen users can generate high-quality videos at a resolution of 1080p.

Meta uses a 30 billion parametre model for video generation and 13 billion parametre for audio generation. These models make Movie Gen capable of creating complex scenes in footage. It uses an LLaMa3 transformer architecture and also utilises the flow matching technique to make the video look natural and realistic.

ALSO READ: Sora is a generative AI platform that creates great-looking videos from just text

Movie Gen looks like a great AI model, however, similar to OpenAI’s Sora, it won’t be available immediately to all users. But the good news is that the model will be coming to Instagram next year, giving consumers an easy way to generate videos for the platform.

Meta hasn’t revealed the consumer pricing of the AI model, however, we should know more about that once it’s available.