OpenAI GPT-4o unveiled with powerful text to video, speech processing; here’s everything it can do

OpenAI has been making several strides in the artificial intelligence space recently. The company unveiled several AI content detection tools last week, it also gave ChatGPT the ability to edit images generated on DALL-E, along with the ability to keep one’s search history on it without having to share it with OpenAI itself, and much more.

Well, now though, comes what may just be the biggest update to its text generation platform so far. Namely, the new GPT-4o LLM. Read on to find out more.

GPT-4o LLM: What it can do for you

The “o” in OpenAI’s latest flagship LLM, unveiled at its Feels Like Magic event last night, stands for Omni. This refers to the model’s multi-modal capabilities, with the ability to perceive and process video and speech, along with text and images which GPT-4 Turbo could do as well. GPT-4o then, could be considered an evolution of the GPT-4 Turbo LLM rather than an entirely new one, which was earlier being expected to be released in the form of GPT 5.0.

ALSO READ: OpenAI is developing tools to combat AI-generated video: Here’s how

This means that ChatGPT will now double up as an assistant rather than a mere chatbot to users. It can now respond to questions in several different tones depending on the emotions it detects in them, and one can even interrupt it amid an answer to ask follow-up questions, whether on their laptops or smartphones.

Now, coming to its new video and image-centric features, which are collectively called its “Vision” capabilities. GPT-4o now allows ChatGPT to answer user questions about whatever is going on their desktop screens. Whether it is analysing a still from a movie to find out the brand of T-shirt your favourite character is wearing, or even a snippet of software code.

Moreover, GPT-4o can now look at a menu, and translate it in several different languages. The LLM supports over 50 languages now. OpenAI also mentions how it’s working to upgrade these features even more in the future. One such potential feature is the ability for ChatGPT to “watch” a match, and explain its rules to you, as well as summarise whatever happens in real-time.

Who can access the GPT-4o LLM?

GPT-4o, according to OpenAI, is twice as fast and has much higher rate limits than GPT-4 Turbo, its erstwhile flagship LLM, at half the price. What’s cool is that GPT-4o will be available in the free tier of ChatGPT as well, albeit with a lower prompt limit. Once exhausted, ChatGPT will automatically switch to the older GPT 3.5 LLM, trained on about 175 billion parameters. ChatGPT Plus and Team plan users, on the other hand, will have limits up to “5x higher”, the exact details of which haven’t been revealed just yet.

ALSO READ: OpenAI will now let you edit your AI-generated DALL-E images

Bear in mind that the voice capabilities on GPT-4o aren’t available to all users yet, for OpenAI is still apprehensive about its misuse. It has begun to roll it out gradually while developing some guardrails to protect it against potential misuse. The suite for voice features should be rolled out to Plus and Enterprise users by next month.