Hello GPT-4o!

Today, OpenAI released a press release with its Spring Update for ChatGPT, announcing a bunch of new features. The highlight is the new flagship model, GPT-4o (omni), which succeeds GPT-4 Turbo. The model, nicknamed "omni" for its all-encompassing abilities, processes and generates content across text, audio, and images in real-time. This capability makes it exceptionally responsive, matching human-like reaction times in conversations. Another big joy for ChatGPT fans is more new tools for ChatGPT free users.

Let's get to know GPT-4 Omni up close!

Key Highlights (TL/DR)

For those who are short in time, we have carefully reviewed the OpenAI press release and highlighted the most important details about the GPT-4o model for you.

• Available for Free Users: Access is being rolled out to ChatGPT users (for free users as well, with usage limits), and the API is available to developers.

• Text and image input rolling out today in API and ChatGPT with voice and video in the coming weeks.

• Outstanding Results on Chatbot Arena: Achieves a lead of 57 ELO on general tasks and 100 ELO in coding.

• Native Audio Understanding: You can talk to the model, and conversation latency has decreased tenfold compared to earlier voice modes.

• Ability to Sing: The model can sing. Really.

• Real-Time Video Understanding: The neural network understands video in real-time.

• New MacOS App for ChatGPT: You can even stream your screen to it!

• Twice as Fast and Cheap as GPT-4 Turbo: no comments.

• New Multilingual Tokenizer: Some languages now require 4.4x fewer tokens. For instance, the model is 3.5 times cheaper for the Russian language.

• Conversational Mode: Will be available to Plus subscribers in the coming weeks.

• More Advanced Audio and Video Capabilities: Is now available to limited user groups.

Key links

– Full Presentation Video
– Model Page with Demos
– Official Post with Updates

ChatGPT Features Available for Free

OpenAI is inclined to make their advanced AI tools accessible to the broadest audience possible. Each week, over a hundred million people interact with ChatGPT. We are beginning to introduce more intelligent tools to ChatGPT Free users in the upcoming weeks.

With roll out GPT-4o, users of ChatGPT Free users will get access to many of the features that were previously available only to ChatGPT Plus users with a paid subscription.

• Access to GPT-4 for free for all users

• Web access: Receive responses from both the model and the internet

• Analyze data and generate charts

• Chat about images you upload

• Upload files and documents for help with summarizing, writing, or analyzing

• Explore and utilize GPT Assistants and the GPT Store

• Enhance user experience with Memory

  
  

GPT-4o Exceptional Performance

GPT-4o is not only adept at handling English text and code like its predecessor GPT-4 Turbo but also shows significant improvements in processing non-English languages. This enhancement comes with a bonus – it operates faster and costs 50% less to use on the API. GPT-4o's advancements are particularly notable in its understanding of audio and visual content, setting new standards in multimodal AI interactions.

OpenAI states that GPT-4o equals the performance of GPT-4 Turbo in traditional benchmarks for text, reasoning, and coding, while establishing new standards in multilingual, audio, and visual comprehension.

Let's look at the results compared to other LLMs.

ELO Score

Last week on the popular platform ChatBot Arena, where various AI models are compared, a model with the curious name im-also-a-good-gpt2-chatbot caused quite a stir. It turns out that this was actually GPT-4o!

Text Evaluation - General Knowledge

Reasoning Abilities: GPT-4o has achieved a new top score of 88.7% on 0-shot COT MMLU, which tests general knowledge. These evaluations were collected using our straightforward new evaluation library. Additionally, in the conventional 5-shot no-CoT MMLU tests, GPT-4o has reached a new high score of 87.2%.

Vision Understanding

Visual Understanding: GPT-4o has reached best-in-class levels in visual perception benchmarks.

Improved Language Processing

OpenAI has upgraded GPT-4o with better tokenization, meaning it needs fewer tokens for each request. This makes it cheaper to use in various languages compared to GPT-4 Turbo.

Safety and Progressive Rollout

OpenAI has equipped GPT-4o with solid safety features to manage the new audio and visual capabilities. The development of the model included strict safety protocols, such as enhanced data filtering and behavioral adjustments. Despite these measures, OpenAI is rolling out these features slowly, keeping an eye on any potential risks and always working to improve safety measures.

Accessibility and Developer Support

GPT-4o is now available to both free and Plus users of ChatGPT, greatly enhancing the practical usability of AI in daily tasks. For developers, GPT-4o is accessible as a text and vision model through the API, offering improved efficiency and cost-effectiveness. This step shows OpenAI's dedication to making advanced AI tools more available while maintaining high standards of safety and ethics.

Conclusion

The Spring Update from OpenAI has brought us great news. The new clear leader among AI models, GPT-4o, has already set new quality, intelligence, cost-effectiveness and speed standards for large language models. Moreover, a large number of previously paid features that are now becoming free to use will likely give an even bigger boost to the popularity of ChatGPT among AI enthusiasts, and the gap from other platforms will only widen. The question is, for how long.

We can’t wait for the upcoming Google I/O event. It'll be interesting to see how they respond.