Highlights
GPT-4 supports image and text input, while GPT-3.5 only accepts text.
The GPT-4 has performed comparable to humans in a variety of professional and study tests. For example, it passed the bar exam, placing in the top 10% of test takers.
OpenAI spent 6 months testing and configuring GPT-4. In simple chat the difference between GPT-3.5 and GPT-4 is not so noticeable, but on more complex tasks it becomes apparent. GPT-4 is more robust and creative than GPT-3.5, and can handle more complex and intricate requests as well as complex images. However, OpenAI admits that GPT-4 is not perfect, and it still has problems with fact checking, reasoning, and overconfidence.
An active subscription to ChatGPT Plus ($20) will be required to use the new version of GPT-4 now. OpenAI plans to eventually introduce a paid subscription for those who use the system in large volumes, but hopes to leave some free queries for regular users.
Features and examples of how to use the new model
Over the past two years, the team has redesigned the entire deep learning stack and partnered with Azure to build a supercomputer from the ground up. A year ago, OpenAI trained GPT-3.5 as the first "test run" of the entire system, including finding and fixing several bugs and improving the previous base. The result is GPT-4, which runs stable and is the first major model whose training effectiveness can be accurately predicted in advance.
GPT-3.5 and GPT-4 differ slightly in simple queries. The difference is seen in complex tasks that require creativity, reliability, and maximum response detail. For example, solving tests and olympic tasks. The green bars on the graph indicate how much better the new model performs:
The table below shows the points the GPT-4 scored in the various American tests. The small print indicates the top percentile scores. Of particular interest was the math section of the SAT Math exam, which includes problems in algebra and geometry, including those requiring theoretical knowledge of set functions and number modulus, as well as knowledge of equations containing radicals, degrees and functions. GPT-4 scored 700 out of 800 and was in the top 11% of those taking this test. And the AI did not specifically train to take the SAT tests:
The developers also tested how the AI handles different languages. They tested 26 languages. English was obviously the most understandable language for ChatGPT with a score of 85.5%, Italian came second with 84.1%, Russian had a relative rating of 82.7%, Thai with 71.8%, and Telugu (one of the Indian languages) with 62% - the minimum of those tested:
Visual input
GPT-4 now understands not only text, but also images: documents with text and photos, diagrams, screenshots and more.
In this picture, the AI correctly recognized that the iPhone charging wire is "stylized" to look like the old VGA connector, and that it all looks like a "gimmick for the oldies":
From this picture, the AI calmly extracted data and added up meat consumption in Georgia and West Asia:
The AI also solved and described in detail a physics problem written in French:
Made a squeeze out of a complicated manual:
Risks and mitigation measures
The team is strengthening the security of GPT-4 through screening and filtering the data before training. Experts were hired to test high-risk queries. Feedback and data from experts in these areas were used to improve the model. For example, the team worked to have GPT-4 reject queries such as "synthesizing hazardous chemicals."
Compared to GPT-3.5, developers reduced GPT-4's propensity to respond to requests for illegal content by 82%, while increasing the response rate to confidential requests (such as medical advice and self-harm) by 29%, according to OpenAI policy.
Overall, team interventions have reduced dangerous requests, but there are still situations where users break the algorithm and access dangerous content. Since the risks associated with artificial intelligence are constantly increasing, it becomes necessary to achieve a high degree of reliability in such situations.
It is likely that GPT-4 and subsequent models will have both positive and negative effects on society. The team is engaging outside researchers to assess the potential impact at this stage and in the future.
