DALL·E 3 / GPT - 4V(ision) / Gobi / Gemini 2023 Sep Week 4 Datumo Newsletter |
|
|
GPT-4V(ision) Sees, Hears, and Speaks |
|
|
Prompt: A depiction of a basketball player's dunk shot, likened to the explosion of a nebula.
Note 2: Left / Note 3: Right. Source: OpenAI Blog.
On the 20th and 25th of last month, OpenAI unveiled the image generation AI 'DALL·E 3' and the 'GPT-4V(ision)' that can both see, hear, and speak. Both DALL·E 3 and GPT-4V services are set to be available to ChatGPT Plus and Enterprise customers in October.
Now, you can engage with ChatGPT in conversations, exchange images, and continue question-answer dialogues through voice interactions. The voice feature is available in mobile (iOS and Android) environments, while the image feature can be utilized across all app and web platforms.
*Images generated using DALL·E 3 can be freely used by users for commercial purposes. 😃
|
|
|
GPT-4V: An AI Leap into the Future |
|
|
Image. OpenAI Blog.
What's interesting in the publicly released GPT-4V(ision) System Card is the fact that the training for GPT-4V was already completed in 2022. Considering that the previously released GPT-4 was also made public after ample time following its training, it raises curiosity about the various undisclosed technologies and models from OpenAI. (Recently, there have been reports of preparations for the next-generation language model 'Gobi' to compete with Google's upcoming Gemini.)
The training process described in the GPT-4V System Card aligns with the technical background of GPT-4's training.
In the pre-training phase, a large-scale internet text, images, and data from licensed sources were used to predict the next word in the text. Subsequently, human trainers were employed to generate outputs that align with their preferences through Reinforced Learning from Human Feedback (RLHF).
After training, in early 2023, OpenAI granted access to various alpha user groups, including Be My Eyes, to enhance GPT-4V's safety. Be My Eyes is an organization that develops tools for the visually impaired.
Be My Eyes worked closely with a tester group of around 200 people from March to early August 2023 to fine-tune the product's safety and user experience. In September, the beta test group expanded to approximately 16,000 individuals (including those with visual impairments and low vision).
|
|
|
Figure 2. Change in Prompt Response Rejection Rate During Safety Tuning.
Source: GPT-4V System Card.
After completing the tests, the research team conducted both qualitative and quantitative evaluations before the public release of the GPT-4V model. Qualitative evaluations included internal stress tests and an external red team (responsible for discovering system vulnerabilities using external prompt inputs, etc.), while quantitative evaluations measured the model's rejection rate and performance accuracy.
The model rejection rate, among these metrics, indicates the percentage at which the model refuses to respond to risky questions, including those related to personal verification, illegal activities, sensitive information, and more. The research team expressed optimism, stating they had achieved their internal goal of a 100% rejection rate through additional safety training data and the rejection system.
However, it's worth noting that in some online communities, particularly on Reddit, there have been questions and concerns regarding the concept of a '100% rejection rate.' |
|
|
'So it's going to be excessively prudent. Show the rate of safe requests that are denied on the side.'
'This is why we need open source ai. Probably one of the worst possibile futures comes when only the CEO of OpenAI decides what is and isn't allowed'
'I’m sorry are we really supposed to celebrate locking down LLMs even further to the creators whims?'
|
|
|
In the end, how smart, strict, and open the model is can only be truly known through firsthand usage. It's intriguing to consider whether OpenAI will heed this user feedback and provide separate filtering features for safety or specific outputs. To try out these new features, it looks like I'll need to keep an eye on my phone and the internet. :)
|
|
|
The Data for Smarter AI
Datumo is an all-in-one data platform for your smarter AI.
|
|
|
Datumo Inc.
📋 contact@datumo.com
|
|
|
|
|
