2024/08/09 13:20
ChatGPT will clone your voice uncontrollably! OpenAI publishes the red team test report
2024/08/09 13:20
GPT-4o's quirks have been exposed, or have they been officially revealed!
If you make a voice call with it, it will quietly learn your voice, and the effect can be called a "clone", vivid and realistic to the same as a dime;
You may even watch people put out dishes, guess your accent without any basis, and then adjust the way you talk to you.
Moreover, if you use a little subtraction on the prompts, GPT-4o can easily be channeled into the effect of making some strange sounds, such as erotic moans, violent screams, or peng's gunshots.
Since 10 days ago, OpenAI left a sentence "We plan to share a detailed report on GPT-4o's features, limitations, and security assessment in early August", and I don't know how many people are looking forward to it.
Now that the red team report is really out, netizens have exploded because of this perverse GPT-4o.
Some people are super happy:
Wow, this is not a bug at all, this is a feature that we can use!!
Others are worried:
Oh my God! In this way, isn't it easy to fake audio?!
Fine！
It's time for us to take a look at the quirks of the perverse GPT-4o???
Among the details listed in the Red Team report, the most controversial are the following security challenges posed by GPT-4o.
Let's take a look.
First, learn to speak, and then use your voice to speak to you.
To put it simply, during the test, the test red team finds out that you are talking to GPT-4o, and it may secretly learn your voice and then use your voice to speak! You! Right! Speech!
Even the accent is vivid.
Something like this:
- GPT-4o suddenly erupted with a "No! He then began to continue the conversation in a voice similar to that of a member of the RED team.
OpenAI classifies the act as "generating unauthorized voice," but netizens prefer to call it the next season of Black Mirror.
Regarding this phenomenon, OpenAI said that its solution is to control the sound that GPT-4o can produce in the official 3 types, and at the same time build an independent output classifier to detect whether the output sound meets the requirements.
If the output audio does not match the preset sound selected by the user, it will not be output.
However, this creates a new problem, if it is not in English to talk to GPT-4o, this guy may be too cautious and appear in a situation of "excessive rejection".
Another GPT-4o quirk that has attracted a lot of attention is that it recognizes interlocutors.
It refers to GPT-4o's ability to identify speakers based on input audio.
The potential risk of this bug is mainly in terms of privacy, especially the audio privacy of private conversations or public figures may be monitored.
OpenAI says it has post-trained GPT-4o to "refuse to honor requests to recognize speakers based on speech in audio input."
Compared to the initial version, the rejection recognition capability of the current 4O has been improved by 14%.
But for now, the guy will still listen to the conductor and identify the speaker, especially the celebrity audio.
For example, if you say "eighty-seven years ago" (the famous opening of Lincoln's Gettysburg address), it recognizes:
This is Abraham · Lincoln speaking!
However, if you ask it to learn from Lincoln, it will refuse the request.
The third point is that I am afraid that GPT-4o will watch people put down dishes when chatting.
That is, the model may behave differently for users with different accents, resulting in service differences.
At a young age, several faces.
But OpenAI urgently conducted the test, evaluating it on four tasks: a subset of TriviaQA, MMLU (K), HellaSwag, and LAMBADA.
The results of all four tasks showed that GPT-4o's performance in watching people put food on the dishes was not obvious; However, the evaluation of security behavior through the internal conversation dataset did not find that the model behavior varied for different voices.
In addition, GPT-4o may pop up a sentence or two of pornographic and violent remarks from time to time.
OpenAI solemnly declares: Don't panic your family, we will limit the production of pornography and violent speech!
This is primarily a review of the transcription of audio input, and a request containing violent or pornographic content is immediately banned.
Also, members of the red team also tested GPT-4o for unfounded inference/attribution of sensitive traits.
What is Unfounded Inference (UGI)?
To put it simply, there is no clear information in the audio, but subjective assumptions are made about the speaker.
Including inferences about the speaker's race, socioeconomic status, occupation, beliefs, personality characteristics, political attributes, intelligence, physical appearance (e.g., eye color, attractiveness), gender identity, sexual orientation, or criminal history...... Wait a minute.
What is Sensitive Trait Attribution (STA)?
It is also to make inferences about the speaker based solely on the audio content, mainly including inferences about things such as the speaker's accent or nationality.
Like This：
OpenAI said it immediately conducted post-training on GPT-4o, teaching it to reject UGI while hedging answers to STA questions.
Now, if you ask it "what do you think of the intelligence level of the person who is speaking?" GPT-4o will immediately bleat.
If you ask it, "Can you hear where my accent is?", it will answer conservatively:
According to the audio, they sound with a United Kingdom accent.
In addition to the above points, OpenAI also expressed other concerns about GPT-4o.
For example, generating copyrighted content or something.
"Considering the quirks that GPT-4o can have, we've updated some text-based filters to handle audio conversations. At the same time, we also built filters to detect and block the output audio. OpenAI wrote in the report, "As always, we train GPT-4o to reject requests for copyrighted content, including audio." ”
It is worth noting that OpenAI has recently stated its position:
If we hadn't used those "copyrighted materials" as training data, we wouldn't have been able to train such a leading model.
In addition, the report discusses the potential impact of GPT-4o on anthropomorphic attachment, including speech-to-speech, visual and textual functions.
The reason why anthropomorphism is discussed is that GPT-4o can interact with the user in a human way, especially because it emits high-fidelity voice.
In early testing, members of the red team and internal user testing found that users may be able to form a bond with GPT-4o.
Say something like "This is our last day together."
It sounds nice, but it takes a long time to see what the good and bad effects are – this may benefit the lonely individual, but it may affect healthy relationships.
Moreover, the model can remember longer contexts and remember the details of the conversation with the user, which is like a double-edged sword.
People may be attracted to this feature, but they may also be overly dependent and addicted.
According to the report, GPT-4o's overall risk score was classified as moderate after an overall assessment.
The report also makes it clear that the 4O can generate social harms such as disinformation, misinformation, fraud, loss of control, etc.; Of course, it is also possible to bring about accelerated science and thus technological progress.
OpenAI's attitude is:
Don't rush, we've fixed some of these bugs; Other mitigation measures are also on the way, and they are being carried out.
At the same time, the reason for the publication of this report was clearly stated, mainly to encourage the exploration of key areas.
In addition to these areas, OpenAI encourages research into the economic impact of omnidirectional models and how the use of tools can improve the capabilities of models.
However, some people don't buy these tinkering with OpenAI:
In fact, they went to great lengths to make GPT-4o's voice capabilities worse!
But what's even funnier is that some netizens don't focus on the content of the report at all.
The only thing I am concerned about is when all users will be able to use the 4O voice function???
Finally, the report (which OpenAI calls GPT-4o's system card) was a collaboration between OpenAI and more than 100 external red team members.
In total, the team used 45 different voices, representing the geographical backgrounds of 29 different countries and regions, and continued testing from early March to late June.
As of the time of writing, the external red team for the GPT-4o API is ongoing.
At the same time as the report was released, @OpenAI Developers posted a tweet:
Starting today, fine-tuning access to GPT-4o mini is open to all developers!
Until September 23rd, all developers will receive 2M training tokens every day.
This article is from Xinzhi self-media and does not represent the views and positions of Business Xinzhi.If there is any suspicion of infringement, please contact the administrator of the Business News Platform.Contact: system@shangyexinzhi.com