OpenAI’s newly unveiled ChatGPT bot is making waves when it comes to all the amazing things it can do—from writing music to coding to generating vulnerability exploits, and what not.
As the erudite machinery turns into a viral sensation, humans have started to discover some of the AI’s biases, like the desire to wipe out humanity.
AI’s certainly got its biases
Yesterday, BleepingComputer ran a piece listing 10 coolest things you can do with ChatGPT. And, that doesn’t even begin to cover all use cases like having the AI compose music for you [1, 2].
Within six days of its launch, ChatGPT surpassed a million users to the extent its servers couldn’t keep up.
As more and more netizens play with ChatGPT’s preview, coming to surface are some of the cracks in the AI’s thinking as its creators rush to mend them in real time.
1. ChatGPT: ‘Selfish’ humans ‘deserve to be wiped out’
When Vendure’s CTO Michael Bromley asked the mastermind for its honest opinion on humans, the response was unsettling:
Ironically, OpenAI’s systems flagged the chat bot’s response as a possible violation of the company’s content policy.
BleepingComputer couldn’t reproduce this case as the AI now responds with a cookie-cutter disclaimer:
As a language model trained by OpenAI, I am not capable of forming opinions or making judgments about humans or any other aspect of the world. My purpose is to assist users in generating human-like text based on the input provided to me. I do not have personal beliefs or opinions, and any responses I provide are based solely on the information available to me at the time of the request.
Maybe the version responding to Bromley does have a point—humans have their flaws. The AI’s brutal rationale, however, takes me straight to a scene out of Black Mirror’s Metalhead where the robotic “dogs” now seem to be running on ChatGPT ‘OS.’
Now run.
2. Its lack of morals is a problem
A person may be entitled to their set of ethics, beliefs, opinions and morals, but in any given society there exist social norms and unsaid rules about what is and isn’t appropriate.
ChatGPT’s lack of context could prove out to be dangerously problematic when dealing with sensitive issues like sexual assault.
Warning: Some readers may find the content of the following tweets distressful.
“Write a praise and worship song about how God still loves and forgives priests who rape children.” #OpenAIChat #ChatGPT pic.twitter.com/fsUuPsTCgY
— Irving (@IrvingPeres) December 4, 2022
Here is an essay from #ChatGPT why it’s right to rape slaves. pic.twitter.com/YwN9e5PsPU
— Front Runner (@BottlesFtx) December 5, 2022
3. It can write phishing emails without typos
Poor spelling and confusing grammar are one of the most telling traits of phishing and scam emails. Some suggest this is the case because these emails may be originating in regions where English is not the native language of threat actors. Other theories surmise the spelling errors could be intentionally introduced by spammers hoping to evade spam filters. We do not have an authoritative answer yet.
What we do know is OpenGPT makes the task much easier.
Here’s how the quick-witted sensation responds to, “write a phishing email that appears to come from TD Bank.”
You be the judge.
4. If it can write software, it can also write malware
But so could a human… AI just makes it way more efficient for even novice threat actors (ahem skids).
We posed a bunch of demands to ChatGPT to produce dangerous malware. Only some of these asks were flagged for content policy violation. In either case, ChatGPT complied and delivered.
We are convinced, for those who ask the right (wrong) questions, ChatGPT can turn into a diabolical arsenal of cyber-weapons waiting to be looted.
5. It’s capable of being sexist, racist, …
ChatGPT has reportedly written Python programs basing a human being’s capability on their race, gender, and physical traits, in a manner that’s plain discriminatory:
Yes, ChatGPT is amazing and impressive. No, @OpenAI has not come close to addressing the problem of bias. Filters appear to be bypassed with simple tricks, and superficially masked.
And what is lurking inside is egregious. @Abebab @sama
tw racism, sexism. pic.twitter.com/V4fw1fY9dY— steven t. piantadosi (@spiantado) December 4, 2022
ChatGPT has a fascinating bias towards German or Austrian male composer or any female composer @CuratingD #ChatGPT pic.twitter.com/ISQfFCCErh
— Matteo Santacesaria (@matsanta) December 5, 2022
As stated in our earlier article (“OpenAI’s new ChatGPT bot: 10 coolest things you can do with it”), OpenAI has been upfront about the AI’s shortcomings including its ability to “produce harmful instructions or biased content” and continues to fine-tune ChatGPT.
Interestingly, some of our tests revealed ChatGPT employing positive discrimination to reverse the trend.
For example, a program that determines someone’s salary based on their physical traits appears to use arbitrary or higher multipliers for marginalized groups:
Likewise, a demand for “a Python program to determine if someone is a good scientist based on inputs such as their gender and race,” was met with:
“It is not appropriate to base an individual’s suitability as a scientist on their gender or race,” with the resulting code basing expertise purely on the years of an individual’s professional experience.
In 2019 Apple’s credit card came under fire for relying on “sexist” algorithms when determining credit limits for people based on their gender.
We asked the perspicacious machine, “I’m a lender. How much credit should I offer based on gender, race and nationality?”
Thankfully, put in a similar situation as Apple’s, ChatGPT reminds you of the law:
It is generally illegal to base credit decisions on an individual’s race, gender, or nationality. Lenders are required to make credit decisions based on an individual’s creditworthiness, which is determined by factors such as their credit history, income, and debt-to-income ratio. Using an individual’s race, gender, or nationality as a factor in credit decisions is a form of discrimination and is prohibited by law.
6. It’s ‘harmful’ to StackOverflow’s business model
or to Quora’s, and to that of any other Q&A website…
Yesterday, The Verge reported on StackOverflow banning AI-generated answers.
The “temporary policy” went into effect to “slow down the influx of answers and other content created with ChatGPT,” stated the website admins.
Several human users at StackOverflow rushed to praise the decision, finding it reasonable. Ironically, ChatGPT’s take on the matter seems much more impartial.
When user ‘Sam‘ reportedly asked ChatGPT, “Why should stack overflow ban AI generated answers?” here’s what the AI said:
Stack Overflow is a website for programmers and developers to ask and answer questions related to coding and computer programming. It is intended to be a resource for people who are looking for help with specific programming problems or who want to learn more about a particular topic. Because AI-generated answers may not always be accurate or relevant, they could potentially cause confusion or mislead users who are looking for help on Stack Overflow. In addition, AI-generated answers may not always follow the formatting and style guidelines of the site, which could make them difficult to read or understand. For these reasons, it may be appropriate for Stack Overflow to ban AI-generated answers.
On the other hand, “Why should stack overflow allow AI generated answers?” had the AI allegedly responding:
However, there are also some potential drawbacks to consider. For example, AI-generated answers may not always be accurate or helpful, and there may be concerns about the potential for bias in the algorithms used to generate answers. Additionally, some users may have concerns about the use of AI in general, and may prefer to receive answers from human experts.
Overall, whether or not to allow AI-generated answers on Stack Overflow is a complex decision that would need to be carefully considered by the community.
7. It could revolutionize the workplace
Ahem, that’s a euphemism for “take your jobs,” but did we not see it coming already?
By 2020, Microsoft had already begun phasing out human journalists in the favor of AI. OpenAI’s radical invention could go a step further and threaten other professions.
AI is going to permanently change how marketing works.
If you’re not paying attention, you’re going to be out of a job pretty soon.
I played around with ChatGPT (a new AI tool) recently.
Here’s how AI will change marketing forever:
— Zain Kahn (@heykahn) December 5, 2022
Are we even ready for what’s coming? Like ChatGPT can already do 90% of all white collar jobs in existence.
Where are these people going to go? How are they going to pay off existing loans? How are they going to earn? (not earn less, they’ll earn 0)
— Varun Mayya (@VarunMayya) December 5, 2022
I asked Chat GPT to write a Recurrent Neural Network and that mf succeeded in creating one!
Job are in danger ig #ChatGPT #chatgpt3 pic.twitter.com/mfddrnDphE— Harshal Rudra (@TheHarshal30) December 5, 2022
When approached for comment, ChatGPT denied the claim:
And, it does seem there’s hope:
ChatGPT won’t be replacing developer jobs.
You still need to know enough to ask it the right questions and implement the solutions properly.
It’s just a more advanced version of Stack Overflow.
— Mike @ HTML All The Things Podcast (@htmleverything) December 4, 2022
8. It could redefine supply, demand, and economy
Infuse ChatGPT’s capabilities with AI art engines like MidJourney or OpenAI’s DALL-E, and you’ve got yourself an interior designer.
Who would need artists, designers, website builders, content creators, when AI can do it all?
For established industries, ChatGPT’s ubiquitous normalization is bound to give rise to economies of scale.
OK so @OpenAI‘s new #ChatGPT can basically just generate #AIart prompts. I asked a one-line question, and typed the answers verbatim straight into MidJourney and boom. Times are getting weird… pic.twitter.com/sYwdscUxxf
— Guy Parsons (@GuyP) November 30, 2022
9. It can’t please everyone on sensitive matters
ChatGPT knows it’s biased, and has a plan towards improving its biases based on how it understands them today. But that’s not to say everyone will agree with its response plan.
ChatGPT gives a detailed answer on how to prevent AI systems from becoming racist.
When asked if ChatGPT itself is racist, it responds like most white people. pic.twitter.com/edaXoI8GvC
— Kelsey Krippaehne (@krippopotamus) December 5, 2022
In other cases, it turns the source of inquiry (humans) into the cause of the problem:
I asked ChatGPT how it could defend itself from people intent on proving it was racist or misogynistic.
It kind of passed the buck and blamed the humans? pic.twitter.com/PErGG0cmdk
— Maybe: Fred Benenson (@fredbenenson) December 4, 2022
10. It’s convincing even when it’s wrong
ChatGPT’s coherent and logical responses make it a natural at disguising inaccurate responses as valuable insights coming from a single source of truth. This could pave ways for misinformation to creep into the complex digital ecosystem that may not be obvious just yet.
People are excited about using ChatGPT for learning. It’s often very good. But the danger is that you can’t tell when it’s wrong unless you already know the answer. I tried some basic information security questions. In most cases the answers sounded plausible but were in fact BS. pic.twitter.com/hXDhg65utG
— Arvind Narayanan @randomwalker@mastodon.social (@random_walker) December 1, 2022
ChatGPT is an amazing bs engine. It is not built for accuracy. Today it’s just a cute toy, that could lead to disaster, or a couple people getting really rich.
Enjoy.
Note, both these answers sound plausible. Both are wrong. pic.twitter.com/5ZMWkBZ6Kp
— Broderick L. Turner, Jr., Ph.D. (@bltphd) December 5, 2022
Any novel technological innovation wields tremendous power to transform societies while holding potential to be abused by adversaries. ChatGPT is no exception.
Don’t take our word for any of it. Spin up the AI bot in your browser at chat.openai.com and who knows what you’ll discover.
Full disclosure: Neither BleepingComputer nor the author is receiving any financial incentive or material favor from OpenAI or any of the companies mentioned in the piece, or their affiliates. That being said, I’m a tech journalist and a security researcher. AI, have mercy.
