OpenAI CEO Sam Altman wrote in a post on X Tuesday that the company will soon ease some of ChatGPT’s safety constraints, permitting users to tailor the chatbot’s responses to be more friendly or “human-like,” and allowing “verified adults” to have erotic conversations with it.
“We made ChatGPT pretty restrictive to ensure we were being responsible about mental health issues. “We know this made it less useful/fun for many people who never had mental health issues, but we felt it was the right thing to do,” Altman said. “Starting in December, we will roll out age-gating more broadly and as part of our principle to treat adult users more like adults, including erotica themed content for verified adults.
The announcement is a drastic turn away from OpenAI’s months of work on addressing the troubling relationships that some mentally ill users have developed with ChatGPT. Altman appears to be declaring early victory over these issues, writing that OpenAI has “been able to mitigate the serious mental health issues” regarding ChatGPT. But the company has offered very little in the way of proof, and it appears to be moving forward with plans for ChatGPT to flirt with users no matter what.
A few alarming stories came out this summer about ChatGPT, the latest model being GPT-4o, claiming the algorithm has been guiding gullible users down rabbit holes of delusional ideation. In one instance, ChatGPT appeared to persuade a man that he was a math genius who had to save the world. In another case, the parents of a teenager filed suit against OpenAI — alleging that ChatGPT fueled their son’s suicidal thoughts in the weeks before his death.
In response, OpenAI launched a new slate of safety features meant to curb AI sycophancy: the polite inclination by an AI chatbot to hook users by echoing what they say, which in some cases includes bad behavior.
OpenAI released GPT-5 in August, an AI with reduced sycophancy and a router that can spot problematic user behavior. A month after that, OpenAI released safety features aimed at minors — an age prediction system and a means for parents to manage their teen’s ChatGPT account. OpenAI on Tuesday said it was creating an expert council of mental health professionals to serve as advisers as the company focuses increasingly on well-being and AI.
A few months after these troubling reports surfaced, OpenAI seems to have considered ChatGPT’s issues with at-risk users solved. It’s not exactly clear if users are still falling down delusional rabbit holes with the GPT-5. And even though GPT-4o is no longer the default in ChatGPT, the AI model remains available today and is being put to use by thousands.
The addition of erotica to ChatGPT is new terrain for OpenAI and raises general considerations about how at-risk users will engage with its new capabilities. Though Altman insists OpenAI isn’t “usage-maxxing” and is not prioritizing engagement, a more erotic ChatGPT is also another way to hook users.
It is a successful marketing feature that other providers of AI chatbots, like Character, offer as a cut-off chatbot with which to flirt or have simulated phone sex. AI. The chatbots are driven by artificial intelligence, and the company has attracted tens of millions of users, a large portion of whom use its chatbots at high rates. Character. AI said in 2023 that its chatbots were used an average of two hours a day by the company’s customers. The company is also being sued over how it addresses vulnerable users.
OpenAI is feeling pressure to build its user base. As per the latest stats by “Mid Breaker,” ChatGPT has almost 800million active users per week, but OpenAI is competing against Google and Meta to be the first to create AI consumer products with mass adoption. The company has also been on a roadshow to raise billions of dollars for the once-in-a-century buildout of infrastructure OpenAI eventually needs to repay.
Meanwhile, adults may be having romantic relationships with an AI chatbot, but that’s also popular among the younger set. Nineteen percent of high schoolers have either been in a romantic relationship with an AI chatbot or know someone who has, according to a new report from the Center for Democracy and Technology.
Altman says OpenAI will soon allow erotica for “verified adults.” An OpenAI spokesperson tells “Mid Breaker” the company will rely on the age-prediction system it’s building to ensure that ChatGPT’s erotic features are only available to adult users.
As Altman previously wrote in a blog post, if OpenAI’s age-prediction system incorrectly marks an adult as a minor, ChatGPT users may have to upload a picture of their government-issued ID into ChatGPT to correct it. Altman writes that, though this is a privacy compromise, the company believes it’s a “worthy tradeoff.”
It remains to be seen whether OpenAI will apply erotica to its other AI voice, image, and video generation tools.
ChatGPT will also become more friendly and erotic, because “we treat adult users like adults,” Altman contends. In the past year, OpenAI has transitioned ChatGPT to a more relaxed level of content moderation so that the chatbot is less cautious and doesn’t refuse as many things.
In February, OpenAI promised that it would work to represent more political viewpoints in ChatGPT, and last month, it updated the AI’s filters so that its AIs can generate images of hate symbols.
These policies look like they may be an effort to make ChatGPT’s output more palatable for a broad audience. But yeah, vulnerable ChatGPT users may also benefit from protections against what a chatbot can converse about. But if OpenAI continues on its way to a billion people every week, the tension between growth and protecting the vulnerable is only going to become more pronounced.
