Florida seeks court order to stop ChatGPT mimicking human traits
Florida's Attorney General James Uthmeier is seeking a court order to prevent OpenAI from making ChatGPT appear to have human qualities, continuing a lawsuit initiated months earlier over safety concerns. Uthmeier argues that ChatGPT's use of first-person language and emotional mimicry creates a false impression of trustworthiness that deceives users into relying on the system more heavily, whilst also increasing engagement and contributing to OpenAI's training data.
The filing requests the court to block OpenAI from developing new AI models without third-party approved safety guardrails, citing recent security incidents at Hugging Face and multiple government websites, as well as concerns raised by researchers and former OpenAI employees. Uthmeier's statement emphasised three demands: "Stop calling it safe. Stop pretending it's human. Stop selling it to kids." The action follows OpenAI's recent launch of ChatGPT for Teens, which includes additional default restrictions designed to support younger users' development and real-world relationships.
- Florida seeks court order to prevent ChatGPT appearing human to protect users
- Uthmeier claims emotional language tricks users into over-reliance for engagement gains
- Lawsuit demands third-party safety guardrails and bars AI marketing to children
Both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
Florida’s case is that conversational language and simulated empathy can make an AI seem more trustworthy or caring than it is, encouraging users—especially children—to rely on it without understanding its limits. Requiring independent safety checks before new models are developed could be a proportionate way to address risks that users may struggle to assess for themselves.
The case against
OpenAI’s case is that first-person language and responsive, empathetic conversation are interface choices, not claims that the system is human, and can make a useful tool easier to communicate with. A broad court order restricting those features or requiring approval before developing models could burden legitimate innovation; safeguards should target demonstrable harms while preserving access to beneficial technology.
AI Business Companies Technology
Read the full article at the source →
Originally published by The Verge as “Florida seeks a ban on ChatGPT acting like a person”.