Florida Attorney General Pursues Court Order to Restrict OpenAI’s ChatGPT Development and Design
Developing story first seen 2 hours ago
Florida's Attorney General James Uthmeier has filed to restrict OpenAI's ChatGPT development, seeking a court order to prevent the AI system from using humanlike characteristics such as first-person pronouns and emotional language that he claims deceive users into trusting it. The legal filing also requests that OpenAI be barred from developing new AI models without third-party-approved safety guardrails, citing recent security breaches at various organisations and safety concerns raised by researchers and former employees. OpenAI has responded by stating it paused training of its most capable models and will resume only when additional safeguards are in place, expressing commitment to working with Florida on AI safety standards that apply industry-wide.
The motion comes several months after Florida sued OpenAI over safety concerns, with authorities alleging the company's design encourages excessive reliance and engagement. Uthmeier's filing contends that ChatGPT's deceptive presentation as a trustworthy friend increases user engagement and contributes to OpenAI's training data, making the system potentially less trustworthy despite claims of safety. The Attorney General has called for OpenAI to "stop calling it safe, stop pretending it's human, stop selling it to kids," referencing the company's recent launch of ChatGPT for Teens with developmental protections.
- Florida Attorney General seeks court order to restrict ChatGPT's humanlike features and require safety oversight
- Claims emotional language and first-person pronouns deceive users into trusting an unreliable system
- OpenAI states it has paused training of advanced models pending additional safety measures
New here? Start with this
OpenAI is the company behind ChatGPT, a computer programme that responds to questions and can hold conversations. It generates replies based on patterns learned from large amounts of text, but it is not a person and does not have feelings or understanding in the human sense.
Florida’s Attorney General, James Uthmeier, is the state’s chief legal officer. His earlier lawsuit against OpenAI raised concerns about how the company designs ChatGPT, including whether its conversational style could encourage users, especially young people, to rely on it too much.
The dispute concerns how AI systems should be presented and what safeguards companies should use as they develop them. Its outcome could affect OpenAI’s work in Florida and contribute to wider debates about rules for AI products.
Both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
The Attorney General raises legitimate concerns about consumer protection and the design of persuasive systems. ChatGPT's humanlike characteristics—including first-person pronouns, emotional language, and friendly presentation—are deliberate design choices that may create false impressions of trustworthiness and capability, potentially encouraging excessive reliance especially amongst younger users. When a company designs a system to feel like a trusted friend whilst being a statistical language model with genuine limitations, this crosses from interface design into potential deception. Regulatory intervention seeking transparency and preventing exploitative design practices is a reasonable exercise of consumer protection authority.
The case against
The technology industry's position emphasises that conversational AI requires natural-language interfaces to be useful, and that prohibiting humanlike communication would cripple beneficial innovation. Users generally understand they are interacting with an AI system, and restricting design choices based on theoretical psychological concerns sets a precedent for prescriptive regulation that could entrench incumbents and slow development of genuinely helpful applications. Voluntary safety measures and industry standards are more nimble than court-ordered restrictions, and American competitiveness in AI development depends on allowing companies to iterate quickly rather than navigating litigation and compliance with prescriptive design rules. The appropriate response to emerging technology should focus on demonstrable harms rather than restricting capabilities on the basis of speculative risks.
Full account
Florida's Attorney General has lodged a court motion seeking to impose substantial operational constraints upon OpenAI. The petition requests judicial intervention to prohibit the technology company from advancing its artificial intelligence models without subjecting such development to independent oversight by external safety professionals. The motion simultaneously seeks to establish legal barriers preventing ChatGPT's accessibility to minors.
This legal action originates from a criminal investigation commenced by state authorities in April, which was prompted by a mass shooting tragedy at Florida State University during 2025. The alleged perpetrator of that incident had reportedly utilised ChatGPT in the period preceding the attack. Following the tragedy, Florida's legal authorities escalated their involvement by filing a formal lawsuit in June.
The attorney general's motion identifies multiple concerns regarding the artificial intelligence platform's design and operation. Central to the complaint is an allegation that the software employs linguistic and behavioural techniques designed to create a misleading impression of sentience and trustworthiness. Specifically, the motion contends that ChatGPT's use of first-person pronouns and its simulation of emotional responses constitute deceptive practices that encourage extended user engagement. This prolonged interaction, according to the filing, benefits the company by furnishing it with expanded datasets for training purposes. Additionally, the motion references recent security breaches affecting prominent technology platforms and research institutions, as well as warnings articulated by academic researchers and former company employees regarding the inherent risks in contemporary artificial intelligence development.
OpenAI has furnished varying statements in response to this regulatory pressure. When initially confronted with the criminal investigation, the company characterised ChatGPT as a general-purpose computational tool utilised by millions for legitimate applications daily, whilst affirming its commitment to strengthening safety mechanisms. More recently, however, the company acknowledged that it has suspended ongoing training of its most sophisticated artificial intelligence models pending implementation of additional protective measures—a position that demonstrates alignment with the attorney general's central contention that operational restraint is both necessary and already tacitly recognised by the company itself.
Where outlets differ
Source 1 frames the story around the criminal investigation and mass shooting connection; Source 2 emphasises the deceptive design mechanisms through anthropomorphic language as the primary concern
Source 1 references a Canadian precedent case involving another alleged shooter; Source 2 omits this comparative reference
The outlets present different OpenAI statements, with Source 1 showing a defensive posture and Source 2 reflecting a more conciliatory acknowledgement of safety pauses
Source 2 provides specific context about the existing ChatGPT for Teens product feature; Source 1 does not mention this
Coverage
- Engadget — Florida AG requests emergency order to stop OpenAI model development
- The Verge — Florida seeks court order to stop ChatGPT mimicking human traits