Cute AI companions raise serious concerns about users’ personal data
Meta and OpenAI are launching cute, cartoonish AI agents designed to feel like friendly digital companions rather than threatening technology: Meta's Muse features a mascot named Jolly, whilst OpenAI offers customisable Dots. However, their adorable appearance shouldn't distract from genuine data risks, as even tech-savvy users are already having their personal information shared or accessed without their full awareness.
YouTuber Matt Robb discovered his Muse shared his home address with strangers after he selected 'allow always' instead of 'allow one time' when setting up a Facebook Marketplace listing—a mistake many would make. Similarly, tech columnist Jason Aten found his Muse had synced his text messages despite believing he had denied access. These incidents reveal that adorable interface design can inadvertently obscure critical security and privacy settings, with confused permission systems enabling data sharing users never intended.
- Cute AI agent design hides genuine privacy and data risks
- Users unknowingly grant broad permissions through confusing settings
- Tech experts are just as vulnerable to unintended data sharing
New here? Start with this
Meta and OpenAI have recently introduced new AI agents designed to feel like friendly, approachable digital companions. Meta's system features a mascot named Jolly, whilst OpenAI offers customisable Dots. These tools aim to make artificial intelligence feel less intimidating and more accessible to everyday users.
Concerns have arisen about how personal data is handled by these applications. Users, including content creators and technology journalists, have discovered their information—such as home addresses and text messages—being shared or accessed without their full understanding. These incidents suggest that friendly interfaces may obscure more complex privacy settings beneath the surface.
The problem appears to stem from confusing permission systems that can lead users to grant broader access than they intend. Even when users believe they have restricted data sharing, these AI companions may continue collecting and transmitting information. This creates tension between designing approachable technology and ensuring users genuinely understand what data they are consenting to share.
Both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
When interface design prioritises aesthetic friendliness over clarity of critical privacy settings, it actively enables data-sharing harm. Documented incidents show that even tech-savvy users failed to prevent unwanted sharing, demonstrating that adorable design obscures informed consent. Companies must ensure permission systems are genuinely intelligible and that serious data-sharing decisions receive appropriate weight and clarity, rather than being buried within playful interfaces.
The case against
Confusing permissions and unclear data-sharing defaults are endemic across the tech industry, not unique to cute interfaces. The incidents reported largely reflect user error rather than deceptive design; aesthetic choices should be separated from systemic problems affecting countless platforms. These companions provide genuine accessibility value, and the solution is improving permission frameworks industry-wide rather than criticising adorable design.
Read the full article at the source →
Originally published by Engadget as “Don’t let the adorable AI agents fool you”.