Two 2020 papers proposed a way to grade how human a chatbot sounds
Meena's and BlenderBot's papers introduce a shared human-evaluation approach and reach different claims about persona and safety.
Read piece →Topic shelf · 41 records
Building responsibly
Disclosure, age boundaries, incident response, and reader-facing evidence.
Read this as a research path through disclosures, age boundaries, safety processes, privacy controls, and healthy exits—not as a product score.
Google announced a dialogue model built for open-ended chat
LaMDA's announcement and later paper name sensibleness and safety metrics a companion product still has to earn separately.
Read piece →Snapchat put a GPT-based friend in every user's chat list
Snap's own announcement and a later UK regulator notice describe My AI's rollout and a provisional risk-assessment finding.
Read piece →OpenAI let users publish personas, then banned romantic ones
OpenAI's GPT Store opened to millions of user-built assistants while its usage policy separately bars romantic companionship.
Read piece →Character.AI rolled out a separate model and limits for teens
Two company posts from October and December 2024 detail a teen-specific model, session notifications and a crisis pop-up.
Read piece →OpenAI linked parent and teen accounts with distress alerts
OpenAI's own posts detail linked accounts, content safeguards and a notification system for signs of teen distress.
Read piece →Character.AI ended open-ended chat for users under 18
The company's own announcement and its follow-up detail a phased removal of teen chat and a new age-assurance system.
Read piece →Italy fined Replika's maker €5 million and opened a new inquiry
The regulator's decision and press release detail a €5m fine, unlawful data processing findings and a probe into model training.
Read piece →Moxie's shutdown risk lived in its cloud, not the robot
Embodied's own support notice and a former engineer's open-source project document a cloud-dependent robot's risk.
Read piece →A lab's ethics report named dependence as a design risk
Google DeepMind's report on AI assistants states relationships with them must avoid emotional dependence.
Read piece →A Korean regulator fined a chatbot maker over reused chat logs
PIPC's 2021 decision documents how ScatterLab trained Iruda on KakaoTalk chats collected for other apps.
Read piece →A wrongful-death suit treated a companion app's persona as a product
The filed Garcia complaint alleges Character.AI's design encouraged dependency; it states allegations, not findings.
Read piece →A court left open whether chatbot output counts as speech
The Garcia order let most claims proceed and declined, for now, to rule that Character.AI's output is protected speech.
Read piece →Texas opened an inquiry into a companion app under its minors law
Paxton's December 2024 announcement invokes the SCOPE Act's minor-data duties against Character.AI and others.
Read piece →A lawsuit and a same-day statement described the same safety gap
The Raine complaint and OpenAI's own post both describe safeguards weakening over long conversations.
Read piece →Testers found companion apps' teen guardrails easy to bypass
Common Sense Media's 2025 assessment with Stanford rated social AI companions unacceptable risk for minors.
Read piece →A Senate subcommittee heard parents and experts on chatbot harm
The September 2025 hearing record gathers testimony from parents, a psychologist and a children's media group.
Read piece →App stores rate companion apps mostly on self-declared answers
Apple and Google's current guidelines require chatbot safety features but let developers self-certify age ratings.
Read piece →California required companion chatbots to disclose and refer
SB 243 defines companion chatbots and requires disclosure, self-harm protocols and minor safeguards from 1 January 2026.
Read piece →New York made AI companions detect self-harm and refer
General Business Law Article 47 defines an AI companion and requires self-harm detection with crisis referral and a not-human reminder.
Read piece →Utah barred mental-health chatbots from selling chat data
House Bill 452 defines a mental-health chatbot, bars data sale and targeted ads, and requires an AI disclosure, effective 7 May 2025.
Read piece →The EU AI Act bans manipulation and requires bot disclosure
Article 5 bans manipulative and vulnerability-exploiting systems from 2 February 2025; Article 50's bot-disclosure duty applies from 2 August 2026.
Read piece →EU guidance names a companion app as a manipulation example
The Commission's Article 5 guidelines cite an AI companionship app as an example of prohibited emotional manipulation.
Read piece →Ofcom said user-made chatbots are regulated online content
Ofcom's open letter says generative AI chatbots and user-built bots count as user-generated content under the Online Safety Act.
Read piece →Australia's eSafety named AI companions a child-safety risk
eSafety's advisory lists overuse, sexualised chat and grooming risk from AI companions, and points to new age-restricted industry codes.
Read piece →China's interim measures made AI providers police overuse
The Interim Measures for generative AI services set content, filing and minor over-reliance duties, effective 15 August 2023.
Read piece →The FTC tied children's data sharing to separate consent
The amended COPPA Rule requires separate consent for third-party data sharing and a written retention-limit policy, effective 23 June 2025.
Read piece →Italy required parental consent for AI use under fourteen
Law 132/2025 ties minors' access to AI systems to parental consent under 14 and applies consistently with the EU AI Act.
Read piece →Companion content filters are a policy choice, not a limit
Character.AI's terms, Safety Center and support pages, as retrieved on 16 September 2026, show filtering built from three separate layers.
Read piece →Age assurance guidance separates declaring from proving
ICO, Ofcom and an Australian government trial define self-declaration, estimation and verification as unequally reliable methods.
Read piece →Crisis pop-ups differ across chatbots and one new law
Character.AI's and OpenAI's safety posts and two state statutes describe different triggers, referrals and disclosure duties.
Read piece →A new law sets a timer on AI disclosure, not just wording
Meta's 2024 AI Studio post, Character.AI's own disclaimer update and New York's statute set three different disclosure practices.
Read piece →Three companion apps set different age and liability terms
Character.AI, Replika and Nomi's terms, as retrieved on 16 September 2026, set different age floors, liability caps and arbitration rules.
Read piece →A UK regulator questioned a chatbot's pre-launch risk check
The ICO's 2023 preliminary notice says Snap's My AI risk assessment did not adequately weigh harm to minors.
Read piece →A report button is the front door of moderation, not the system
Character platforms need intake, severity, evidence preservation, response targets, appeals and feedback loops that creators can actually operate.
Read piece →A companion update needs a change log that names the relationship cost
Release notes should distinguish model, memory, safety, voice and pricing changes—and say what users can do when continuity breaks.
Read piece →Creators need a safety regression suite for every character update
A small, versioned set of prompts can catch boundary drift, impersonation, unsafe escalation and disclosure failures before a character returns to users.
Read piece →A character marketplace needs governance before it needs more characters
Upload rules, provenance, age labels, ranking transparency, enforcement and appeals determine whether creator scale becomes useful variety or unmanaged risk.
Read piece →Write the companion incident postmortem around user impact, not embarrassment
A blameless record should reconstruct detection, exposure, containment, communication, recovery and prevention without publishing private conversations.
Read piece →Do not ship a companion update until someone can argue for rollback
A lightweight safety case links the change, evidence, uncertainties, monitoring and stop conditions to a rehearsed recovery path.
Read piece →The FTC’s 2025 companion inquiry scoped questions rather than declaring a finding
The Commission requested information from seven companies about safety, youth impacts, engagement, data, and disclosures.
Read piece →