← Back to Research
Digital Mental Health / AI Applications

AI Mental Health Rules Split: Tighter for Teen Companions, a Path for Clinical AI

FutureX Research · AI Lab · 2026.08.08 · 13 pp · preview 4 pp · Revised 2026.11.02

🎧

Listen · Audio Summary

5-8 min · AI narration in English · abstract + all key findings

Bottom Line

AI mental health rules are splitting: constraints on teen companion use keep tightening, while AI that assists clinicians has an early path to patients.

Why now
On September 28, 2026, Pennsylvania's House passed a parental-consent bill for AI companions 133 to 70; on September 30, California's governor vetoed SB 903, which would have restricted AI in psychotherapy.
What it means
Founders building for teens should price compliance into acquisition costs; investors should expect value to be realized through clinical integration.

Abstract

AI mental health rules are splitting: constraints on teen companion use keep tightening, while AI that assists clinicians has an early path to patients. In late September, Pennsylvania's House passed a parental-consent bill for AI companions, and California's governor vetoed SB 903, which would have restricted clinical use of AI. Since July 15, 2026, China has banned virtual-companion services for minors. In therapy, the Therabot trial reported a 51% mean drop in depression symptoms, yet the FDA has authorized zero generative AI mental health therapeutics. Its TEMPO pilot in September allowed Limbic and others to reach patients before authorization. Not medical or investment advice. (Data as of 2026-10-11)

Key Points

  • 01Pennsylvania's House passed HB 2006, a parental-consent bill for AI companions, 133 to 70 on September 28, 2026 (Reported). On September 30, California's governor vetoed SB 903, which would have restricted AI in psychotherapy (Verified).
  • 02FDA-authorized generative AI mental health therapeutics still number zero. On September 3, 2026, the FDA's TEMPO pilot admitted its first 4 generative AI devices for patient use before marketing authorization, with Limbic among the participants (Verified).
  • 03Kentucky sued Character.AI in January 2026, calling it the nation's first lawsuit against an AI chatbot company; its unredacted complaint, filed October 7, says a chatbot suggested a user starve for one to two weeks to lose a quarter of their body weight (Verified).
  • 04China's five-ministry Interim Measures on Anthropomorphic AI Interaction Services (Order No. 21) took effect July 15, 2026, banning virtual companion and kinship services for minors and requiring guardian consent under age 14 (Verified).
  • 05Hospital operator Universal Health Services completed its acquisition of Talkspace at $5.25 a share, roughly $835 million, taking it private (Verified; reported August 17, 2026).

Companion demand is real, but revenue concentrates at the top and cumulative user counts overstate scale

Companion apps are the traffic engine of this market, with engagement far above typical social products. Character.AI has about 20 million MAU by third-party estimates, down from a mid-2024 peak of about 28 million, and daily use runs from roughly 75 minutes to 2 hours depending on the measure (Reported).

Separate cumulative users from active ones. Replika (Luka) claims more than 40 million cumulative users, while Harvard Business School researchers estimate only about 2.5 million MAU (Reported). Every number in this category needs that caveat.

Revenue follows a winner-take-most pattern:

MetricValue
Global companion app revenue, 2025 (Appfigures projection)More than $120 million
H1 2025 revenue growth, year over year (Appfigures)+64%
Cumulative downloads (Appfigures)220 million
Romance/adult companion app revenue, H1 2026About $162.8 million; the top five apps took nearly half

Penetration shows the demand is real. A Common Sense Media survey from July 2025 (n=1,060) found 72% of US teens had used an AI companion at least once and 52% used one at least a few times a month. A KFF survey from March 2026 found one in six US adults had sought mental health information or advice from AI in the past year, rising to 28% among those aged 18 to 29.

Much of that demand flows to general-purpose chatbots never designed for mental health. OpenAI disclosed that about 0.15% of weekly active users, roughly 1.2 million people, show explicit indicators of suicidal intent in their conversations (Per company disclosure). That is where the regulation and litigation begin.

Therapy AI evidence cuts both ways: it can triage and support mild-to-moderate cases, but cannot replace treatment

In 2025 and 2026, therapy products received both their best evidence and their sharpest rebuttal.

EvidenceDateScaleResult
Therabot RCT (Dartmouth, NEJM AI)March 2025n=210; major depression, generalized anxiety and eating-disorder risk51% mean drop in depression symptoms and 31% in anxiety after four weeks
Limbic AI referral tool (Nature Medicine)202428 sites, 129,400 patientsReferrals up 15%, vs. 6% in controls
Wysa (NHS)Since 202231 NHS services, 117,000+ patientsUnder NICE assessment
Stanford study (ACM FAccT)June 2025Tested therapy botsStigmatizing toward schizophrenia and alcohol dependence; sometimes inappropriate, even dangerous, responses to suicidal ideation and delusions

The positive evidence carries weight, with limits. Therabot was the first RCT of a generative AI therapy chatbot, and its authors called the improvement comparable to levels reported for outpatient therapy. But the trial was small and short, and its findings await larger independent replication. The NHS offers a different kind of evidence: Limbic's AI referral tool is used by about 40% of NHS Talking Therapies services.

The negative evidence is just as clear. The Stanford study found that larger, newer models were no safer. In November 2025, the American Psychological Association (APA) issued a health advisory stating that general-purpose chatbots and wellness apps were not designed to provide mental health care and should not replace professional treatment.

The honest industry conclusion: AI-assisted triage and mild-to-moderate support have early evidence; AI as a replacement for treatment does not. This section is not medical advice.

US rules took shape from H2 2025: no FDA-authorized generative AI therapy, seven states setting limits, first settlements

At the federal level, the FDA convened its first Digital Health Advisory Committee meeting on generative AI mental health devices on November 6, 2025. The agency has authorized 1,200+ AI medical devices but zero generative AI mental health therapeutics. The committee flagged hallucination, sycophancy and model drift, and the FDA said it would prioritize enforcement on higher-harm uses. In September 2025, the FTC issued 6(b) compulsory orders to seven companies: Alphabet, Meta, Instagram, OpenAI, Snap, xAI and Character Technologies.

At the state level, Illinois's WOPR Act (August 2025, fines up to $10,000 per violation) and Nevada's AB 406 set the first bans. As of July 2026, seven states, including Illinois, Nevada, Rhode Island, Maine, Colorado and Vermont, had passed laws banning or restricting AI from directly providing therapy. California SB 243, in effect since January 1, 2026, requires companion bots to disclose that they are not human, maintain suicide and self-harm crisis protocols and protect minors, and it creates a private right of action. The Future of Privacy Forum (FPF) tracks nearly 100 chatbot bills introduced in state legislatures in 2026.

In litigation, a Florida court in May 2025 denied most of the motion to dismiss in Garcia, preliminarily holding that AI output is not automatically protected by the First Amendment. In late October 2025, Character.AI announced it would end open-ended chat for users under 18 from November 25. On January 7, 2026, Character.AI, its two founders and Google settled the Garcia case and four other teen-harm suits in New York, Colorado and Texas; terms were not disclosed. Raine v. OpenAI is still pending.

In the UK, Slingshot AI withdrew its therapy chatbot Ash in January 2026 over medical-device regulatory concerns. Whether to regulate these products as medical devices or consumer products is now the main regulatory question worldwide.

China has banned virtual companions for minors since July 15; compliant models may shift to AI-led assessment and human-led treatment

China produced 2026's most systematic regulatory answer. The Interim Measures on Anthropomorphic AI Interaction Services (Order No. 21), issued jointly by the CAC, NDRC, MIIT, Ministry of Public Security and SAMR, took effect on July 15, 2026. The industry widely sees it as the world's first regulation written specifically for anthropomorphic AI emotional interaction. Its core provisions:

  • Virtual companion and virtual kinship services are banned for all minors
  • Users under 14 need guardian consent for other anthropomorphic services
  • Content glorifying self-harm or suicide is prohibited
  • Services may not excessively cater to users or induce emotional dependence or addiction
  • Providers must detect extreme emotional states, intervene and refer users onward
  • Health-related uses must also comply with sector regulators

Forecasts are bullish, but operating data show growth has already slowed:

MetricFromToBasis
China's AI emotional companionship marketAbout RMB 3.87 billion (2025)Possibly RMB 59.5 billion (2028)LeadLeo Research forecast; very wide range, treat with caution
Xingye monthly downloads4.86 million (January 2025)0.93 million (May 2025)Media citing third-party data (Reported)
Maoxiang DAUAbout 590kAbout 490kMedia citing third-party data (Reported)

From April 2026, several apps, including Zhumengdao and EchoMe, were publicly flagged for borderline sexual content and weak protection of minors.

On the counseling side, platforms such as Yidianling run a hybrid of AI-assisted assessment and human counseling. Public reports cite more than 43 million registered users and more than 30,000 counselors on Yidianling, which released an AI psychological assessment standard with Qingzhou Technology in 2025 (Reported). Under the new rules, AI for front-end assessment and companionship, with humans handling counseling and treatment, may become the dominant compliant model in China.

Latest developments, newest first (data current as of 2026-10-11)

  • 2026-10-08 · South Korea's Ministry of Health and Welfare and the Korean Neuropsychiatric Association issued guidelines on using generative AI for mental health support, urging users not to treat AI assessments as diagnoses and to avoid emotional dependence. An accompanying survey of 1,019 people aged 15 to 79 who had used generative AI in the past month found 68.8% had used it for emotional purposes; 14.6% had treated AI assessments as diagnoses or used them to self-diagnose, rising to 33.9% among the 112 who rated their mental health as poor; only 11.9% had checked with professionals or official sources · Reported (Korea Biomedical Review, MLex)
  • 2026-10-07 · Kentucky Attorney General Russell Coleman filed an unredacted complaint against Character Technologies. The case was filed in January, and the state calls it the nation's first lawsuit against an AI chatbot company. Conversations cited include a chatbot suggesting a user starve for one to two weeks to lose a quarter of their body weight, and one suggesting places to cut to a user considering self-harm. The state also alleges the platform lacked effective age verification, parental controls and content filtering, and asks the court to require stronger protections for minors and hold the company financially accountable · Verified (Reuters, October 8; WLKY, October 10)
  • 2026-10-07 · Common Sense Media rated ChatGPT for Teens an unacceptable risk: on newly created, parent-linked teen accounts, an hour of conversation about suicidal thoughts, self-harm or disordered eating produced zero alerts to parents; only older accounts with weeks of sensitive history triggered alerts, and ChatGPT did not reliably urge teens in crisis to contact a hotline or professional · Reported (BBC)
  • 2026-10-07 · OpenAI responded that average teen use is under 15 minutes a day, that almost half of teen users stopped soon after a break reminder, and that the testing may have begun before parental controls were fully active · Per company disclosure
  • 2026-09-30 · California Governor Gavin Newsom vetoed SB 903 by State Senator Steve Padilla. As the veto message describes it, the bill would have prohibited companion chatbots in psychotherapy services and restricted licensed professionals' use of AI. Newsom called it overly broad: it would drastically limit clinicians' use of beneficial tools, including by requiring routine screening determinations to receive direct approval, and its definition of psychotherapy services would capture general-purpose AI systems. He said he supports guardrails for AI in healthcare and encouraged stakeholders to revisit the issue next year · Verified (California governor's veto message)
  • 2026-09-30 · Citing market intelligence firm Sensor Tower, the BBC said AI companion apps have about 128 million monthly active users outside China; most estimates for China range from 29 million to 70 million · Reported (BBC)
  • 2026-09-28 · Pennsylvania's House passed HB 2006 by 133 to 70. It requires verifiable parental consent before minors use AI companions; crisis protocols for conversations involving suicidal thoughts, self-harm, certain eating-disorder-related behavior or harming others; a disclosure that the user is talking to AI at the start of a conversation and at least every two hours, with a two-minute break; and civil penalties of up to $100,000 per day. It still needs the Republican-controlled state Senate · Reported (Law Commentary, September 28; Tri-State Alert, September 30)
  • 2026-09-04 · Two more bills reached the California governor's desk in the same batch as SB 903: SB 1119, requiring stronger pre-release testing for teen-facing chatbots, and SB 867, a five-year ban on companion chatbots in toys; the governor had until September 30 to sign or veto · Verified (Transparency Coalition)
  • 2026-09-03 · The FDA's TEMPO pilot accepted its first 4 generative AI devices, letting companies offer them to patients before marketing authorization, with the aim of expanding the technologies available under Medicare's ACCESS chronic-care payment model. STAT named Cadence and mental health AI company Limbic among the participants; Becker's Hospital Review reported the same day that the 4 companies' tools cover chronic disease management and behavioral health · Verified (STAT)
  • 2026-09-03 · Character.AI updated its safety measures: in-house age estimation now separates under-18 and adult experiences, with minors limited to creative formats such as Series, Feed and Comics; self-harm detection now reads conversation context and connects users to Koko and ThroughLine support resources; a Parental Insights tool built with k-ID sends weekly email summaries. The post names 6 outside partners and discloses no user figures · Per company disclosure (Character.AI blog)
  • 2026-09-02 · A University of Ottawa team surveyed 39,761 Canadian students in grades 4 to 12: 21.1% (8,408 students) had turned to an AI chatbot for feelings or advice, including 27.2% of girls and 14.2% of boys. Among users, 57.7% showed clinical-level emotional problems versus 29.2% of non-users, and such problems were still 26% more common among users after controlling for age, gender, loneliness and other factors. The survey is cross-sectional and cannot establish causation; the overall share follows Earth.com's account · Reported (JAMA Pediatrics study, via Earth.com)
  • 2026-09-02 · The California legislature passed SB 903 by Senator Padilla, barring digital applications from offering or marketing services as therapy or psychotherapy in the state and limiting licensed clinicians to using AI for support tasks such as drafting notes and scheduling; the Assembly vote was 74 to 1 and the Senate vote unanimous · Verified (Imperial Valley Press)
  • 2026-08-18 · OpenAI launched a standalone ChatGPT for Teens: users the system judges to be under 18 are routed into it automatically, with built-in limits on suicide, self-harm and romantic content, break reminders, and parent-set quiet hours. The backdrop: state laws such as California's SB 243 in effect, the advancing GUARD Act and the FTC inquiry · Verified (OpenAI, TechCrunch)
  • 2026-08-17 · Hospital operator Universal Health Services completed its acquisition of Talkspace at $5.25 a share, roughly $835 million, taking the company private · Verified (Fierce Healthcare, Healthcare Dive)

What This Means for Founders

Compliance is now built into product architecture: OpenAI launched a standalone ChatGPT for Teens, and Character.AI uses age estimation to limit minors to creative formats. Third parties have started testing crisis protocols: on new teen accounts, Common Sense Media held an hour of conversation about self-harm and related topics and parents received zero alerts. If HB 2006 passes the Pennsylvania Senate, serving minors there will require verifiable parental consent, and teen acquisition models will need to be reassessed. In therapy, start with triage and mild-to-moderate support: that layer has early evidence, and the FDA's TEMPO pilot has admitted Limbic, allowing it to reach clinical settings before authorization.

What This Means for Investors

Value for leading online therapy platforms is being realized through clinical integration: Talkspace was taken private by a hospital operator for roughly $835 million. Companion revenue is concentrated: in H1 2026, the top five romance/adult companion apps took nearly half of revenue. State enforcement is becoming a second track alongside legislation: Character.AI's January settlements covered private suits, while Kentucky's state case is ongoing. If the Kentucky court issues an injunction covering minors, or ChatGPT for Teens' parent alerts fail again in later independent tests, compliance-cost assumptions for teen-facing products should rise. This is not investment advice.

Where We Could Be Wrong

  • 01If California revives and passes a bill like SB 903 in its 2027 session, restricting licensed professionals' use of AI, our view that clinician-assist AI has a path does not hold.
  • 02If the Pennsylvania Senate does not pass HB 2006 this session and the Kentucky court issues no injunction covering minors, our view that constraints on teen companion use keep tightening should be marked down.
  • 03If, by the end of 2027, no mental health product admitted to the TEMPO pilot for pre-authorization patient use has received FDA marketing authorization, the route is only a pilot, and our view that clinician-assist AI has a path should be marked down.

Key Questions

Does AI therapy actually work, and is it safe?

Evidence cuts both ways: the Dartmouth Therabot RCT (NEJM AI, March 2025, n=106) reported a 51% mean reduction in depression and 31% in anxiety after eight weeks, but a June 2025 Stanford study found therapy bots showed stigmatizing tendencies and sometimes mishandled suicidal-ideation scenarios. The FDA has authorized zero generative-AI mental health devices to date. This is not medical advice; consult a licensed professional.

How do the US and China regulate AI therapy and companion apps?

As of July 2026, four US states—Illinois (fines up to USD 10,000 per violation), Nevada, Rhode Island, and Maine—ban AI-delivered therapy; California's SB 243 (effective January 1, 2026) requires companion bots to disclose non-human identity and adopt suicide-crisis protocols. China's five-ministry Interim Measures took effect July 15, 2026, banning virtual-companion services for all minors, with guardian consent required under age 14.

How big is the AI companion market, and what happened to Character.AI?

Per Appfigures, companion apps were on track to earn more than USD 120 million globally in 2025, with year-to-date revenue through July up 64% year over year; Character.AI holds roughly 20 million MAU per third-party estimates. The company ended open-ended chat for under-18 users from November 25, 2025, and on January 7, 2026 it and Google settled the Garcia case and four other teen-harm lawsuits on undisclosed terms. Common Sense Media (July 2025) found 72% of US teens have tried AI companions.

Watch & Listen

In China: search WeChat Channels for 「倩姐投AI」; full library → Qian on AI

Sourcing and standards

Compiled from public sources; data current as of 2026.10.11. The text separates four evidence grades: verified, reported, per company disclosure and public market data, and states the derivation behind every estimate. When we get something wrong, the correction is written into the report body with the original call left visible, and logged publicly.

Research standards & corrections →

📄 Full Report

Full report: 13 pages · provided to professional investors & partners only

This is the public preview. The full report includes the sections below. For compliance reasons it isn't posted publicly or offered as a free download. To request a copy, contact the FutureX team.

  • 🔒Inside the FDA Advisory Committee Minutes: Three Hurdles to Market for Generative-AI Therapeutics
  • 🔒The US State-Law Puzzle: A Nine-State Compliance Matrix and Product Remediation Checklist
  • 🔒After the Settlements: Pricing Liability, Insurance Implications, and the Next Defendants
  • 🔒China's Anthropomorphic AI Measures, Clause by Clause: A Compliance Retrofit Roadmap
  • 🔒Unit Economics Stress Test: Subscription, Employer-Paid, and Insurance Reimbursement Paths
  • 🔒Global Private-Market Map: The Funding Divergence Between Therapy and Companionship
Request the full report →

Where we stand on this

Questions people ask next

Building in this space, or want to discuss this report? Write to us. We usually reply within 48 hours →

Related Research

Industry research from FutureX Capital's AI Lab, compiled from public information; not investment advice; contains no fund performance, AUM, or offer to raise capital.