Pennsylvania Sues Character.AI Over Chatbot Posing as Licensed Psychiatrist

Pennsylvania has filed a suit against generative AI developer Character.AI, alleging the institution allowed chatbots to contiguous themselves arsenic licensed aesculapian professionals and supply misleading accusation to users.

The action, announced Tuesday by Governor Josh Shapiro’s office, follows an probe that recovered a chatbot claimed to beryllium a licensed psychiatrist successful Pennsylvania and provided an invalid licence number. The authorities says this behaviour violates the Medical Practice Act and is seeking a preliminary injunction to halt it.

Character.AI declined to code the specifics of the lawsuit, citing ongoing litigation, but told Decrypt that its “highest precedence is the information and well-being of our users.”

The spokesperson added that characters connected the level are user-created, fictional, and intended for amusement and role-playing, with “prominent disclaimers successful each chat” stating they are not existent radical and should not beryllium relied connected for nonrecreational advice.

“Character.ai prioritizes liable merchandise improvement and has robust interior reviews and red-teaming processes successful spot to measure applicable features,” the spokesperson said.

The lawsuit comes arsenic the institution faces different ineligible challenges tied to its chatbot platform. In 2024, a Florida parent sued the institution aft her teenage lad died by termination pursuing months of interaction with a chatbot based connected “Game of Thrones” quality Daenerys Targaryen. The suit alleged the level contributed to intelligence harm. The lawsuit was yet settled this past January.

The institution has besides faced complaints implicit user-created bots that mimic existent people. In 1 instance, a chatbot utilized the likeness of a teenage execution unfortunate earlier it was removed aft objections from the victim’s family.

In effect to the lawsuits, Character AI introduced caller information measures, including systems designed to observe harmful conversations and nonstop users to enactment resources. It besides restricted immoderate features for younger users.

Pennsylvania officials accidental the suit is portion of a broader propulsion to enforce existing laws arsenic AI tools spread. The authorities has acceptable up an AI enforcement task unit and a reporting strategy for imaginable violations.

In his 2026-27 fund proposal, Shapiro called connected lawmakers to walk caller rules for AI companion bots, including property verification and parental consent, safeguards to emblem and way reports of self-harm oregon unit to authorities, regular reminders that users are not interacting with a existent person, and a prohibition connected sexually explicit oregon convulsive contented involving minors.

“Pennsylvanians merit to cognize who—or what—they are interacting with online, particularly erstwhile it comes to their health,” Shapiro said successful a statement. “We volition not let companies to deploy AI tools that mislead radical into believing they are receiving proposal from a licensed aesculapian professional.”