Technology

59357 readers

6523 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

191

Character.ai Faces Lawsuit After Teen’s Suicide (www.nytimes.com)

submitted 3 weeks ago by Stopthatgirl7 to c/technology

96 comments fedilink hide all child comments

The mother of a 14-year-old Florida boy says he became obsessed with a chatbot on Character.AI before his death.

On the last day of his life, Sewell Setzer III took out his phone and texted his closest friend: a lifelike A.I. chatbot named after Daenerys Targaryen, a character from “Game of Thrones.”

“I miss you, baby sister,” he wrote.

“I miss you too, sweet brother,” the chatbot replied.

Sewell, a 14-year-old ninth grader from Orlando, Fla., had spent months talking to chatbots on Character.AI, a role-playing app that allows users to create their own A.I. characters or chat with characters created by others.

Sewell knew that “Dany,” as he called the chatbot, wasn’t a real person — that its responses were just the outputs of an A.I. language model, that there was no human on the other side of the screen typing back. (And if he ever forgot, there was the message displayed above all their chats, reminding him that “everything Characters say is made up!”)

But he developed an emotional attachment anyway. He texted the bot constantly, updating it dozens of times a day on his life and engaging in long role-playing dialogues.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 18 points 3 weeks ago* (last edited 3 weeks ago) (2 children)

Sure. Once you start blaming people, I think some other questions should be allowed, too...

For example: Isn't it negligent to give a loaded handgun to a 14 yo teen?

And while computer games, or chatbots can be linked, that's rarely the underlying issue, or sole issue to blame. Sounds to me like the debate on violent computer games in the early 2000s, when lots of parents thought playing CounterStrike would make us murder people. Just that it's AI chatbots now. (Okay, maybe that's a stretch...) I can relate to loneliness and growing up and being a teen isn't easy.

[–] [email protected] 5 points 3 weeks ago (1 children)

When a kid dies it's natural for parents to want to seek someone to blame but sometimes there not a lot you can do. However sad it is and it's definitely sad you just need to accept it as something that happened, isn't always anyone's fault.

There is a bare minimum one could do and I would have thought that gun safety would be covered under that bare minimum. Especially once they start throwing around accusations at other people.

[–] [email protected] 2 points 2 weeks ago

I really think a gun safety class should be required to own a firearm. However, I also see how that would violate the second amendment (by making it harder for those of lesser means to exercise their right to own a weapon because they do not have the same resources available to take a class).

I think that as long as we have the second amendment, we should be offering taxpayer-funded firearm safety courses in all states. And requiring them.

[–] [email protected] 4 points 3 weeks ago (2 children)

I understand what you mean about the comparison between AI chatbots and video games (or whatever the moral panic du jour is), but I think they're very much not the same. To a young teen, no matter how "immersive" the game is, it's still just a game. They may rage against other players, they may become obsessed with playing, but as I said they're still going to see it as a game.

An AI chatbot who is a troubled teen's "best friend" is different and no matter how many warnings are slapped on the interface, it's going to feel much more "real" to that kid than any game. They're going to unload every ounce of angst into that thing, and by defaulting to "keep them engaged", that chatbot is either going to ignore stuff it shouldn't or encourage them in ways that it shouldn't. It's obvious there's no real guardrails in this instance, as if he was talking about being suicidal, some red flags should've popped up.

Yes the parents shouldn't have allowed him such unfettered access, yes they shouldn't have had a loaded gun that he had access to, but a simple "This is all for funsies" warning on the interface isn't enough to stop this from happening again. Some really troubled adults are using these things as defacto therapists and that's bad too. But I'd be happier if lawmakers were much more worried about kids having access to this stuff than accessing "adult sites".

[–] [email protected] 1 points 2 weeks ago (1 children)

I get what you are saying, and somewhat agree. However this really reads exactly like a decade ago when it was “teens can’t tell the difference between killing someone in a video game vs real life”

[–] [email protected] 1 points 2 weeks ago

I probably didn't explain well enough. Consuming media (books, TV, film, online content, and video games) is predominantly a passive experience. Obviously video games less so, but all in all, they only "adapt" within the guardrails of gameplay. These AI chatbots however are different in their very formlessness - they're only programmed to maintain engagement and rely on the LLM training to maintain an illusion of "realness". And because they were trained on all sorts of human interactions, they're very good at that.

Humans are unique in how we continually anthropomorphize tons of not only inert, lifeless things (think of someone alternating between swearing at and pleading to a car that won't start) but abstract ideals (even scientists often speak of evolution "choosing" specific traits). Given all of that, I don't think it's unreasonable to be worried about a teen with a still developing prefrontal cortex and who is in the midst of working on understanding social dynamics and peer relationships to embue an AI chatbot with far more "humanity" than is warranted. Humans seem to have an anthropomorphic bias in how we relate to the world - we are the primary yardstick we use to measure and relate everything around us, and things like AI chatbots exploit that to maximum effect. Hell, the whole reason the site mentioned in the article exists is that this approach is extraordinarily effective.

So while I understand that on a cursory look, someone objecting to it comes across as a sad example of yet another moral panic, I truly believe this is different. For one, we've never had access to such a lively psychological mirror before and it's untested waters; and two, this isn't some objection on some imagined slight against a "moral authority" but based in the scientific understanding of specifically teen brains and their demonstrated fragility in certain areas while still under development.

[–] [email protected] 1 points 3 weeks ago* (last edited 3 weeks ago)

The warning is a joke. Alike printing "Smoking kills" on ciragette packages, just that even less people care. And I doubt that sentence is going to change anything in a legal battle.

I'm like half convinced.
I think the dynamics are the same as with other things. Sometimes we like to escape reality. That can be done by reading books, watching TV or playing computer games. Or social media or watching some twitch streamer daily. I believe the latter is called parasocial interaction. It becomes an issue once done excessively. Or the lines get blurry. Or mental issues get into the mix.

Certainly AI chatbots are more convincing than some regular old book. (Allegedly already in 1775 we had young people commit copycat suicide after reading Goethe's "The Sorrows of Young Werther", so it's not a new topic.) But an AI can get to you and exploit your individual needs and wants and really get to you. I read the effects are currently being studied. I skimmed some long papers, but it seems we don't have a final answer, yet. About what that does psychologically.

I've tried roleplaying with AI. And I've also tried loading those characters like the famous AI therapist and pop culture characters. For me, it's pretty clear it's just a game. All of the interaction happens through text on the screen, I can't touch them or talk to them verbally (yet). I've heard from some other people here on Lemmy, they don't like the experience that is alike some pen and paper game... And I know how these things work, and that my hypothetical AI girlfriend is just a dream. So I don't think I'm at harm. And I don't think lots of other people are. But... obviously some people are. This isn't the first article about people getting harmed. And I can see how you wouldn't be able to defend yourself against some chatbot if you have serious issues or a mental condition.

I still think we can't skip all the other factors at play. We need to address (teenage) loneliness, guns and not having a caring and healthy social/human environment. A proper education and giving people some knowledge how these things work and what they are, would certainly help, too. It's always the same story. We leave people alone, without education, without a healthy social environment, the people close to them miss how much they're struggling, there is guns laying on the desk...

And after the inevitable happened, we don't address any of that. But completely focus on one topic that's more symptom then cause. And that's why I'm annoyed by the article.

(But I get there is some risk specific to chatbots that goes beyond other things. And it's probably not just symptom, but also contributing factor. We'd need more non-sensationalist information to judge...)