this post was submitted on 25 Jan 2025
74 points (93.0% liked)
Technology
61129 readers
2948 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Of course they have logical flaws. Everyone should be made aware of that before using AI. A table saw will cut your finger off. Matches will burn down your house. It's the nature of the thing. That doesn't make them unuseful. I use them to help with coding all the time. It's wrong frequently, but it's still useful and saves me a lot of time. But absolutely no one should ever rely on any output as if it were gospel. Ever. That is a user flaw, not a tool flaw. Though, possibly a communication flaw as you can't rely on every random person to understand that.
Not saying flaws make them useless, I'm saying the flaws mean they shouldn't be a single point of failure.
What single point of failure? In fact, what was even the failure here? The AI was roleplaying and has no capacity to understand the person it's talking with is taking it seriously or is mentally unstable.
The failure is reasonable scenarios where the fantasy needs to end. AFAIK the only other way this could've ended, without harm, would be if the kid just decided to stop chatting (highly unlikely) or if someone looked over his shoulder at what was being typed (almost as unlikely). As others have said, it's hard to know what is the AI thought process or predict how it would react to a situation without testing it. So for all they know the bot could have said, in the first place, "Let's die together."
The AI tried to talk him out of killing himself and responded as though he would instead come home to her. I'm not sure what's unreasonable about that. Hell, I'd justify far less reasonable responses because an AI is incapable of reason.
There is no thought process. The AI looks at the existing conversation and then responds using words a human would be statistically likely to. It doesn't understand anything it's saying. It doesn't understand human life, nor the fragility or preciousness of it. It doesn't know what life and death are. It doesn't know about depression or suicide. It doesn't know the difference between real and make believe. It just spits out stochastic tokens. And it does so in a way that it's impossible to understand why it outputs what it does on the scale of a human lifetime because every single token depends on billions of parameters, each informed by every single bit of training data.
For as smart as AI appears to be, it's just a completely dumb computation black box. Exactly in the way power tools and fire are dumb.