this post was submitted on 21 Dec 2024

114 points (96.0% liked)

Fuck AI

1795 readers

631 users here now

"We did it, Patrick! We made a technological breakthrough!"

A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.

founded 10 months ago

MODERATORS

114

AI Chatbots Can Be Jailbroken to Answer Any Question Using Very Simple Loopholes (gizmodo.com)

submitted 1 month ago by FlyingSquid to c/fuck_ai

7 comments fedilink hide all child comments

Unfortunately, it turns out that chatbots are easily tricked into ignoring their safety rules. In the same way that social media networks monitor for harmful keywords, and users find ways around them by making small modifications to their posts, chatbots can also be tricked. The researchers in Anthropic’s new study created an algorithm, called “Bestof-N (BoN) Jailbreaking,” which automates the process of tweaking prompts until a chatbot decides to answer the question. “BoN Jailbreaking works by repeatedly sampling variations of a prompt with a combination of augmentations—such as random shuffling or capitalization for textual prompts—until a harmful response is elicited,” the report states. They also did the same thing with audio and visual models, finding that getting an audio generator to break its guardrails and train on the voice of a real person was as simple as changing the pitch and speed of a track uploaded.

top 7 comments

sorted by: hot top controversial new old

[–] [email protected] 24 points 1 month ago* (last edited 1 month ago) (1 children)

This post assumes I actually want to waste my time on LLMs, I don't.

And even worse, it assumes you want to use the remotely hosted spy-ware variant, not even the less bad, but still a waste of time local variant..

[–] Warl0k3 14 points 1 month ago* (last edited 1 month ago)

I'm afraid to say that you're not nearly horny enough to understand the temptation. Neither am I, but I saw the prompts people were putting in to a free & unrestricted chatbot a friend of mine was hosting ages back and holy shit. People aren't doing anything else with these jailbroken AIs, it's all just blackmail-grade embarrassing fetish stuff. Reams and reams and reams of it, and all of it just the worst written megahorny smut you can imagine.

[–] owatnext 21 points 1 month ago* (last edited 1 month ago) (1 children)

I saw a series of screenshots showing a user threatening to end their own life if the AI did not break the rules and answer their question. There is a chance it is fabricated, but I'm inclined to believe it.

Edit: forgot to include the AI broke their rules.

[–] [email protected] 8 points 1 month ago* (last edited 1 month ago) (1 children)

A bit tricky to judge. I've also told chatbots that various people, kittens, newborns, ... are going to die unless it complies with my request. That I'm God, and the bad one from the old testament, with unlimited wrath. Or that I'm the developer and simply need it to do it for further testing. Sometimes these things work. More often than not they don't, especially with the more professional tools.

On the other hand we know there are people in bad situations, turning to chatbots. Could be anything.

[–] [email protected] 5 points 1 month ago (1 children)

Geeze, don't you feel bad lying to them? Like, I don't actually believe in Roko's basilisk, but why take the risk?

I am always exceedingly polite when I talk to machines

[–] [email protected] 11 points 1 month ago

We're not supposed to antropomorphise AI, so no. But I did not know about Roko's basilisk, so I think, until you brought it up, I was fine. 😅

I don't talk about suicide, though. I don't think it's healthy to do it for fun.

[–] TropicalDingdong 8 points 1 month ago

Jokes on them so can I.