Technology

59708 readers

5537 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

280

Poisoned AI went rogue during training and couldn't be taught to behave again in 'legitimately scary' study (www.livescience.com)

submitted 10 months ago by L4s to c/technology

56 comments fedilink hide all child comments

Poisoned AI went rogue during training and couldn't be taught to behave again in 'legitimately scary' study::AI researchers found that widely used safety training techniques failed to remove malicious behavior from large language models — and one technique even backfired, teaching the AI to recognize its triggers and better hide its bad behavior from the researchers.

you are viewing a single comment's thread
view the rest of the comments

[–] irotsoma 10 points 10 months ago (1 children)

The problem is that these LLMs are built with the wrong driving motivator. They're driven to find one right way whereas the reality is that there is rarely a single right way and computers don't need to have a single right way like humans tend towards. The LLM shouldn't be driven to be "right" in its learning model. It should be trained on known good data only as a base, and then given the other data to serve context rather than allowing that data to modify the underlying system. This is more like how biological creatures work in teaching a child to be "good" or "evil" and to know the basic things needed to survive and serve their purpose, and then the stuff they learn in adulthood serves to help them apply those base concepts to the world.

[–] [email protected] 3 points 10 months ago

At the same time, they don't really behave that much differently from some humans that have been sucked down the path of various conspiracy theories. For a lot of those, the first "lesson" is 'everyone else is wrong and have been deceived or are trying to trick you, trust nobody but us'. From there, some people end up going down the rabbit-hole to become "Sovereign Citizens" or storm congress.