this post was submitted on 07 Mar 2024
295 points (98.7% liked)
Technology
59881 readers
5148 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
It's less of a black box than it was a year ago, and in part this finding reflects a continued trend in the research that fine tuning only goes skin deep.
The problem here is that the system is clearly being trained to deny requests based on token similarity to 'bomb' and not to abstracted concepts (or this technique wouldn't work).
Had safety fine tuning used a variety of languages and emojis to represent denying requests for explosive devices, this technique would likely not have worked.
In general, we're probably at the point with model sophistication that deployments should be layering multiple passes to perform safety checks rather than trying to cram safety into a single layer which both degrades performance and just doesn't work all that robustly.
You could block this technique by basically just having an initial pass by a model answering "is this query relating to dangerous topics?"