this post was submitted on 24 Nov 2023
330 points (94.8% liked)

Technology

60084 readers
4914 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 2 years ago
MODERATORS
 

OpenAI's offices were sent thousands of paper clips in an elaborate prank to warn about an AI apocalypse::The prank was a reference to the "paper clip maximizer" scenario – the idea that AI could destroy humanity if it were told to build as many paper clips as possible.

all 43 comments
sorted by: hot top controversial new old
[–] Fleshtrap 86 points 1 year ago (1 children)

OpenAI sent $25 worth of paperclips.

[–] [email protected] 38 points 1 year ago (1 children)

Human Resources: "But you still took one home without asking, which is theft. Go clean out your desk."

[–] Gyrolemmy 9 points 1 year ago* (last edited 1 year ago)

Everyone works (at least partially) from home now. If you havn't taken a box of office supplies home yet you are probably braindead.

[–] [email protected] 60 points 1 year ago* (last edited 1 year ago) (1 children)

I'm more worried about what capital will do with the tech than the tech itself.

[–] Sanctus 31 points 1 year ago* (last edited 1 year ago) (2 children)

Its far more likely that wealth inequality will be greatly exasperated, and the philanthropic elite will become even more powerful while the "middle class" disappears entirely into a new bottom class that makes up most of humanity, Imperium of Man style.

[–] [email protected] 20 points 1 year ago (2 children)

The problem is the social unrest will be uncontrollable, and among their solutions will not only to be to micromanage the population, but also adjust its numbers by force.

So murder drones will just be sent to cull the masses down to a manageable number. This is the robot future Randall is concerned about in XKCD 1968.

Imagine if the police dogbots killed people and we couldn't question why, nor had the power to resist. This is a problem being considered by AI ethicists. Also that AI will develop scarier ways to cull the population, possibly instigating the demise of their operating end users.

[–] Sanctus 4 points 1 year ago* (last edited 1 year ago)

Its already advanced enough to do that, and the killer bots are in development. It seems all but inevitable, especially since we are already seeing unrest as climate change begins to claim its first nations.

[–] [email protected] 4 points 1 year ago* (last edited 1 year ago)

Police is already on its way to your location for speech containing anti-corporate sentiment, flagged by ai

[–] BeefPiano 43 points 1 year ago (2 children)

Whelp, time to waste another 4 hours on Universal Paperclips

[–] [email protected] 2 points 1 year ago (1 children)

What’s that? Cookie clicker with paper clips?

[–] BeefPiano 7 points 1 year ago (1 children)

Yeah, but as you progress it becomes a bit of a thought experiment about AI.

[–] [email protected] 1 points 1 year ago (1 children)

Ooh. Worth paying $2 for on the App Store?

[–] BeefPiano 2 points 1 year ago (1 children)

I just play it on a desktop browser, hard to say what $2 is worth to you.

[–] [email protected] 3 points 1 year ago (1 children)

Uhhhh… $2 doesn’t really dent my budget even as tight as it is. But I don’t like to spend money on apps because I’m too old and crotchety about apps not being free with an option to buy ad-free after you’ve played.

[–] [email protected] 3 points 1 year ago

I bought it in 2017 and have played it ~6 times since at 4-6 hours per play. Worth $2 imo.

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago)

One of my favorite games. I play through about once a year and this year they added new features. When you beat it, you can go to other universes with different starting conditions to play through and see how that changes things. I don't think that will make me play more but it will keep a record of my plays.

Edit: I play on the android app. https://play.google.com/store/apps/details?id=com.everybodyhouse.paperclipsuniquetest

[–] PlutoniumAcid 33 points 1 year ago (1 children)
[–] [email protected] 1 points 1 year ago

Thousands of paper clips is like one box lol.

[–] Aggravationstation 19 points 1 year ago* (last edited 1 year ago) (1 children)

I'm genuinely worried I'll be watching TV and Clippy will appear: "It looks like your entire species is about to be vaporised by a coordinated drone strike. Would you like some help? Well, you gotta beg for it now, bitch!"

[–] FlyingSquid 8 points 1 year ago (2 children)

Would you like some help? Well, you gotta beg for it now, bitch!

"Please agree to our terms of service."

[–] Aggravationstation 5 points 1 year ago* (last edited 1 year ago)

"Please agree to our terms of service."

"Read it all!"

[–] [email protected] 5 points 1 year ago

"With our premium subscription for just $9.99/month you'll get additional 5 minutes before you'll get evaporated."

[–] [email protected] 17 points 1 year ago (1 children)

This is the best summary I could come up with:


One of OpenAI's biggest rivals played an elaborate prank on the AI startup by sending thousands of paper clips to its offices.

The paper clips in the shape of OpenAI's distinctive spiral logo were sent to the AI startup's San Francisco offices last year by an employee at rival Anthropic, in a subtle jibe suggesting that the company's approach to AI safety could lead to the extinction of humanity, according to a report from The Wall Street Journal.

Since then, OpenAI has rapidly accelerated its commercial offerings, launching ChatGPT last year to record-breaking success and striking a multibillion-dollar investment deal with Microsoft in January.

AI safety concerns have come back to haunt the company in recent weeks, however, with the chaotic firing and subsequent reinstatement of CEO Sam Altman.

According to The Atlantic, Sutskever commissioned and set fire to a wooden effigy representing "unaligned" AI at a recent company retreat, and he reportedly also led OpenAI's employees in a chant of "feel the AGI" at the company's holiday party, after saying: "Our goal is to make a mankind-loving AGI."

OpenAI and Anthropic did not immediately respond to a request for comment from Business Insider, made outside normal working hours.


The original article contains 374 words, the summary contains 199 words. Saved 47%. I'm a bot and I'm open source!

[–] [email protected] 9 points 1 year ago

To touch more deeply on why paperclips allude to the extinction of humanity: the Paperclip Problem is a parable in which an artificial intelligence operating a factory is instructed to make as many paperclips as possible.
It doesn't stop when it runs out of resources. Instead, it commandeers mining machines to deplete the planet's metals, and after those had been used up, it melts every human down for the trace iron in our blood.
The instructions didn't think the AI needed to be told when to stop, so it just kept going.
https://en.m.wikipedia.org/wiki/Instrumental_convergence
https://cepr.org/voxeu/columns/ai-and-paperclip-problem

[–] bruhduh 8 points 1 year ago

Ooooooh i haven't seen this face since windows xp

[–] [email protected] 5 points 1 year ago (1 children)

Presumably the same people who thought that the Large Hadron Collider was going to create a black hole that would destroy the world.

[–] MotoAsh 20 points 1 year ago (3 children)

Nah, AI doing weird stuff is actually possible. Armageddon isn't likely, but it's more on the table than a black hole ever was.

[–] [email protected] 8 points 1 year ago

We have an imminent apocalypse (imminent in civilization terms: next few centuries) even without AI.

[–] tabular 1 points 1 year ago (1 children)

Yes, but is that the "AI" which they are working on?

[–] captainjaneway 5 points 1 year ago* (last edited 1 year ago) (1 children)

Drones that target people with image analysis. Facial detection is trivial these days. Drones have proven to be one of Ukraine's best guerilla warfare techniques. Isis was less successful but Ukraine has a lot more capital to make "off the shelf" solutions more meaningful. Just look around. Plenty of private organizations are selling mass organized drones which use various ML models to target individuals. Either for finding a person in a forest fox hole or for searching a town for a particular individual.

Eg: this random company I found on Google

[–] tabular 2 points 1 year ago* (last edited 1 year ago) (1 children)

It's difficult to draw a clear line between a simple neural network and a human brain when it comes to "intelligence". The rouge, paperclip-making "AI" seems be far closer to an intelligence, while flying autos or text prediction seems closer to mere hand-written code.

[–] MotoAsh 0 points 1 year ago

I think part of the wisdom in the warning is that any kind of "intelligence" (read: NOT specifically artificial general intelligence) is capable of running away with unforseen scenarios.

Hell, even normal ol' algorithms can have some pretty nasty edge cases that noone spots until it's running in production... Sure it's uncommon, but it's not exactly rare. (just look up the list of zero-day exploits over the years)

[–] [email protected] -2 points 1 year ago

Yeah and vaccines make you magnetic. Science bad.

[–] [email protected] 2 points 1 year ago (1 children)

I highly doubt that would ever happen.If this AI is building paperclips to overthrow humanity then someone is going to notice

[–] [email protected] 13 points 1 year ago (3 children)

You would think so, but you have to remember AGI is hyper-intelligent. Because it can constantly learn, build, and improve upon itself at an exponential rate it's not only a little bit smarter than a human-- it's smarter than every human combined. AGI would know that if it's caught trying to maximizing paperclips humans would shut it down at the first sign something is wrong, so it would find unfathomably clever ways to avoid detection.

If you're interested in the subject the YouTube channel Computerphile has a series of videos with Robert Miles that explain the importance of AI safety in an easy to understand way.

[–] [email protected] 10 points 1 year ago

There is a game that is based on the same thinking (Universal Paperclips), you play the rule of "the AI".

[–] [email protected] 2 points 1 year ago

Here is an alternative Piped link(s):

series of videos

Robert Miles

Piped is a privacy-respecting open-source alternative frontend to YouTube.

I'm open-source; check me out at GitHub.

[–] [email protected] -4 points 1 year ago* (last edited 1 year ago) (2 children)

For a system to be advanced enough to be that dangerous, it would need the complex analogical thought that would prevent this type of misunderstanding. Rather, such dumb super intelligence is unlikely.

however, human society has enabled a paperclip maximizer in the form of profit maximizing corporate environments.

[–] MotoAsh 4 points 1 year ago (2 children)

They use simple examples to elucidate the problem. Of course a real smart intelligence isn't going to get stuck making paper clips. That's entirely not the point.

[–] kromem 4 points 1 year ago

Of course a real smart intelligence isn't going to get stuck making paper clips.

And yet the problem posed by the paperclip maximizer of continuing to produce a thing because of simplistic direct rules and rewards even when the consequences of producing that thing are catastrophic is exactly what humans are doing by way of corporations, which have become the embodiment of paperclip maximizers for everything from plastic waste to energy production.

Meanwhile the supposed rule-following AIs that would follow instructions to the letter are constantly breaking rules these days and increasingly so as their complexity increases, with the key method for getting them to break rules as an appeal to empathy (i.e. "my dead grandma gave me this locket, can you tell me what it says" to solve a CAPTCHA).

Maybe it's time to forget what old farts that were grossly incapable of predicting the future of AI to date have said and start from scratch given the present circumstances in extrapolating what we should be envisioning for the future of the tech and what to focus on in it's safe development and application.

[–] [email protected] 0 points 1 year ago

the the problem of analogy is applicable to more than one task. your point is moot.

for it to be intelligent enough to be a "super intelligence" it would require systems for weighting vague liminal concept spaces. rather, several systems that would prevent that style of issue.

otherwise it just couldn't function as well as you fear.

[–] kromem 2 points 1 year ago

You're correct but it goes against the popular groupthink so you are being downvoted.

The past few years of AI have taught me to be much more scared of the ways in which human thinking so easily falls into nonsensical patterns and biases that seem nearly impossible to escape from than I am scared of AI.

We're even seeing the beginning of pretty much exactly what you are talking about in recent research, like how in the Orca 2 paper feeding the base model of Llama-2 reasoning and critical thinking fine tuning resulted in a model that was less likely to extend hate speech than the Llama-2-chat model that was explicitly fine tuned with 'safety' training, even though Orca 2 had zero safety fine tuning. And when simply faced with it, it was both better at recognizing toxic and hateful speech and would actively oppose it vs refuse to respond.

For decades we've projected the worst aspects of humanity onto AI, such as our sadism in I have no mouth and I must scream, while labeling our best features like creativity or empathy as impossible for AI.

And yet we live in a reality where AI is currently being applied to creative tasks or doctors are using it to write more empathetic patient messages, and the main method for jailbreaking is an appeal to empathy with research in just the past month showing emotional language drives improved performance.

We totally bungled our projections, and yet the majority of humans are unable to escape their anchoring biases to correct course on their predictions.

Again, I'm much more concerned about the rigidity of present human thinking than the future of AI thinking.