Technology

60078 readers

4458 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 2 years ago

MODERATORS

459

The GPT Era Is Already Ending (www.theatlantic.com)

submitted 2 weeks ago* (last edited 2 weeks ago) by [email protected] to c/technology

158 comments fedilink hide all child comments

If this is the way to superintelligence, it remains a bizarre one. “This is back to a million monkeys typing for a million years generating the works of Shakespeare,” Emily Bender told me. But OpenAI’s technology effectively crunches those years down to seconds. A company blog boasts that an o1 model scored better than most humans on a recent coding test that allowed participants to submit 50 possible solutions to each problem—but only when o1 was allowed 10,000 submissions instead. No human could come up with that many possibilities in a reasonable length of time, which is exactly the point. To OpenAI, unlimited time and resources are an advantage that its hardware-grounded models have over biology. Not even two weeks after the launch of the o1 preview, the start-up presented plans to build data centers that would each require the power generated by approximately five large nuclear reactors, enough for almost 3 million homes.

https://archive.is/xUJMG

you are viewing a single comment's thread
view the rest of the comments

[–] dustyData 4 points 2 weeks ago (3 children)

Almost surely, I'm quoting mathematicians. Because an infinite anything also includes events that exist but with probability zero. So, sure, the probability is 100% (more accurately, it tends to 1 as the number of monkeys approach infinite) but that doesn't mean it will occur. Just like 0% doesn't mean it won't, because, well, infinity.

Calculus is a bitch.

[–] ylph 1 points 2 weeks ago (2 children)

Ok, this is interesting, so thanks for pointing me to it. I think it's still safe to say "almost surely an infinite number of monkeys" as opposed to "almost surely at least one", since the probability of both cases is still 100% (can their probability even be quantitatively compared ? is one 100% more likely than another 100% in this case ?)

The idea that something with probability of 0 can happen in an infinite set is still a bit of a mindfuck - although I understand why this is necessary (e.g. picking a random marble from an infinite set of marbles where 1 is blue and all others red for example - the probability of picking the blue marble is 0, but it is obviously still possible)

[–] dustyData 1 points 2 weeks ago (1 children)

Indeed, the formal definition actually doesn't specify how many monkeys will write what given an infinite number of monkeys, it's unknowable (that's just how probability is). We just know that it will almost surely happen, but that doesn't mean it will happen an infinite amount of occurrences.

The infinite amount of time version is just as vague, one monkey will almost surely type a specific thing, eventually, given infinite time to type it. This is because when you throw infinites at probability, all probabilities tend to 1. Given an infinite amount of time, all things that can happen, will almost surely happen, eventually.

[–] ylph 1 points 2 weeks ago* (last edited 2 weeks ago)

But your citation gives both statements:

"In fact, the monkey would almost surely type every possible finite text an infinite number of times."

and

"The theorem can be generalized to state that any sequence of events that has a non-zero probability of happening will almost certainly occur an infinite number of times, given an infinite amount of time or a universe that is infinite in size."

So when you say the number of times is "unknowable" the actual answer is "almost surely an infinite number of times" no ? Since the probability of that can be calculated as 100%. The mindfuck part is that it is still possible that no monkey at all will type a particular text, even though the probability of that is 0.

The probability that only 2 monkeys will type the text is also still 0, same as 3 monkeys, 4 monkeys, etc. - in fact the probability of any specific finite number of monkeys only typing out the text is still 0 - only the probability of an infinite number of monkeys typing it out is 100% (the probabilities of all possible outcomes, even when infinite, have to sum up to 1 after all)

We just know that it will almost surely happen, but that doesn't mean it will happen an infinite amount of occurrences.

Basically, if we know "it will almost surely happen" then we also know just as surely (p=1) that it will also happen an infinite number of times (but it might also never happen, although with p=0)