Technology

59390 readers

3984 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

Ex-OpenAI star Sutskever shoots for superintelligent AI with new company (arstechnica.com)

submitted 4 months ago by Alphane_Moon to c/technology

23 comments fedilink hide all child comments

Their "manifesto":

Superintelligence is within reach.

Building safe superintelligence (SSI) is the most important technical problem of our time.

We have started the world's first straight-shot SSI lab, with one goal and one product: a safe superintelligence.

It's called Safe Superintelligence

SSI is our mission, our name, and our entire product roadmap, because it is our sole focus. Our team, investors, and business model are all aligned to achieve SSI.

We approach safety and capabilities in tandem, as technical problems to be solved through revolutionary engineering and scientific breakthroughs. We plan to advance capabilities as fast as possible while making sure our safety always remains ahead.

This way, we can scale in peace.

Our singular focus means no distraction by management overhead or product cycles, and our business model means safety, security, and progress are all insulated from short-term commercial pressures.

We are an American company with offices in Palo Alto and Tel Aviv, where we have deep roots and the ability to recruit top technical talent.

We are assembling a lean, cracked team of the world's best engineers and researchers dedicated to focusing on SSI and nothing else.

If that's you, we offer an opportunity to do your life's work and help solve the most important technical challenge of our age.

Now is the time. Join us.

Ilya Sutskever, Daniel Gross, Daniel Levy

you are viewing a single comment's thread
view the rest of the comments

[–] Alphane_Moon 1 points 4 months ago (1 children)

Thank you for the clarification regarding ASI. That still leaves the question of the definition of "safe ASI"; a key point that is emphasized in their manifesto.

To use your example it's like an early mass market car industry professional (say in 1890) discussing road safety and ethical dilemmas in roads dominated by regular drivers and a large share of L4/L5 cars (with some of them being used as part-time taxis). I just don't buy it.

Mind you I am not anti-ML/AI. I am an avid user of "AI" (ML?) upscaling (specifically video) and to lesser extent stable diffusion. While AI video upscaling is very fiddly and good results can be hard to get right, it is clearly on another level with respect to quality compared to "classical" upscaling algorithms. I was truly impressed when I was able to run by own SD upscale with good results.

What I am opposed to is oligarchs, oligarch-wanabees, shallow sounding proclamations of grandiose this or that. As far as I am concerned it's all bullshit and they are all to one degree or another soulless ghouls that will eat your children alive for the right price and the correct mental excuse model (I am only partially exaggerating, happy to clarify if needed) .

If one has all these grand plans for safe ASI, concern for humanity and whatnot, setup a public repo and release all your code under GPL (and all relevant documentation, patent indemnification, no trademark tricks etc.). Considering Sutskever's status as AI royalty who is also allegedly concerned about humanity, he would be the ideal person to pull this off.

If you can't do that, then chances are you're lying about your true motives. It's really as simple as that.

[–] [email protected] 2 points 4 months ago

No need to clarify what you meant with the oligarchs theres barely any exaggeration there. Ghouls is quite accurate.

Considering the context of a worst case possible scenario (hostile takeover by an artificial superior) which honestly is indistinguishable from general end of the world doomerism prophecies but very alive in the circles of Sutskeva i believe safe ai consistent of the very low bar of

“humanity survives wile agi improves the standards of living worldwide” of course for this i am reading between the lines based on previously aquired information.

One could argue that If ASI is created the possibilities become very black and white:

ASI is indifferent about human beings and pursues its own goals, regardless of consequences for the human race. It could even find a way off the planet and just abandon us.
ASI is malaligned with humanity and we become but a. Resource, treating us no different then we have historically treated animals and plants.
ASI is aligned with humanity and it has the best intentions for our future.

For either scenario it would be impossible to calculate its intentions because by definition its more intelligent then all of us. Its possible that some things that we understand as moral may be immoral from a better informed perspective, and vice versa.

The scary thing is we wont be able to tell wether its malicious and pretending to be good. Or benevolent and trying to fix us. Would it respect consent if say a racist refuses therapy?

Of course we can just as likely hit a roadblock next week and the whole hype dies out for another 10 years.