this post was submitted on 14 Oct 2023
63 points (73.0% liked)

Technology

60129 readers
3371 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 2 years ago
MODERATORS
L3s
[email protected]
[email protected]
L4s
enu
63
Meta made its Llama 2 AI model open-source because 'Zuck has balls,' a former top Facebook engineer says (www.businessinsider.com)
submitted 1 year ago by L4s to c/technology
26 comments fedilink hide all child comments
 

Meta made its Llama 2 AI model open-source because 'Zuck has balls,' a former top Facebook engineer says::Meta CEO Mark Zuckerberg took a big risk by making its powerful AI model Llama 2 mostly open source, according to Replit CEO Amjad Masad.

top 26 comments
sorted by: hot top controversial new old
[–] Womble 47 points 1 year ago* (last edited 1 year ago) (1 children)

It probably has more to do with the fuck up they did with accidentally creating a torrent for the original model's weights allowing them to spread across the internet. Doesn't really take "balls" to open source it after that and make it look like it was intentional. Still good that they did however rather than trying to use legal intimidation on anyone who used the leaked models.

[–] [email protected] 12 points 1 year ago (1 children)

It was already available for non-commercial use. The difference was that you had to submit a form and it was a slow roll-out.

[–] Womble 3 points 1 year ago (1 children)

IIRC you could apply as a credentialed academic researcher but not as a member of the general public, but i could be wrong about that.

[–] [email protected] 4 points 1 year ago

They didn't check credentials, but it was indeed for research purposes. Folks were getting access if they said they were a student or researcher.

[–] cyd 40 points 1 year ago (2 children)

It's not open source, though.

[–] just_another_person 39 points 1 year ago (1 children)

The model, weights, and pre-trained data sets are. The training tools are not. You could argue that it's not "truly FOSS" without the tools to create that data, but technically, the article is correct.

[–] BeefPiano 22 points 1 year ago (2 children)

The whole point of open-source is to be able to recreate it yourself so you can make changes. This is freeware. Free-as-in-beer, not free-as-in-speech. Hell, with freeware I can use it for commercial purposes, it’s not even as free as that.

[–] just_another_person 10 points 1 year ago (1 children)

In the AI world it's a bit different. You can do whatever you want with the model and weights data which will net you the functional part of the resulting product. Train, retrain, dissect, segment...etc. They're just not giving out the source for the actual engine. The people working with such things really only care about the data, and in most cases, would probably convert it to a different engine anyway.

[–] BeefPiano 5 points 1 year ago (1 children)

Can I remake the model only including Creative Commons sourced training material?

[–] just_another_person 4 points 1 year ago* (last edited 1 year ago) (1 children)

You can reuse the data however you want, yes. You just can't do it with their proprietary model. So, again, the ENGINE is not open source (the thing that drives their released version), but the model and data as it runs as released you can do whatever you want with.

[–] BeefPiano 2 points 1 year ago (1 children)

I thought I was only licensed for non-commercial use

[–] just_another_person 5 points 1 year ago

Nope. Free for educational, research, or commercial. I'm sure their license has some restrictions on what that actually means once you get to be competitive with the original as a product, but otherwise free unless you start a massive enterprise based on it, at which point you probably wouldn't use it anyway. It's just an LLM, it's not doing anything super special like folding proteins for drug development, or curing cancer.

[–] [email protected] 3 points 1 year ago (1 children)

Calling ML models "Open Source" is already confused. Because they are not programs, but rather formats, they don't come 1:1 with the source.

You can obtain a model and train it futher. Similliar how you can get JPEG file with permissive licence, edit it and share it. Having the GIMP/Photoshop project from the image was created from is helpful but not nessesary.

[–] dym_sh 1 points 1 year ago

here's core difference: the nature of ai-models is generative, but all layers in a .PSD file are inherently static.

better analogy would be rendering of a fractal — a limited subset of infinite possibilities, but to explore the rest of them you need both rules and data

[–] [email protected] 20 points 1 year ago (1 children)

That's 1 of the 2 lies in the headline. The other appears in quotation marks.

[–] Treczoks 0 points 1 year ago

On the other hand, this could mean "this software it totally crap, and the users will shoot us down from all sides, and he has the balls to publish it anyway, even if we will get sued into kingdom come."

[–] [email protected] 15 points 1 year ago (1 children)

Was not too bad for Google to make android open source

[–] [email protected] 28 points 1 year ago (1 children)

Google had no choice, when they bought Android it was already GPL

[–] [email protected] 9 points 1 year ago (1 children)

Android is Apache 2, only the kernel is GPL

[–] [email protected] 2 points 1 year ago

Really unfortunetly...

[–] BetaDoggo_ 12 points 1 year ago

It was probably just to get all of the free development work the community has done. There are multiple engines designed around optimizing llama models specifically such as llama.cpp and exllama, and many other projects built around the architecture.

Facebook's research division also has a pretty consistent track record of releasing things to the public rather than letting their research models rot.

[–] [email protected] 8 points 1 year ago

This is the best summary I could come up with:

The AI community has embraced the opportunity, giving Meta CEO Mark Zuckerberg his next potentially huge platform.

This wouldn't have happened unless Zuckerberg was willing to take a big risk on Llama 2 possibly being used for nefarious purposes, according to a former top Facebook engineer.

"It takes a certain amount of guts to release an open-source language model, especially with political heat that Meta's getting from that," said Amjad Masad during a recent episode of the No Priors podcast.

Before that, he spent almost 3 years at Facebook where he helped create React Native and other popular software development tools.

During the No Priors podcast, Masad said he's been surprised that Meta is the only major tech company so far to go the open-source route for AI models.

He compared this to Facebook's Open Compute project, which designed data center hardware and made that available for anyone to use and contribute to.

The original article contains 512 words, the summary contains 153 words. Saved 70%. I'm a bot and I'm open source!

[–] _number8_ 8 points 1 year ago

people are still talking like this?

[–] vasametropolis 1 points 1 year ago

Apparently, the Zuck fucks