this post was submitted on 10 Jul 2023
354 points (91.7% liked)

Technology

59674 readers
4997 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
 

Which of the following sounds more reasonable?

  • I shouldn't have to pay for the content that I use to tune my LLM model and algorithm.

  • We shouldn't have to pay for the content we use to train and teach an AI.

By calling it AI, the corporations are able to advocate for a position that's blatantly pro corporate and anti writer/artist, and trick people into supporting it under the guise of a technological development.

top 50 comments
sorted by: hot top controversial new old
[–] pensivepangolin 98 points 1 year ago (6 children)

I think it’s the same reason the CEO’s of these corporations are clamoring about their own products being doomsday devices: it gives them massive power over crafting regulatory policy, thus letting them make sure it’s favorable to their business interests.

Even more frustrating when you realize, and feel free to correct me if I’m wrong, these new “AI” programs and LLMs aren’t really novel in terms of theoretical approach: the real revolution is the amount of computing power and data to throw at them.

[–] assassin_aragorn 59 points 1 year ago (6 children)

The funniest thing I've seen on this is the ChatGPT CEO, Altman, talking about how he's a bit afraid of what they've created and how it needs limitations -- and then when the EU begins to look at regulations, he immediately rejects the concept, to the point of threatening to leave the European market. It's incredibly transparent what they're doing.

Unfortunately I don't know enough about the technology to say if the algorithms and concepts themselves are novel, but without a doubt they couldn't exist without modern computing power capabilities.

[–] [email protected] 20 points 1 year ago* (last edited 1 year ago) (5 children)

I can tell for a fact that there's nothing new going on. Only the MASSIVE investment from Microsoft to allow them to train on an insane amount of data. I am no "expert" per se, but I've been studying and working with AI for over a decade - so feel free to judge my reply as you please

load more comments (5 replies)
[–] [email protected] 11 points 1 year ago (1 children)

The concepts themselves are some 30 years old, but storage capacity and processing speed have only recently reached a point where generative AI outperforms competing solutions.

But regarding the regulation thing, I don't know what was said or proposed, and this is just me playing devil's advocate: but could it be that the CEO simply doesn't agree with the specifics of the proposed regulations while still believing that some other, different kind of regulation should exist?

[–] [email protected] 15 points 1 year ago (1 children)

Certainly could be, but probably an optimistic take. Most likely they're just trying to do what corporations have been doing for ages, which is to weaponize government policy to prevent competition. They don't want restrictions that will materially impact their product, they want restrictions that will materially impact startups to make it more difficult for them to intrude on the established space.

[–] jumperalex 7 points 1 year ago

I think if you fed your response into ChatGPT and asked it to summarize in two words it would return,

"Regulatory Capture"

load more comments (4 replies)
[–] [email protected] 24 points 1 year ago (4 children)

Even more frustrating when you realize, and feel free to correct me if I’m wrong, these new “AI” programs and LLMs aren’t really novel in terms of theoretical approach: the real revolution is the amount of computing power and data to throw at them.

This is 100% true. LLMs, neural networks, markov chains, gradient descent, etc. etc. on down the line is nothing particularly new. They've collectively been studied academically for 30+ years. It's only recently that we've been able to throw huge amounts of data, computing capacity, and time to tweak said models to achieve results unthinkable 10-ish years ago.

There have been efficiencies, breakthroughs, tweaks, and changes over this time too, but that's just to be expected. But largely its just sheer raw size/scale that's just been achievable recently.

[–] FunnyUsername 10 points 1 year ago (4 children)

We all remember SmarterChild....right?

[–] FartsWithAnAccent 5 points 1 year ago (2 children)

No, I have clearly forgotten: What was that?

[–] [email protected] 9 points 1 year ago

AOL chatbot that did basic stuff.

Here you go.

If you ask Siri, "Do you sleep?" Siri will respond, "I don't need much sleep, but it's nice of you to ask." Meanwhile, if you asked SmarterChild the same question, he would respond, "No, but I dream. I dream of a better world. A world where man and machine can coexist in peace and happiness."

load more comments (1 replies)
[–] [email protected] 3 points 1 year ago
load more comments (2 replies)
[–] [email protected] 7 points 1 year ago (3 children)

LLMs aren’t really novel in terms of theoretical approach: the real revolution is the amount of computing power and data to throw at them.

This is 100% true. LLMs, neural networks, markov chains, gradient descent, etc. etc. on down the line is nothing particularly new. They’ve collectively been studied academically for 30+ years.

Well LLMs and particularly GPT and its competitors rely on Transformers, which is a relatively recent theoretical development in the machine learning field. Of course it's based in prior research, and maybe there even is prior art buried in some obscure paper or 404 link, but if that's your measure then there is no "novel theoretical approach" for anything, ever.

I mean I'll grant that the available input data and compute for machine learning has increased exponentially, and that's certainly an obvious factor in the improved output quality. But that's not all there is to the current "AI" summer, general scientific progress played a non-minor part as well.

In summary, I disagree on data/compute scale being the deciding factor here, it's deep learning architecture IMHO. The former didn't change that much over the last half decade, the latter did.

load more comments (3 replies)
[–] pensivepangolin 5 points 1 year ago

Okay, I’m glad I’m not too far off the mark then (I’m not an AI expert/it’s not my field of study).

I think this also points to/is a great example of another worrying trend: the consolidation of computing power in the hands of a few large companies. Without even factoring in the development of true AI/whether that can or will happen anytime soon, the LLMs really show off the massive scale of both computational power consolidation AMD data harvesting by only a very few entities. I’m guessing I’m not alone here in finding that increasingly concerning, particularly since a lot of development is driving towards surveillance applications.

[–] jumperalex 3 points 1 year ago

by that logic there was nothing novel about solid state transistors since they just did the same thing as vacuum tubes; no innovation there I guess. No new ideas came from finally having a way to pack cooler, less power hungry, smaller components together.

[–] [email protected] 8 points 1 year ago

LLMs are pretty novel. They are made possible by invention of the Transformer model, that operates significantly different compared to, say, RNN.

[–] assassinatedbyCIA 6 points 1 year ago (1 children)

It also plays into the hype cycle they’re trying to create. Saying you’ve made an AI is more likely to capture the attention of the masses then saying you have a LLM. Ditto that point for the existential doomerism that they ceo’s have. Saying your tech is so powerful that it might lead to humanity’s extinction does wonders in building hype.

[–] pensivepangolin 4 points 1 year ago

Agreed. And all you really need to do is browse any of the headlines from even respectable news outlets to see how well it’s working. It’s just article after article uncritically parroting whatever claims these CEO’s make at face value at least 50% of the time. It’s mind-numbing.

load more comments (2 replies)
[–] Iceblade02 34 points 1 year ago (4 children)

IMO content created by either AI or LLMs should have a special license and be considered AI public domain (unless they can prove that they own all content the AI was trained on). Commercial content made based on content marked with this license would be subject to a flat % tax that should be applied to the product price which would be earmarked for a fund distributing to human creators (coders, writers, musicians etc.).

[–] [email protected] 12 points 1 year ago (2 children)

I think the cleaner (and most likely) outcome is AI generated work is considered public domain, and since public domain content can already be edited and combined and arranged to create copyrighted content this would largely clear up the path for creators to use AI more prominently in their workflows

load more comments (2 replies)
load more comments (3 replies)
[–] Chocrates 32 points 1 year ago (1 children)

both sound the same to me IMO. Private companies scraping ostensibly public data to sell it. No matter how you word it they are trying to monetize stuff that is out in the open.

[–] Dran_Arcana 7 points 1 year ago* (last edited 1 year ago) (1 children)

I don't see why a single human should be able to profit off learning from others but a group of humans doing it for a company cannot. This is just how humanity advances at whatever scale.

[–] Chocrates 9 points 1 year ago (1 children)

I had a comment about the morality of it at first but I pulled it out. This is not an easy question to answer. Corporations gate keeping knowledge seems weird and dystopian but the knowledge is out there and they are just making connections between it. It also touches on copyright and fair use.

[–] Dran_Arcana 3 points 1 year ago

I agree it's much more complicated an issue than most people give it credit.

[–] [email protected] 25 points 1 year ago (3 children)

I'm not sure what you're trying to say here; LLMs are absolutely under the umbrella of AI, they are 100% a form of AI. They are not AGI/STRONG AI, but they are absolutely a form of AI. There's no "reframing" necessary.

No matter how you frame it, though, there's always going to be a battle between the entities that want to use a large amount of data for profit (corporations) and the people who produce said content.

[–] Silinde 8 points 1 year ago (2 children)

True, and this is the annoying thing about people unqualified to talk about AI giving their opinions online. People not involved in the industry hear "AI" and expect HAL-9000 or Ava from Ex Machina rather than the software that the weather service uses to predict if it will rain tomorrow, or the models your doctor uses to help determine your risk of Heart Disease.

This is compounded further when someone makes a video simplifying what an LLM is and mentioning that the latest models use it, which leads to the chimes of "bUt iT'S jUsT aN Llm BrO iTs nOt AI" and "ItS jUsT a LOaD oF DaTa aND aLGorItHMs, tHaTs NoT AI". A little bit of knowledge is a dangerous thing.

load more comments (2 replies)
[–] [email protected] 7 points 1 year ago* (last edited 1 year ago)

On the flip side, the same battle is also fought between giant corporations that amass intellectual property and the people who want to actually use that intellectual property instead of letting it sit in some patent troll's hoard until a lawsuit op presents itself. Seeing as there are quite a few reasonably decent open-source LLMs out there like Koala and Alpaca also training on data freely available on the Internet, I'm actually rooting for the AI companies in this case, in the hopes of establishing a disruptive precedent.

load more comments (1 replies)
[–] [email protected] 25 points 1 year ago (1 children)

I see it like this:

Our legal system has the concept of mechanical licensing. If your song exists, someone can demand the right to cover it and the law will favor them. The result of an LLM has less to do with your art that a cover of your song does.

There are plenty of cases of a cover eclipsing the original version of a song in popularity and yet I have never met a single person argue that we should get rid of the right to cover a song.

[–] [email protected] 15 points 1 year ago

Sure, you have the legal right to cover someone else's song without asking permission first, but you still have to pay them royalties afterwards, at fair market rates.

[–] itsnotlupus 24 points 1 year ago (2 children)

I'll note that there are plenty of models out there that aren't LLMs and that are also being trained on large datasets gathered from public sources.

Image generation models, music generation models, etc.
Heck, it doesn't even need to be about generation. Music recognition and image recognition models can also be trained on the same sort of datasets, and arguably come with similar IP right questions.

It's definitely a broader topic than just LLMs, and attempting to enumerate exhaustively the flavors of AIs/models/whatever that should be part of this discussion is fairly futile given the fast evolving nature of the field.

[–] [email protected] 4 points 1 year ago* (last edited 1 year ago) (2 children)

Still, all those models are, even conceptually, far removed frow AI. They would most properly be called Machine Learning Models (MLMs).

[–] itsnotlupus 5 points 1 year ago

The term AI was coined many decades ago to encompass a broad set of difficult problems, many of which have become less difficult over time.

There's a natural temptation to remove solved problems from the set of AI problems, so playing chess is no longer AI, diagnosing diseases through a set of expert system rules is no longer AI, processing natural language is no longer AI, and maybe training and using large models is no longer AI nowadays.

Maybe we do this because we view intelligence as a fundamentally magical property, and anything that has been fully described has necessarily lost all its magic in the process.
But that means that "AI" can never be used to label anything that actually exists, only to gesture broadly at the horizon of what might come.

load more comments (1 replies)
load more comments (1 replies)
[–] aezart 21 points 1 year ago (1 children)

If an LLM was trained on a single page of GPL code or a single piece of CC-BY art, the entire set of model weights and any outputs from the model must be licensed the same way. Otherwise this whole thing is just blatant license laundering.

[–] paperbenni 11 points 1 year ago

This depends on how transformative the act of encoding the data in an LLM is. If you have overfitting out the ass and the model can recite its training material verbatim then it's an illegal copy of the training material. If the model can only output content that would be considered transformative if a human with knowledge of the training data created it, then so is the model.

[–] [email protected] 15 points 1 year ago

In fairness, AI is a buzzword that came out well before LLMs. It's used to mean "tHe cOmpUtER cAn tHink!". We play against "AI" in games all the time, but they arent AI as we know it today.

ML (machine learning) is a more accurate descriptor but blah doesn't have the same pizzazz as AI does.

The larger issue is that innovation is sometimes done for innovation's sake. Profits gets mixed up there and a board has to show profits to shareholders and then you get VCs trying to "productize" and monetize everything.

What's more is there are only a handful of players in the AI space, but because they are giving API access to other companies, those companies are building more and more sketchy uses of that tech.

It wouldn't be a huge deal if LLMs trained on copywritten material and then gave the service away for free. As it stands, some LLMs are churning out work that could be protected under copywrite law by humans (AI work can't be copywritten under US law), and turning a profit.

I don't think "it was AI" will hold up in court though. May need to do some more innovation.

Also there are some LLMs being trained on public domain info, to avoid copywrite problems. But works go into the public domain after 70 years past the copywrite holder's death (disney being the biggest extender of that rule), so your AI will be a tad out dated in it's "knowledge".

[–] QHC 14 points 1 year ago

I think you are likely right, but it's more general than just about training costs. The term "AI" carries a ton of baggage, both good and bad.

To some extent, I think we also keep pushing back the boundary of what we consider "intelligence" as we learn more and better understand what we're creating. I wonder if every future tech generation will continue this cycle until/unless humanity actually does create a general artificial intelligence--every iteration getting slightly closer but still falling short of "true" AI, then being looked at as a disappointment and not worthy of the term anymore. Rinse and repeat.

[–] Zeth0s 13 points 1 year ago (1 children)

That's absolutely not correct. AI is a field of computer science/scientific computing built on the idea that some capabilities of biological intelligences could be simulated or even reproduced "in silicon", i.e. by using computers.

Nowadays is an extremely broad term that covers a lot of computational methodologies. LLM in particular are a evolution of methods born to simulate and act as human neural network. Nowadays they work very differently, but they still provide great insights on how an "artificial" intellicenge can be built. It is only one small corner of what will be a real general artificial intelligence, and a small step in that direction.

AI as a name is absolutely unrelated with how programs based on the methodologies are built.

Human intelligences are in charge of all copyright part. AI and copyright are orthogonal, people are those who cannot tell the 2 and keep talking about AI.

There is AI, and there is copyright, it is time for all of us to properly frame the discussion on "copyright discussion related to 's product"

[–] assassin_aragorn 3 points 1 year ago (3 children)

What I'm getting at moreso is that comparisons to humans for purposes of copyright law (e.g. likening it to students learning in school or reading library books) don't hold water just because it's called an AI. I don't see that as an actual defence for these companies, and it seems to be somewhat prevalent.

load more comments (3 replies)
[–] BURN 12 points 1 year ago (1 children)

AI has been a blanket term for Machine Learning, LLMs, Decision Trees and every other form of “intelligence”.

Unfortunately I think that genie is out of the bottle and it’s never going back in.

load more comments (1 replies)
[–] [email protected] 11 points 1 year ago

Both of those statements are reasonable. You shouldn't have to pay to utilize anything you scrape from the internet, so long as you don't violate copyright by redistributing it

[–] [email protected] 8 points 1 year ago (2 children)

I use to tune my LLM model

Large Language Model model

[–] [email protected] 7 points 1 year ago (3 children)

Automated Teller Machine machine.

[–] dragontamer 6 points 1 year ago

Chai Tea? Chai means tea bro. Do you want coffee coffee with your cream cream?

load more comments (2 replies)
[–] Geek_King 4 points 1 year ago* (last edited 1 year ago) (1 children)

Automated Teller Machine Machine, Personal Identification Number Number, Network Interface Card Card

This has been a problem for as long as acronyms have existed (and yes it bothers me too).

load more comments (1 replies)
[–] [email protected] 6 points 1 year ago* (last edited 1 year ago)

They are 100% AI. It's an umbrella term. Simple pathing algorithms in games are also AI.

[–] [email protected] 3 points 1 year ago

Honestly, I see 0 difference. I think you are suggesting that somehow it is more logical to give information to AI for free sounds more reasonable than to LLM (which is absolutely AI). I see no reason at all to believe so. Maybe you can elaborate?

[–] [email protected] 3 points 1 year ago

I think this was always the case even before these models took off. LLM is more correct but it was always jsut called AI.

load more comments
view more: next ›