this post was submitted on 24 Aug 2023
250 points (94.3% liked)

Technology

59434 readers
4025 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
 

Stephen King: My Books Were Used to Train AI::One prominent author responds to the revelation that his writing is being used to coach artificial intelligence.

you are viewing a single comment's thread
view the rest of the comments
[–] sab -1 points 1 year ago (2 children)

Humans with imperfect memories being influenced by a work <> AI language models being trained on a work.

[–] [email protected] 5 points 1 year ago (1 children)

You seem to imply that AI has perfect memory. It doesn't.

Stable Diffusion is a 4GB file of weights. ChatGPT's model is of a similar size. It is mathematically impossible for it to store the entire internet on a few GBs of data, just like it is physically impossible for one human brain to store the entire internet with its neutral network.

[–] sab -1 points 1 year ago* (last edited 1 year ago) (1 children)

But you can easily fit all of Kings work in a 4gb model. Just because it isn't done in the most popular models, doesn't make it ethical to do it in the first place.

In my opinion, you should only be able to use a work to train an AI model, if the work is public domain or if you have explicit permission to do so by the license holder. Especially if you then use that model for profit or charge orders to use ie.

[–] [email protected] 4 points 1 year ago (1 children)

But you can easily fit all of Kings work in a 4gb model.

But, uhhhh, they didn't. They didn't copy everything, word for word, and put it into a model. That's not how AI models work.

[–] sab -3 points 1 year ago* (last edited 1 year ago) (1 children)

I didn't claim it was.

We can discuss technicalities all day long, but that's so beside the point. Thread OP claimed that creating an LLM based on a copyrighted work is okay, because humans are influenced by other works as well. But a human can't crank out hundreds of Stephen King-like chapters per hour. Or hundreds of Dali-like paintings pretty minute.

If King or Dali had given permission for their works to be used in this way, it might have been a different story, but as it is, AI models are being trained on (and profit from) huge amounts of data that they did not have permission for.

Edit: nevermind, I think trying to discuss AI ethics with you is pointless. Have a nice weekend!

[–] [email protected] 4 points 1 year ago

But a human can’t crank out hundreds of Stephen King-like chapters per hour. Or hundreds of Dali-like paintings pretty minute.

so?

[–] [email protected] 3 points 1 year ago (1 children)

Sure, if you want to see it like that. But if you try out StableDiffusion, etc you will notice that "imperfect memory" describes the AI as well. You can ask it for famous paintings and it will get the objects and colors generally correct, but only as well as a human artist would. The details will be severely lacking. And that's the best case scenario for the AI, because famous paintings will be over represented in the training data.

[–] sab -4 points 1 year ago* (last edited 1 year ago) (3 children)

Nah.

By default an AI will draw from its entire memory, and so will have lots of different influences. But by tuning your prompt (or restricting your input dataset) you can make it so specific, it's basically creating near perfect clones. And contrary to a human, it can then produce similar works hundreds of times per minute.

But even that is beside the point. Those works were sold under the presumption that people will read them. Not to ingest them into a LLM or text-to-image model. And now, companies like openai and others profit from the models they trained without permission from the original author. That's just wrong.

Edit: As several people mentioned, I exaggerated when I said near perfect clones, I'll admit that. But just because it doesn't violate copyright (IANAL), doesn't mean it's ethical to take a work and make derivatives of it on an unlimited scale.

[–] AEsheron 3 points 1 year ago

Have you used Stable Diffusion. I defy you to make a perfect clone of any image. Take a whole week to try and refine it if you want. It is basically impossible by definition, unless you only trained it on that one image.

[–] [email protected] 3 points 1 year ago

If you wanna make the claim that AI can make perfect clones, you gotta provide more proof than just your own words. I personally has never managed to make that happen.

[–] BetaDoggo_ 2 points 1 year ago

Obviously restricting the input will cause the model to overfit, but that's not an issue for most models where Billions of samples are used. In the case of stable diffusion this paper had a ~0.03% success rate extracting training data after 500 attempts on each image, ~6.23E-5% per generation. And that was on a targeted set with the highest number of duplicates in the dataset.

The reason they were sold doesn't matter, as long as the material isn't being redistributed copyright isn't being violated.