this post was submitted on 14 Jun 2023
2 points (75.0% liked)

Machine Learning

1788 readers
11 users here now

founded 4 years ago
MODERATORS
[email protected]
2
The Curse of Recursion: Training on Generated Data Makes Models Forget (arxiv.org)
submitted 1 year ago by [email protected] to c/[email protected]
2 comments fedilink hide all child comments
top 2 comments
sorted by: hot top controversial new old
[โ€“] radical_action 3 points 1 year ago (1 children)

Not wholly surprising, and will be interesting to see what is developed to deal with this collpase.

[โ€“] [email protected] 1 points 1 year ago

I wonder whether there even is something to deal with this collapse. It just seems to me that data generated by the model itself will never have any new cases to learn from. I think it will just keep reinforcing the current knowledge already in the model. This is not always a bad thing, we have been doing that(the reinforcing) for so long now under "epochs". But even with epochs you end up overfitting after a while