this post was submitted on 09 Jan 2025
478 points (99.2% liked)
Opensource
1715 readers
58 users here now
A community for discussion about open source software! Ask questions, share knowledge, share news, or post interesting stuff related to it!
⠀
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Also the incessant ammounts of power/energy that they consume.
Running an llm llocally takes less power than playing a video game.
Source?
Any paper about any neural network.
Using a model to get one output is just a series of multiplications (not even that, we use vector multiplication but yeah), it's less than or equal to rendering ONE frame in 4k games.
I know you are agreeing with me, but it being a "series of multiplications" is not terribly informative, that's basically a given. The question is how many flops, and how efficient are flops.