638
Remember how ChatGPT totally aced the bar exam? Wow! yeah, turns out that was just a lie
(www.nytimes.com)
this post was submitted on 15 May 2024
638 points (98.9% liked)
TechTakes
1435 readers
145 users here now
Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.
This is not debate club. Unless it’s amusing debate.
For actually-good tech, you want our NotAwfulTech community
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I'm not even going to engage in this thread cause it's a tar pit, but I do think I have the appropriate analogy.
When taking certain exams in my CS programme you were allowed to have notes but with two restrictions:
The idea was that you needed to actually put a lot of work into making it, since the entire material was obviously the size of a fucking book and not an A4 page, and you couldn't just print/copy it from somewhere. So you really needed to distill the information and make a thought map or an index for yourself.
Compare that to an ML model that is allowed to train on data however long it wants, as long as the result is a fixed-dimension matrix with parameters that helps it answer questions with high reliability.
It's not the same as an open book, but it's definitely not closed book either. And the LLMs have billions of parameters in the matrix, literal gigabytes of data on their notes. The entire text of War and Peace is ~3MB for comparison. An LLM is a library of trained notes.
My question to you is how is it different than a human in this regard? I would go to class, study the material, hope to retain it, so I could then apply that knowledge on the test.
The ai is trained on the data, "hopes" to retain it, so it can apply it on the test. It's not storing the book, so what's the actual difference?
And if you have an answer to that, my follow up would be "what's the effective difference?" If we stick an ai and a human in a closed room and give them a test, why does it matter the intricacies of how they are storing and recalling the data?
holy fuck you're a moron
please go read a book, and look at some art. no, marvel media doesn't count.
Me, about to suggest some actually really good, thought provoking Marvel comics that somehow got made alongside the relentless superhero soap opera: oh wait now isn't the time, we're dunking on the AI bro