this post was submitted on 11 Aug 2023
39 points (91.5% liked)
BecomeMe
817 readers
2 users here now
Social Experiment. Become Me. What I see, you see.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Title feels misleading, it gets stack overflow questions wrong 52% of the time
However it got 77% of easy Leetcode questions correct. Also I believe that's first try, which is not generally how chatgpt should be used.
Also also, you should probably be using a coding specific model if you want good coding results
Probably because the model has seen thousands of possible solutions to those exact Leetcode problems. Actual questions people ask on StackOverflow tend to be much more specialized.