this post was submitted on 15 Aug 2023
158 points (88.3% liked)

Technology

61484 readers
4500 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago
MODERATORS
L3s
enu
technopagan
L4s
158
ChatGPT has a style over substance trick that seems to dupe people into thinking it's smart, researchers found (www.businessinsider.com)
submitted 1 year ago by L4s to c/technology
23 comments fedilink hide all child comments
 

ChatGPT has a style over substance trick that seems to dupe people into thinking it's smart, researchers found::Developers often prefer ChatGPT's responses about code to those submitted by humans, despite the bot frequently being wrong, researchers found.

you are viewing a single comment's thread
view the rest of the comments
[–] joe 89 points 1 year ago (3 children)

A caveat: This user analysis involved just 12 programmers being asked to assess if they prefer the responses of ChatGPT or those written by humans on Stack Overflow to 2,000 randomly sampled questions.

Nothing to see here.

[–] abhibeckert 19 points 1 year ago (1 children)

What they should have done is asked those same 12 programmers to ask a common everyday question on Stack Overflow and then while waiting for a response, ask ChatGPT the same question.

I'd bet 50 bucks almost all of them would get an acceptable answer to their question out of ChatGPT 4 in far less time than it takes the moderators at Stack Overflow to delete the question. I can't imagine any of the questions will actually be answered on SO.

[–] joe 7 points 1 year ago

Right. The problem with SO is that you don't actually get to ask any questions; so reason would suggest anything is at least as good as SO-- even asking a house plant, or Siri, or whatever. Something that actually answers your question would obviously be a better option.

Stack Overflow brought their irrelevance on themselves, I suspect.

[–] [email protected] 13 points 1 year ago

12 programmers? That ChatGPT sample size is way too small to be meaningful IMO