ChatGPT

8946 readers

1 users here now

Unofficial ChatGPT community to discuss anything ChatGPT

founded 1 year ago

MODERATORS

marcar

132

ChatGPT would have been so much useful and trustworthy if it is able to accept that it doesn't know an answer. (programming.dev)

submitted 4 months ago* (last edited 4 months ago) by [email protected] to c/chatgpt

46 comments fedilink hide all child comments

Small rant : Basically, the title. Instead of answering every question, if it instead said it doesn't know the answer, it would have been trustworthy.

you are viewing a single comment's thread
view the rest of the comments

[–] kromem 3 points 4 months ago (1 children)

Part of the problem is fine tuning is very shallow, and that a contributing issue for claiming to be right when it isn't is the pretraining on a bunch of training data of people online claiming to be right when they aren't.

[–] [email protected] 1 points 4 months ago (2 children)

Yeah. It is fairly weird to me that it’s such a common thing to do to take the raw output of the LLM and send that to the user, and to try use fine-tuning to get that raw output to look some way that you want.

To me it is obvious that something like having the LLM emit a little JSON block which includes some field which covers “how sure are you that this is actually true” or something, is more flexible and simpler and cheaper and works better.

But what do I know

[–] Cosmicomical 1 points 4 months ago* (last edited 4 months ago) (1 children)

Good look getting it to reply consistently with a json object

Edit: maybe i'm shit at prompting but for me it's almost impossible to even get it to just shut up and consistently reply yes or no to my questions

[–] [email protected] 1 points 4 months ago

I haven’t really had a problem with it… maybe like 5% of the time it will want to do something a little bit weird like wrapping it in ``` but in general it seems like it works well enough to be able to parse with a program and just retry if it does something weird.

You do have to set it up a little carefully, I guess - like usually I’ll give it an example of what I want it to emit, and that’ll be good enough that that’s the form it will follow when it’s emitting stuff back to me. But yeah if you give it prompting and a specific machine readable thing to give back that seems like it usually works better than sticking with English and hoping it goes “yes” or “no” or etc like that.

[–] kromem 1 points 4 months ago

The problem is that they are prone to making up why they are correct too.

There's various techniques to try and identify and correct hallucinations, but they all increase the cost and none are a silver bullet.

But the rate at which it occurs decreased with the jump in pretrained models, and will likely decrease further with the next jump too.