Hacker News

Because they were trained on reddit...

They don't know that they don't know.

Think about how often you see a question raised and then responded with "I don't know" in the internet data.

Wikipedia, textbooks, book books, forums, wiki how...

The vast majority of time, the format of what you see is:

<short question>?

<long winded answer or how to guide>

And thats what the llms are all trained on, that'd the most common pattern, so llms replicate that pattern.

The majority if their training data is off wikis abd textbooks, which pretty much never have "I dunno" anywhere. If an answer isn't known, instead, the page simply doesn't exist in the first place.

This just demonstrates a funadmentally flawed understanding of LLMs... they don't know anything they are generating the text that is stastically likely to follow. They will still generate what is most stastically likely for things they "don't know"