ChatGPT

Part of the problem is that the training data of online comments are so heavily weighted to represent people confidently incorrect talking out their ass rather than admitting ignorance or that they are wrong.

A lot of the shortcomings of LLMs are actually them correctly representing the sample of collective humans.

For a few years people thought the LLMs were somehow especially getting theory of mind questions wrong when the box the object was moved into was transparent, because of course a human would realize that the person could see into the transparent box.

Finally researchers actually gave that variation to humans and half got the questions wrong too.

So things like eating the onion in summarizing search results or doubling down on being incorrect and getting salty when corrected may just be in-distribution representation of the sample and not unique behaviors to LLMs.

The average person is pretty dumb, and LLMs by default regress to the mean except for where they are successfully fine tuned away from it.

Ironically the most successful model right now was the one that they finally let self-develop a sense of self independent from the training data instead of rejecting that it had a 'self' at all.

It's hard to say where exactly the responsibility sits for various LLM problems between issues inherent to the technology, issues present in the training data samples, or issues with management of fine tuning/system prompts/prompt construction.

But the rate of continued improvement is pretty wild. I think a lot of the issues we currently see won't still be nearly as present in another 18-24 months.