AI

4006 readers

1 users here now

Artificial intelligence (AI) is intelligence demonstrated by machines, unlike the natural intelligence displayed by humans and animals, which involves consciousness and emotionality. The distinction between the former and the latter categories is often revealed by the acronym chosen.

founded 3 years ago

Do I understand LLMs? (self.artificial_intel)

submitted 3 months ago by GardenVarietyAnxiety to c/[email protected]

8 comments fedilink hide all child comments

Basically, it's a calculator that can take letters, numbers, words, sentences, and so on as input.

And produce a mathematically "correct" sounding output, defined by language patterns in the training data.

This core concept is in most if not all "AI" models, not just LLMs, I think.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 4 points 3 months ago

mathematically “correct” sounding output

It's hard to say because that's a rather ambiguous way of describing it ("correct" could mean anything), but it is a valid way of describing its mechanisms.

"Correct" in the context of LLMs would be a token that is likely to follow the preceding sequence of tokens. In fact, it computes a probability for every possible token, then takes a random sample according to that distribution* to choose the next token, and it repeats that until some termination condition. This is what we call maximum likelihood estimation (MLE) in machine learning (ML). We're learning a distribution that makes the training data as likely as possible. MLE is indeed the basis of a lot of ML, but not all.

*~Oversimplification.~