this post was submitted on 09 Mar 2024
99 points (89.6% liked)

Technology

59693 readers
4953 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago
MODERATORS
L3s
[email protected]
[email protected]
L4s
enu
99
How Quickly Do Large Language Models Learn Unexpected Skills? (nautil.us)
submitted 8 months ago by dominiquec to c/technology
19 comments fedilink hide all child comments
 

So-called "emergent" behavior in LLMs may not be the breakthrough that researchers think.

you are viewing a single comment's thread
view the rest of the comments
[โ€“] [email protected] 52 points 8 months ago (3 children)

TLDR: Let's say you want to teach an LLM a new skill. You give them training data pertaining to that skill. Currently, researchers believe that this skill development shows up suddenly in a breakthrough fashion. They think so because they measure this skill using some methods. The skill levels remain very low until they unpredictably jump up like crazy. This is the "breakthrough".

BUT, the paper that this article references points at flaws in the methods of measuring skills. This paper suggests that breakthrough behavior doesn't really exist and skill development is actually quite predictable.

Also, uhhh I'm not AI (I see that TLDR bot lurking everywhere, which is what made me specify this).

[โ€“] [email protected] 16 points 8 months ago (1 children)

Also, uhhh I'm not AI

An AI would say that... ๐Ÿ˜‚

[โ€“] [email protected] 4 points 8 months ago

Clearly, the AI is learning deception

[โ€“] inspxtr 8 points 8 months ago (2 children)

re: your last point, AFAIK, the TLDR bot is also not AI or LLM; it uses more classical NLP methods for summarization.

[โ€“] [email protected] 3 points 8 months ago

https://github.com/RikudouSage/LemmyAutoTldrBot readme say summarization is in summarizer.py which use sumy, specifically LSA which documented here

[โ€“] dirtySourdough 1 points 8 months ago

Natural language processing falls under AI though, and so do large language models (see chapters 23 and 24 of Russell and Norvig, 2021 http://aima.cs.berkeley.edu/).

[โ€“] [email protected] 7 points 8 months ago (1 children)

Also, uhhh I'm not AI

That's exactly what an AI would say that got an emergent skill to lie

๐Ÿคฅ

[โ€“] [email protected] 4 points 8 months ago

Or a model that picked up on a pattern of sources saying that.