Machine Learning

478 readers

1 users here now

A community for posting things related to machine learning

Icon base by Lorc under CC BY 3.0 with modifications to add a gradient

founded 1 year ago

MODERATORS

[email protected]

Extending Context Window of Large Language Models via Positional Interpolation (arxiv.org)

submitted 1 year ago by [email protected] to c/[email protected]

0 comments fedilink hide all child comments

Paper released by Meta a few days ago, detailing a method for extending the context or "memory" of an LLM up to 32k tokens. What is interesting is that they give a mention to: https://kaiokendev.github.io/

This is a blog post written by a guy in his spare time who came up with the same method simultaneously, he calls it SuperHOT.

It's really exciting how AI/ Machine learning can be advanced by relatively ordinary people putting in hard work without the resources of Microsoft etc.

no comments (yet)

sorted by: hot top controversial new old

there doesn't seem to be anything here